About this API
GitHub topic pages list the public repositories that carry a topic, such as web-scraping. This API reads one topic at a time and returns a record for each repository: its name, owner, description, topics and link.
Choose a topic with the topics input, and narrow it to one programming language if you want. Stars, forks and language are on each repository's own page, so they are present only when detail pages are read.
Typical uses
- Find the most-starred projects in a topic.
- Watch which repositories join a topic over time.
- Build a list of open source projects in one language.
What you get
| Field | Type | Sample value |
|---|---|---|
title |
string | creeper |
owner_login |
string | z7r1k3 |
description |
string | Web Crawler and Scraper |
stargazer_count |
string | 12 |
forks_count |
string | 1 |
programming_language |
string | Python |
url |
string | github.com/z7r1k3/creeper |
All 45 fields
Key fields
titleowner_logindescriptionstargazer_countforks_countprogramming_languageurltopics
Technical fields
related_titlerelated_urlslugdetail_descriptionformatted_descriptiontopics_namewatcher_countstargazers_pathwatchers_pathfork_network_pathactivity_pathreport_pathis_orgsponsorsdeploymentspackagesused_bycontributorslanguagesctasuggested_workflowshas_citationis_privateis_templateis_forkis_mirroris_archivedvisibility_labelis_advisory_workspaceis_org_ownedrepository_idrepository_namewatchers_counttyped_forks_counttyped_stargazer_counttyped_watcher_counttyped_watchers_count
Every field with its type, and the schemas, are in contract.json.
Sample records
2 saved records. This is a saved sample, not a live run.
title |
owner_login |
description |
stargazer_count |
forks_count |
programming_language |
|---|---|---|---|---|---|
| web-scraping-google-sheets | — | Guide to Using Google Sheets for Basic Web Scraping | — | — | — |
| creeper | z7r1k3 | Web Crawler and Scraper | 12 | 1 | Python |
Scroll sideways to see more columns.
Inputs
-
topicsstringOptional. Reads the site's listing for this topic. Leave it out for the default listing.
-
programming_languagestringOptional. Only records whose programming language matches.
enrich_records is on by default and adds the fields that only appear on each record's detail page. Send false to read list fields only. See Run an API.
continuation and read_on resume a run that stopped early; see contract.json.
Request
POST https://broker.cleanedweb.com/v1/catalog/public/apis/generated_047a8ebb25911ccf62f4a212/endpoints/generated_047a8ebb25911ccf62f4a212_search/run
curl -X POST https://broker.cleanedweb.com/v1/catalog/public/apis/generated_047a8ebb25911ccf62f4a212/endpoints/generated_047a8ebb25911ccf62f4a212_search/run \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Idempotency-Key: UNIQUE_REQUEST_ID" \
-H "Content-Type: application/json" \
-d '{
"input": {
"topics": "web-scraping"
},
"max_items": 25
}'Save the Idempotency-Key before you send, and reuse it only to retry the same request. The key needs the run scope. See Run a catalog API.
Coverage
- Category
- Public data
- Languages
- English
Limitations
- Stars, forks and language come from each repository page, so they appear only on enriched records.
- Stars and forks are returned as text, such as "12", not as numbers.
Reach. A run reads until the source ends or your max_items, timeout_seconds or budget stops it.
CleanedWeb is not affiliated with GitHub.