How to Scrape Grants.gov Opportunities

Federal grant research becomes repetitive when every team reruns the same portal searches and copies deadlines into separate spreadsheets. The Grants.gov Scraper turns topic and agency searches into structured opportunity rows that can feed a shared pipeline.

Search several funding themes

Start with narrow phrases and a controlled result cap:

{"queries":["artificial intelligence","climate resilience"],"statuses":["posted"],"includeDetails":true,"maxResultsPerQuery":25}

Search results provide the opportunity number, title, agency, dates, status, document type, and assistance listings. Detail mode adds the published synopsis, applicant types, funding instruments and categories, award floor and ceiling when numeric, archive date, and agency contacts.

Use agency codes when a program team follows one funder, such as NSF. Keep the query beside every row so analysts know which pipeline theme produced the match. Deduplicate on opportunity_id; one notice may match several related phrases and should remain one opportunity with several thematic associations downstream.

Forecasted records are useful for planning, but their dates and funding structure can change. Store status and collection time, and require a fresh posted notice before treating a forecast as an application-ready opportunity. Empty award ranges do not imply unlimited funding; they mean the source did not publish a usable numeric value in that record.

Schedule searches daily or weekly depending on the program. Compare opportunity IDs and revision-sensitive fields, then route new rows to an owner for eligibility review. The Actor does not decide whether an organization qualifies or prepare an application. Preserve the official source URL and confirm the current notice before committing staff time or submitting materials.