How to Scrape World Bank Development Indicators

the official World Bank Indicators API already exposes structured records, so browser scraping would add fragility without adding data. The World Bank Indicators Scraper handles pagination, retries, normalization, deduplication, and Apify dataset output.

Pick a bounded query

The Actor accepts country or region codes, indicator codes, year range, null handling, and maximum observations per series. Start with one narrow query and a small result cap. Inspect the returned rows before increasing volume. A broad default is convenient for a demo, but it is a poor production specification.

{
  "countries": [
    "IND",
    "USA"
  ],
  "indicators": [
    "NY.GDP.MKTP.CD"
  ],
  "yearFrom": 2015,
  "yearTo": 2025
}

Each row contains country code and name, indicator code and label, year, value, unit, decimal precision, source ID, observation status, last-updated date, and canonical indicator URL. The natural key is country_code plus indicator_code plus year. Use it when merging reruns; dataset row order is not an identity.

Export and verify

Run the Actor from the Apify Console, API, or a saved Task. JSON preserves arrays and nulls best. CSV is convenient for a quick spreadsheet review, but inspect how arrays were serialized before importing it into a warehouse.

Open several source links and compare identifiers, dates, and one multi-value field with the published record. Coverage and release lag vary by series and country. Missing is not zero. Preserve null rows when auditing coverage, read the indicator definition, and never combine series until units and methodology are compatible.

A good first run answers two questions: does the query find the intended population, and can every important row be traced back to the publisher? Scale only after both answers are yes.