Compare GDP by Country With World Bank Data

A useful monitor remembers what it saw. The World Bank Indicators Scraper supplies structured snapshots from the official World Bank Indicators API; your workflow decides what counts as a change.

Define the state

Use country, indicator, and year as the stable key; compare values and the source last-updated date to detect revisions. Save the input beside each snapshot so a query change is not mistaken for a source change. Schedule the saved Apify Task at the source's actual update cadence. Hourly polling is wasteful for a feed that changes daily.

On each successful run:

1. Load the previous successful dataset.
2. Join on country_code plus indicator_code plus year.
3. Separate new records, changed records, and records that disappeared.
4. Alert only after applying the business filters.
5. Retain the raw snapshot and run ID for investigation.

Do not replace the previous snapshot after a failed or empty run. An upstream outage otherwise looks like mass deletion.

Keep alerts useful

Store each observation at country:indicator:year granularity. Keep a raw long table, then build a wide analytical view. This prevents a late revision to one indicator from rewriting unrelated source data.

Alert payloads should include the source identifier, the fields that changed, the old and new values, and a direct source URL. That is enough for an owner to verify the event without opening the raw dataset.

Coverage and release lag vary by series and country. Missing is not zero. Preserve null rows when auditing coverage, read the indicator definition, and never combine series until units and methodology are compatible.

Measure alert precision for the first few weeks. If reviewers repeatedly dismiss the same pattern, tighten the source query or the routing rule. Hiding noise after it enters the alert channel is harder than excluding it at collection time.