Monitor UN SDG Data Releases and Revisions

An SDG pipeline that only appends the newest year will miss revisions to earlier observations and metadata. The UN SDG Indicators Scraper returns the fields required to compare official records across snapshots.

Hash the full evidence row

Test the change detector with a revised value, changed footnote, new dimension, removed row, and identical rerun. These fixtures keep a future parser change from generating hundreds of false revisions or, worse, hiding real changes from the analysts responsible for published sustainability claims.

Use series code, area code, time period, and normalized dimensions as the logical key. Hash value, bounds, unit, nature, source, base period, and footnotes. A changed hash then becomes a reviewable revision rather than a silent overwrite.

{"seriesCodes":["SI_POV_DAY1"],"areaCodes":["840","356"],"yearFrom":2015,"yearTo":2026,"maxResultsPerSeries":250}

Store collection time and release context downstream. Classify observations as new, unchanged, revised, or missing from the latest snapshot. Do not delete a historical row because a later run omitted it; first confirm the run, filters, upstream availability, and potential source changes.

Coverage monitoring matters as much as value monitoring. Track the newest year and count of observations by series and area, plus the data-nature mix. Alert when a scorecard would shift to incomparable years or dimensions. Review material changes before refreshing external reports, and keep the previous published edition accessible. This process turns a dynamic official database into an auditable input suitable for long-lived sustainability products.