Build a Daily CISA KEV Security Feed

A daily security feed needs reliable state handling more than a complicated crawler. The CISA KEV Scraper retrieves the published catalog and applies bounded filters, while a downstream store can distinguish complete snapshots, new CVEs, and changed guidance.

Preserve a complete baseline

For a full feed, remove keyword, vendor, product, date, and ransomware filters and set a cap above the current catalog size:

{"query":"","sort":"newest","maxResults":2000}

Record the run status, item count, catalog version, release timestamp, and collection time. Validate that the run completed before replacing the prior baseline. If a source or network failure produces an incomplete set, keep the last complete baseline and alert on pipeline health instead of generating false deletion and reappearance events.

Use cve_id as the primary key. Compare required action, due date, known ransomware use, notes, vendor, product, and description between versions. New CVEs deserve an enrichment workflow; modified records deserve a change event with both old and new values. A catalog removal should be reviewed rather than automatically treated as remediation.

Store the complete dataset in a warehouse or object store, then publish smaller views for asset owners, ransomware response, selected vendors, and overdue remediation. The source feed should remain immutable enough to answer what CISA published at a given time.

Do not turn every catalog row directly into a high-priority ticket. Join to assets, validate affected versions, and apply business context. Track unmatched records as inventory-quality signals. This architecture separates source collection from organization-specific decisions and makes the system easier to test when filters, naming mappings, or routing rules change.