Build a Vulnerability Intelligence Feed

A production feed needs more than a scheduled scrape. The NVD CVE Scraper handles collection from NIST's official NVD CVE API 2.0; the rest of the pipeline should make ownership and state explicit.

Recommended data flow

Saved Apify Task -> raw dataset -> keyed upsert -> qualification -> owner queue -> alert

Use cve_id for idempotency and keep last_modified for change detection. Enrich with asset owners after collection, then create tickets only when product matching and exposure checks pass.

Keep raw and derived data separate. The raw table should mirror source fields and collection metadata. The derived table can add scores, owners, workflow status, and internal notes. When a scoring rule changes, rebuild the derived table rather than editing source history.

Failure handling

Promote a snapshot only when the Actor run succeeds and passes basic checks: nonzero rows when rows are expected, valid keys, parseable dates, and no unexpected schema loss. Retry network failures with a bounded policy. Do not create downstream deletion events from a failed collection.

CVSS is technical severity, not your business risk. Join results to an asset inventory and consider exposure, exploit status, ownership, and compensating controls before opening remediation tickets.

Track three operational metrics: successful run rate, time from source publication to owner delivery, and the percentage of alerts that owners accept. Those numbers expose different failures. Availability problems hurt the first, an infrequent schedule hurts the second, and weak qualification hurts the third.

Start with one destination and one owner group. Add CRM, warehouse, or webhook branches after the keyed dataset and alert rules have stayed stable through several runs.