Search Inside a Substack Newsletter by Keyword

Site-search quality on newsletters is inconsistent — the API path is better. Thirdwatch's Substack Scraper reads each publication's own JSON API — titles, authors, dates, engagement, and full article bodies — with no login and no browser.

Skip the setup: Run this as a ready-to-go task on Apify — pre-loaded with the configuration from this guide.

Why search inside a newsletter

Finding 'every post a newsletter wrote about hiring' usually means Google site: queries with flaky coverage. Substack's own search index is the authoritative source, and the Actor exposes it directly.

Set search and the run returns matching posts with full metadata.

The scraper hits Substack's public archive and post endpoints, so an entire newsletter's output is one input away.

How does this compare to the alternatives?

Why this Actor

• Reads the publication's own JSON API — fast, stable, and cheap.
• includeContent upgrades metadata rows to full article text.
• search queries the newsletter's archive index directly.
• Every row carries engagement fields: reactions and comment counts.
• Multiple publications in one run — newsletter monitoring as a batch.

How to do it in 3 steps

Step 1: Configure the input

Set the inputs as shown below — publicationUrls takes the targets, search runs a keyword query inside each publication and includeContent expands rows to full article text.

Step 2: Run the Actor

Run it from the console, the API, or the linked saved task. One dataset row is written per post.

Step 3: Use the output

Each row carries title, subtitle, author, date, reactions, comment count, cover image — and the full body when includeContent is set.

{
  "publicationUrls": [
    "https://newsletter.pragmaticengineer.com"
  ],
  "search": "hiring",
  "maxPosts": 10
}

Each dataset row looks like:

{
  "title": "How to write a useful status update",
  "subtitle": "And why most updates fail",
  "slug": "status-update",
  "url": "https://newsletter.pragmaticengineer.com/p/status-update",
  "author": "Gergely Orosz",
  "postDate": "2025-06-10T08:00:00.000Z",
  "reactions": 412,
  "commentCount": 89,
  "publication": "Pragmatic Engineer",
  "type": "post"
}

What to watch for

Paywalled posts return metadata but not the locked body — expected, since the public API doesn't serve it. Very long archives take a run of proportional length; use maxPosts to bound the pull.

Related use cases

• Snapshot a Substack Publication's Full Archive
• Build a Full-Text Dataset from a Substack Newsletter
• Monitor Several Substack Newsletters in One Run
• Check a Substack's Latest Posts on a Schedule
• All Thirdwatch use-case guides

Run the Substack Scraper on Apify Store — pay per result, free to try, no credit card to test.