Check a Substack's Latest Posts on a Schedule

Five rows a week is all a new-issue monitor needs. Thirdwatch's Substack Scraper reads each publication's own JSON API — titles, authors, dates, engagement, and full article bodies — with no login and no browser.

Skip the setup: Run this as a ready-to-go task on Apify — pre-loaded with the configuration from this guide.

Why monitor new posts

RSS exists, but it doesn't give you engagement numbers or a dataset you can query — and some publications disable feeds. A scheduled small pull of the latest posts gives you a durable, structured monitor.

Set maxPosts to five and attach a schedule — the run is cheap enough to run daily.

The scraper hits Substack's public archive and post endpoints, so an entire newsletter's output is one input away.

How does this compare to the alternatives?

Why this Actor

• Reads the publication's own JSON API — fast, stable, and cheap.
• includeContent upgrades metadata rows to full article text.
• search queries the newsletter's archive index directly.
• Every row carries engagement fields: reactions and comment counts.
• Multiple publications in one run — newsletter monitoring as a batch.

How to do it in 3 steps

Step 1: Configure the input

Set the inputs as shown below — publicationUrls takes the targets, search runs a keyword query inside each publication and includeContent expands rows to full article text.

Step 2: Run the Actor

Run it from the console, the API, or the linked saved task. One dataset row is written per post.

Step 3: Use the output

Each row carries title, subtitle, author, date, reactions, comment count, cover image — and the full body when includeContent is set.

{
  "publicationUrls": [
    "https://newsletter.pragmaticengineer.com"
  ],
  "maxPosts": 5,
  "sort": "new"
}

Each dataset row looks like:

{
  "title": "How to write a useful status update",
  "subtitle": "And why most updates fail",
  "slug": "status-update",
  "url": "https://newsletter.pragmaticengineer.com/p/status-update",
  "author": "Gergely Orosz",
  "postDate": "2025-06-10T08:00:00.000Z",
  "reactions": 412,
  "commentCount": 89,
  "publication": "Pragmatic Engineer",
  "type": "post"
}

What to watch for

Paywalled posts return metadata but not the locked body — expected, since the public API doesn't serve it. Very long archives take a run of proportional length; use maxPosts to bound the pull.

Related use cases

• Snapshot a Substack Publication's Full Archive
• Search Inside a Substack Newsletter by Keyword
• Build a Full-Text Dataset from a Substack Newsletter
• Monitor Several Substack Newsletters in One Run
• All Thirdwatch use-case guides

Run the Substack Scraper on Apify Store — pay per result, free to try, no credit card to test.