Scrape Ashby, Workday, and Recruitee Career Pages

Employer career pages contain the cleanest version of a job: the direct apply link, the employer's wording, and the listing's current availability. The nuisance is that every applicant-tracking system exposes the data differently.

The Career Site Job Scraper handles supported ATS career pages behind one input. It covers common platforms including Lever, Greenhouse, Workday, Ashby, Recruitee, BambooHR, and Keka.

Start with public career-page URLs

Use careerPages mode when you know which employers to track:

{
  "sourceMode": "careerPages",
  "careerPageUrls": [
    "https://jobs.lever.co/solopulseco",
    "https://boards.greenhouse.io/mark43"
  ],
  "maxJobsPerSite": 100,
  "scrapeDescriptions": true
}

Keep the employer-to-URL mapping in your own table. Companies migrate between ATS vendors and sometimes change their career subdomain without redirecting the old page.

Normalize after collection

ATS platforms disagree on department names, location formatting, and employment types. Preserve the raw fields, then add normalized columns for search and reporting.

def job_key(row):
    if row.get("url"):
        return row["url"].split("?")[0].rstrip("/")
    return "|".join(
        str(row.get(k, "")).strip().lower()
        for k in ("company", "title", "location")
    )

Avoid deduplicating on title alone. A company may have the same role in several cities, and remote listings may intentionally repeat by country.

Detect new and removed openings

Save a snapshot of stable job keys for each employer. On the next run, current - previous gives new jobs and previous - current gives removed jobs. Mark removals as no longer observed instead of deleting them immediately; temporary ATS outages do happen.

For hiring-velocity analysis, count openings and closures by employer and week. Do not interpret a one-day drop as a layoff signal without checking the source page. Career-site changes can reflect requisition housekeeping or a platform migration.

Keep failure states visible

Record when each employer was checked and whether extraction succeeded. "No jobs" and "page could not be read" must remain different states. This is especially important in aggregators, where an empty successful response is legitimate for a small company.

The Career Site Job Scraper page has field details and ready-to-run examples. Begin with two employers from different ATS platforms, inspect the normalized output, then expand your watchlist.