Build an arXiv Research Dataset

Define the research scope before collection: categories, title or abstract phrases, exclusions, dates implied by the sort window, and maximum results. Run each expression separately with the arXiv Papers Scraper and attach a query label to every row.

Preserve an immutable source table. Build derived paper, author, and category tables from it. Use the base arXiv ID for paper-level counts while retaining version, published date, and updated date. Do not infer peer-review status from a DOI or journal-reference field alone.

When queries overlap, report both unique papers and query matches. The overlap itself can be analytically useful, but only if it is not hidden by premature deduplication.