How to Build a US Rent Comps Dataset for Deal Underwriting

To build a rent comps dataset, run the US Rentals Scraper across your submarket's ZIP codes with source: "both" and includeDetails: true. Each row returns price_min, price_max, square_feet_min, beds_min, year_built and a floorplans array of per-unit rent, bed, bath and area. Explode the floor plans, divide rent by area, and you have a real rent-per-square-foot distribution instead of an average of headline numbers. Built for acquisitions analysts, appraisers and asset managers.

Why rent comps are a data problem, not a search problem

Roughly a third of US households rent, and the Census Bureau's Housing Vacancy Survey tracks that population quarterly at national and regional level. What it does not give you is what a 720-square-foot one-bedroom actually asks three blocks from the asset you are buying. That number only exists on the listing pages, and it changes weekly.

The way most teams answer it does not survive review. Someone opens six browser tabs, writes down the headline rent for each competing property, and averages them. That number is wrong in a specific and predictable way: the headline rent for a 300-unit community is the cheapest studio, not the average unit. Comparing it to your subject property's two-bedroom is comparing the floor of one distribution to the middle of another.

The fix is unit-level data. A community's listing page publishes a floor-plan table — plan name, beds, baths, square feet, rent, availability — and that table is the actual comp set. Once you have thirty or sixty of those rows across four or five competing properties, you can compute rent per square foot by bedroom count, look at the spread rather than the mean, and defend the assumption in an investment committee memo.

How does this compare to the alternatives?

Rent comp data has three sources, and they answer different questions. A subscription market-data platform gives you modelled submarket averages, which is right for a market study and wrong for a single asset. Your own broker's rent survey is unit-level but covers whichever properties the broker called this month. A listings pull is unit-level, covers everything advertised, and is repeatable on a schedule — which matters because a comp set assembled in March is stale by June.

The US Rentals Scraper covers Apartments.com and Realtor.com. If your comp set needs Zillow inventory alongside it, the Zillow Suite Scraper uses the same input shape and lands in the same kind of table.

How to build a rent comps dataset in 5 steps

How do I draw the boundary of the comp set?

Draw it geographically first and filter second. Pass the ZIP codes that cover your submarket as separate entries in queries — one entry per ZIP, not one entry for the metro. Both sites cap how deep a single search paginates, so a query for "Austin, TX" returns the first slice of a very large result set rather than the neighbourhood you care about.

SUBMARKET_ZIPS = ["78704", "78745", "78741", "78702"]

Do not narrow the bedroom filter yet. You want the full unit mix in the pull so you can compute the mix-adjusted numbers later; filtering to two-bedrooms at the source throws away the denominator.

How do I pull both sources in one run?

Set source: "both" and the Actor runs each query against Apartments.com and Realtor.com, merges the results and stamps a source field on every row. Set includeDetails: true — this is the step that populates square_feet, baths, year_built and the floorplans array, none of which appear on an Apartments.com search card.

import os, requests, pandas as pd

ACTOR = "thirdwatch~us-rentals-scraper"
TOKEN = os.environ["APIFY_TOKEN"]

resp = requests.post(
    f"https://api.apify.com/v2/acts/{ACTOR}/run-sync-get-dataset-items",
    params={"token": TOKEN},
    json={
        "queries": SUBMARKET_ZIPS,
        "source": "both",
        "maxResults": 120,
        "includeDetails": True,
        "maxDetailPages": 60,
        "minBeds": 0,
        "maxBeds": 3,
        "propertyTypes": ["apartments", "condos", "townhomes"],
        "proxyConfiguration": {"useApifyProxy": True, "apifyProxyCountry": "US"},
    },
    timeout=3600,
)
listings = pd.DataFrame(resp.json())
print(f"{len(listings)} listings across {listings['source_query'].nunique()} ZIP codes")

maxResults is per query per source, so 120 with source: "both" asks for up to 240 rows per ZIP. maxDetailPages caps how many of those get the detail-page visit, which is the expensive part — set it to the number of comps you actually intend to read.

How do I explode community floor plans into unit-level comps?

A multi-unit community comes back as one row with price, beds, baths and square_feet all null by design, because a community has a range rather than a single value. The unit-level detail lives in floorplans, an array of {name, beds, baths, square_feet, price, price_min, price_max, availability}. Flatten it and each plan becomes its own comp.

rows = []
for _, listing in listings.iterrows():
    plans = listing.get("floorplans") or []
    if plans:
        for plan in plans:
            rows.append({
                "listing_id": listing["listing_id"],
                "source": listing["source"],
                "listing_name": listing["listing_name"],
                "zip_code": listing["zip_code"],
                "year_built": listing.get("year_built"),
                "plan": plan.get("name"),
                "beds": plan.get("beds"),
                "baths": plan.get("baths"),
                "square_feet": plan.get("square_feet"),
                "rent": plan.get("price") or plan.get("price_min"),
            })
    else:
        rows.append({
            "listing_id": listing["listing_id"],
            "source": listing["source"],
            "listing_name": listing["listing_name"],
            "zip_code": listing["zip_code"],
            "year_built": listing.get("year_built"),
            "plan": None,
            "beds": listing.get("beds") or listing.get("beds_min"),
            "baths": listing.get("baths") or listing.get("baths_min"),
            "square_feet": listing.get("square_feet") or listing.get("square_feet_min"),
            "rent": listing.get("price") or listing.get("price_min"),
        })

comps = pd.DataFrame(rows).dropna(subset=["rent", "square_feet", "beds"])

Single-family and condo rentals from Realtor.com have no floorplans array, because the whole property is one unit — the else branch catches those and treats the listing itself as the comp.

How do I normalise to rent per square foot?

Once every row is one unit, the metric is arithmetic. Compute rent per square foot, then look at the distribution by bedroom count rather than the mean, because the mean of a comp set with one lease-up in it is not a number you want to underwrite against.

comps["rent_psf"] = comps["rent"] / comps["square_feet"]

summary = (
    comps.groupby("beds")["rent_psf"]
    .agg(n="count", p25=lambda s: s.quantile(0.25),
         median="median", p75=lambda s: s.quantile(0.75))
    .round(2)
)
print(summary)

subject_sqft = 720
band = comps[(comps["beds"] == 1) & comps["square_feet"].between(650, 800)]
print(f"Subject 1BR indicated rent: "
      f"${band['rent_psf'].median() * subject_sqft:,.0f}/mo "
      f"(n={len(band)})")

Filtering the comp band on square footage before you take the median is what makes the number defensible. A one-bedroom at 550 square feet and one at 900 both say "1BR" and rent per square foot differs by a fifth between them.

How do I keep the comp set fresh?

Re-run the same input on an Apify schedule monthly and append each pull with its scraped_at timestamp. Keying on listing_id, which is stable per site, lets you follow a single plan's asking rent over time — and a plan that has been asking the same rent for four months while sitting available is telling you the ask is above market.

curl -X POST "https://api.apify.com/v2/schedules?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "austin-rent-comps-monthly",
    "cronExpression": "0 6 1 * *",
    "timezone": "America/Chicago",
    "isEnabled": true,
    "actions": [{
      "type": "RUN_ACTOR",
      "actorId": "thirdwatch~us-rentals-scraper",
      "runInput": {
        "queries": ["78704", "78745", "78741", "78702"],
        "source": "both",
        "maxResults": 120,
        "includeDetails": true,
        "maxDetailPages": 60
      }
    }]
  }'

Sample output

Two rows from a comp pull. The first is an Apartments.com community with its floor-plan table attached; the second is a Realtor.com single-family rental where the property itself is the unit.

[
  {
    "listing_id": "s286zb6",
    "source": "apartments",
    "source_query": "78758",
    "listing_url": "https://www.apartments.com/the-watson-austin-tx/s286zb6/",
    "listing_name": "The Watson",
    "address": "11901 Burnet Rd, Austin, TX 78758",
    "city": "Austin", "state": "TX", "zip_code": "78758",
    "latitude": 30.40504, "longitude": -97.71511,
    "price": null,
    "price_min": 1615, "price_max": 4884,
    "price_formatted": "$1,615 - $4,884",
    "currency": "USD",
    "beds_min": 0.0, "beds_max": 2.0,
    "baths_min": 1.0, "baths_max": 2.0,
    "square_feet_min": 425, "square_feet_max": 1403,
    "property_type": "apartment",
    "year_built": 2026,
    "floorplans": [
      {"name": "S1WF", "beds": 0, "baths": 1, "square_feet": 434, "price": 1615},
      {"name": "A2", "beds": 1, "baths": 1, "square_feet": 712, "price": 1889}
    ],
    "specials": "2 Months Free",
    "listing_status": "for_rent",
    "scraped_at": "2026-09-07T22:04:48+00:00"
  },
  {
    "listing_id": "4471203856",
    "source": "realtor",
    "source_query": "78704",
    "listing_name": "2107 Ford St",
    "address": "2107 Ford St, Austin, TX 78704",
    "zip_code": "78704",
    "price": 2950, "price_min": 2950, "price_max": 2950,
    "beds": 2.0, "baths": 1.0, "square_feet": 1080,
    "property_type": "house",
    "broker_name": "Austin Realty Group",
    "list_date": "2026-08-19T00:00:00+00:00",
    "days_on_market": 19,
    "listing_status": "for_rent",
    "scraped_at": "2026-09-07T22:05:02+00:00"
  }
]

Read the two rows differently. The community row has price null and a populated floorplans array — the array is the comp set. The single-unit row has a scalar price, beds, baths and square_feet, so it is already one comp. source tells you which shape to expect, and year_built is the vintage control you use when a 2026 lease-up sits next to 1980s product.

Common pitfalls

Averaging headline rent. price_min for a large community is the cheapest studio. Underwriting a two-bedroom against it understates market rent substantially. Always drop to floorplans when it is populated.

Forgetting that detail mode is what fills the comp fields. With includeDetails off, Apartments.com rows have no square feet, no baths, no year built and no floor plans. That is a site limit, not a setting you can work around — the search card simply does not carry them.

Treating a ZIP query as a hard boundary. A ZIP search on either site can return listings in immediately adjacent ZIPs. Every row carries its true zip_code, so filter on that field rather than trusting source_query.

Ignoring concessions. An asking rent with two months free on a twelve-month lease is an effective rent roughly a sixth lower. The specials field carries the advertised concession text; read it before you call the ask a comp.

Pagination ceilings on whole-city queries. Both sites stop paginating a single search well before a large metro is exhausted. Pass ZIP codes or neighbourhood names as separate queries instead. Thirdwatch's Actor de-duplicates on listing_id within a run, so overlapping areas are safe, and it distinguishes an empty result from a blocked one so a thin comp set never gets mistaken for a soft market.

Related use cases

• Analyse a metro rental market with US listings data — the market-level view above the asset-level comp set
• Track rental days on market as a demand signal — how fast the comps you just pulled are actually leasing
• Track rent concessions and lease-up specials — turn asking rent into effective rent
• Build property management lead lists from Apartments.com — the same pull, read as a contact list
• Screen rental yield with Zillow Rent Zestimate — the for-sale side of the same underwriting question
• Guide to scraping real estate data and all Thirdwatch use-case guides