Measure Local Business Digital Presence by City and Vertical

Thirdwatch's Google Maps Lead Enrichment actor produces the raw material for a defensible digital-presence statistic: one row per business with fetch_status (did the site respond at all), email_is_freemail (own domain versus consumer mailbox), email_domain_mx_ok (does the domain actually receive mail) and social_profiles_count (how many channels the business runs). Analysts and policy researchers use it to compare cities and trades on observed digital maturity rather than on self-reported survey answers.

Why measure local business digital presence from observed data

Almost every published figure on small-business digital adoption comes from a survey. Someone asks a sample of firms whether they have a website, whether they use email, whether they are on social media, and the answers become a policy indicator. The EU's Digital Decade programme, for example, sets a target that at least 90 percent of SMEs reach a basic level of digital intensity by 2030, tracked through the Digital Economy and Society Index and its underlying Eurostat ICT usage survey. It is a serious instrument, and it is still a questionnaire: annual, nationally aggregated, and dependent on a respondent's own account of their own firm.

Observed measurement answers different questions. A regional development agency deciding where to place a digital-skills programme does not need a national average; it needs to know whether the plumbers in one metro are measurably further behind the plumbers in another. An academic studying service-sector formalisation wants a variable that does not depend on who answered the phone. A market analyst sizing a vertical SaaS opportunity wants to know what share of a trade runs its own mail domain versus a personal Gmail, because that ratio predicts almost everything downstream.

All three need the same shape: a defined cohort of businesses in a city and trade, an attempt to reach each one's website, and a recorded outcome for every attempt — including the failures. The failures are the measurement. A row with fetch_status: dns_error is not a gap in your dataset; it is a business whose web presence has stopped working, and it belongs in the denominator.

How does this compare to the alternatives?

Four routes exist to a digital-presence number, and they trade off timeliness against granularity in fairly predictable ways.

The DIY route is the one that quietly ruins studies. If your own crawler's success rate changes between waves because you fixed a timeout or added a retry, your "digital presence declined in Q3" finding is an artefact of your instrument. A fixed extraction pipeline with a documented failure taxonomy is worth more here than raw flexibility. The Google Maps Lead Enrichment actor page lists the full field set, and the business data scraping guide covers the broader pattern of turning listings into structured records.

How to measure local business digital presence in 5 steps

Step 1: How do I define a comparable cohort across cities and trades?

Build the cohort from the query text itself, because source_query is what you will group by later. Write one query per city-by-vertical cell, keep the trade wording identical across cities, and vary only the location.

{
  "queries": [
    "plumbers in Austin TX",
    "plumbers in Columbus OH",
    "plumbers in Portland OR",
    "dentists in Austin TX",
    "dentists in Columbus OH",
    "dentists in Portland OR"
  ],
  "maxResults": 600,
  "maxPagesPerSite": 4,
  "concurrency": 8,
  "verifyEmailDomains": true,
  "onlyWithContact": false,
  "retryBlockedWithProxy": false
}

Identical trade wording matters more than it looks. "plumbers" and "plumbing services" resolve to overlapping but differently ranked sets, and a wording change between cities becomes an unmeasured confound. Note also that a single query saturates near 120 businesses, so maxResults: 600 across six cells is a realistic ceiling rather than an ambition. For larger territories, split by suburb and treat each suburb as its own cell.

Step 2: How do I keep every failure in the denominator?

Leave onlyWithContact at false and never turn it on for a measurement run. That single flag is the difference between a rate and a conditional rate.

from apify_client import ApifyClient

client = ApifyClient("apify_api_xxxxxxxxxxxxxxxx")

run = client.actor("thirdwatch/google-maps-lead-enrichment").call(run_input={
    "queries": cohort_queries,        # one per city x vertical cell
    "maxResults": 600,
    "maxPagesPerSite": 4,             # hold constant across every wave
    "verifyEmailDomains": True,
    "onlyWithContact": False,         # failures are the measurement
    "retryBlockedWithProxy": False,   # hold constant too
})

rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(len(rows), "businesses across", len({r["source_query"] for r in rows}), "cells")

retryBlockedWithProxy deserves the same discipline. Switching it on lifts reachability by roughly six percentage points, which is useful for outreach and fatal for comparison if one wave has it on and another has it off. Pick a setting, record it in your methodology, and leave it alone.

Step 3: How do I compute the presence indicators?

Three indicators come straight off the rows. Reachability uses fetch_status over all businesses; the maturity and channel measures use the reachable subset.

import pandas as pd

df = pd.DataFrame(rows)
df["city"] = df["source_query"].str.split(" in ").str[-1]
df["vertical"] = df["source_query"].str.split(" in ").str[0]

df["reachable"] = df["fetch_status"].eq("ok")
df["has_site"] = ~df["fetch_status"].isin(["no_website", "social_website"])

reach = df[df["has_site"]]
ok = df[df["reachable"]]
ok = ok.assign(
    own_domain_email=ok["primary_email"].notna() & ~ok["email_is_freemail"].fillna(True),
    live_mail_domain=ok["email_domain_mx_ok"].fillna(False),
    multi_channel=ok["social_profiles_count"].fillna(0) >= 3,
)

panel = pd.DataFrame({
    "n_listed":       df.groupby(["city", "vertical"]).size(),
    "site_listed":    df.groupby(["city", "vertical"])["has_site"].mean(),
    "reachable":      reach.groupby(["city", "vertical"])["reachable"].mean(),
    "own_domain":     ok.groupby(["city", "vertical"])["own_domain_email"].mean(),
    "mail_live":      ok.groupby(["city", "vertical"])["live_mail_domain"].mean(),
    "socials_mean":   ok.groupby(["city", "vertical"])["social_profiles_count"].mean(),
    "multi_channel":  ok.groupby(["city", "vertical"])["multi_channel"].mean(),
}).round(3)

print(panel)

has_site separates two genuinely different states that a naive count merges: a business with no website at all (no_website) and one whose "website" is a Facebook page (social_website). The second is a real digital-presence finding — the firm is online, just not on infrastructure it controls — and reporting it as a crawl failure would be wrong.

Step 4: How do I attach confidence intervals before comparing cells?

Every indicator here is a proportion over a modest sample, so publish intervals rather than point estimates. The Wilson score interval behaves properly near 0 and 1, which matters because reachability sits around 0.9.

from statsmodels.stats.proportion import proportion_confint, proportions_ztest

def wilson(series):
    k, n = int(series.sum()), int(series.count())
    lo, hi = proportion_confint(k, n, alpha=0.05, method="wilson")
    return pd.Series({"rate": k / n, "n": n, "lo": round(lo, 3), "hi": round(hi, 3)})

by_cell = ok.groupby(["city", "vertical"])["own_domain_email"].apply(wilson).unstack()
print(by_cell)

a = ok[(ok.city == "Austin TX") & (ok.vertical == "plumbers")]["own_domain_email"]
b = ok[(ok.city == "Portland OR") & (ok.vertical == "plumbers")]["own_domain_email"]
stat, p = proportions_ztest([a.sum(), b.sum()], [len(a), len(b)])
print(f"Austin {a.mean():.2f} vs Portland {b.mean():.2f}  p={p:.3f}")

At a hundred businesses per cell, a 70 percent rate carries an interval of roughly plus or minus nine points. That is wide enough to separate a strong cohort from a weak one and far too wide to rank adjacent cities. Compare verticals within a city and cities within a vertical; do not build a league table out of cells that overlap.

Step 5: How do I re-measure the same market over time?

Anchor the panel on place_id, which is stable across runs, and hold every instrument setting fixed between waves.

wave = pd.DataFrame(rows).set_index("place_id")
prior = pd.read_parquet("presence_wave_2026Q2.parquet").set_index("place_id")

joined = prior.join(wave, how="inner", lsuffix="_t0", rsuffix="_t1")
churn = pd.DataFrame({
    "went_dark":  (joined.fetch_status_t0.eq("ok") & ~joined.fetch_status_t1.eq("ok")).mean(),
    "came_online": (~joined.fetch_status_t0.eq("ok") & joined.fetch_status_t1.eq("ok")).mean(),
    "professionalised": (joined.email_is_freemail_t0.fillna(True)
                        & ~joined.email_is_freemail_t1.fillna(True)).mean(),
}, index=["rate"]).round(3)
print(churn)

wave.to_parquet("presence_wave_2026Q3.parquet")

The matched subset is what carries the longitudinal claim. Businesses that appear in one wave and not the other reflect Google's ranking as much as real entry and exit, so report the matched panel as your headline and the unmatched share as a caveat.

Sample output

Each row is one business, and the diagnostic block at the bottom is what turns it into a measurement rather than a contact. fetch_status is the outcome of the attempt; email_is_freemail distinguishes an own-domain mailbox from a consumer one; email_domain_mx_ok confirms the domain can actually receive mail; social_profiles_count counts distinct networks. The three rows below show the three states you need to keep apart: a fully present business, a reachable business with no email at all, and a listing whose site no longer resolves.

[
  {
    "source_query": "plumbers in Austin TX",
    "business_name": "Austin Plumbing",
    "business_category": "Plumber",
    "business_address": "5115 N Lamar Blvd Ste 100, Austin, TX 78751",
    "business_rating": 4.8,
    "place_id": "ChIJq0fCfwkpW4YR4EqPvo68RGQ",
    "website": "https://austinplumbing.com/",
    "final_url": "https://www.austinplumbing.com/",
    "primary_email": "service@austinplumbing.com",
    "email_domain": "austinplumbing.com",
    "email_is_freemail": false,
    "email_domain_mx_ok": true,
    "email_mx_provider": "microsoft365",
    "emails_count": 1,
    "primary_phone": "+15129004663",
    "phones_count": 3,
    "social_profiles_count": 7,
    "has_contact": true,
    "fetch_status": "ok",
    "http_status": 200,
    "http_server_header": "nginx",
    "pages_fetched": 4,
    "scraped_at": "2026-07-27T22:51:47+00:00"
  },
  {
    "source_query": "plumbers in Columbus OH",
    "business_name": "Northside Drain Co",
    "business_category": "Plumber",
    "business_rating": 4.4,
    "place_id": "ChIJl8Xb2p6OQIgRxV1p3nQdY4A",
    "website": "http://northsidedrain.net/",
    "final_url": "http://northsidedrain.net/",
    "primary_email": null,
    "email_domain": null,
    "email_is_freemail": null,
    "email_domain_mx_ok": null,
    "emails_count": 0,
    "primary_phone": "+16145550188",
    "phones_count": 2,
    "facebook_url": "https://www.facebook.com/northsidedrain",
    "social_profiles_count": 1,
    "has_contact": true,
    "fetch_status": "ok",
    "http_status": 200,
    "http_server_header": "LiteSpeed",
    "pages_fetched": 3,
    "scraped_at": "2026-07-27T22:52:03+00:00"
  },
  {
    "source_query": "dentists in Portland OR",
    "business_name": "Rosewood Dental Studio",
    "business_category": "Dentist",
    "business_rating": 4.9,
    "place_id": "ChIJqcOc4b6glVQRcYbPh1nQ5rM",
    "website": "https://rosewooddentalpdx.com/",
    "primary_email": null,
    "emails_count": 0,
    "primary_phone": "+15035550142",
    "social_profiles_count": 0,
    "has_contact": true,
    "fetch_status": "dns_error",
    "http_status": null,
    "pages_fetched": 0,
    "scraped_at": "2026-07-27T22:52:11+00:00"
  }
]

The second row is the one analysts most often mishandle. It is reachable, it has a phone and a Facebook page, and it has no email anywhere across four pages — a genuine measurement of a business that has chosen click-to-call over published email. Counting it as a failure understates presence; excluding it overstates email adoption.

Common pitfalls

Coverage is the first honest caveat: Google Maps is a ranked sample, not a registry, and a query saturates near 120 businesses. Your cohort is "visible operators in this trade and city", and the methodology section should say exactly that. Second, hit rates vary enormously by vertical — US home services land around 55-60 percent email while wellness and design businesses abroad reach 85-90 percent — so a cross-vertical comparison measures the trade's publishing conventions at least as much as its digital maturity. Compare like with like.

Third, roughly 5-11 percent of sites are unreachable in any wave, and around 5 percent of the email domains found do not resolve at all because they are template placeholders; email_domain_mx_ok is what separates those from live mailboxes. Fourth, validation is domain-level, not mailbox-level: the field proves the domain accepts mail, never that info@ exists. Fifth, roughly one business in four yields no email even on a clean crawl, which is a finding rather than a defect.

The instrument settings are yours to freeze. maxPagesPerSite is capped at 4 by design — deeper crawls recovered only two of eighteen missing emails in testing while tripling bandwidth — so fix it at 4, fix retryBlockedWithProxy, and change neither between waves. The Actor keeps the rest stable for you: a consistent link-ranking order, obfuscated-email decoding, a payload-size check that catches empty shell pages returning HTTP 200, and an explicit fetch_status taxonomy so every failure is classified rather than silently dropped.

Related use cases

• Build a city-by-city local lead list from Google Maps — the same query-per-cell structure, tuned for non-overlapping coverage instead of measurement.
• Diagnose missing emails in a scraped local lead list — the engineering view of fetch_status, when a slice is worth re-running and when it is permanently dead.
• Segment leads by email provider: Workspace vs Microsoft 365 — turns email_mx_provider into cohorts, a useful second maturity axis alongside freemail share.
• Cut cost per contactable lead in local prospecting — the run-settings economics if you later convert a research pull into outreach.
• Build a local business database with Google Maps — listing-level coverage before any website enrichment.
• More Thirdwatch guides — the full library of extraction and analysis walkthroughs.