Expected vs Observed

Expected
vs Observed

A US county health atlas comparing measured outcomes with what demographically similar counties record.

Open the atlas How it was checked
Diabetes mortality 2018-2024: smoothed difference from the model prediction, in mortality-index points (SMR x 100)

96%

median share of county-to-county variation in 40 PLACES measures (crude prevalence) reproduced by demographics and state, on counties the model did not see

Demographics + state
explain most
PLACES variation

Across 40 crude-prevalence measures, county demographics plus state indicators reproduce a median 96.2% of the variation in counties the model did not see. This approximates the inputs to PLACES; it does not rerun CDC's model or show that the underlying health differences are artificial. Uninsurance measured directly by the Census is less predictable, partly because of sampling error. Coverage varies by measure, from 2,299 to 3,143 counties; these results describe counties with available data. The uninsurance comparison uses PLACES ages 18-64 and Census ages 19-64.

Chart data

Once demographics and state are known, PLACES adds little

Across 21 recorded outcomes (deaths, Medicare diagnoses and Census disability), demographics plus state, and Medicare Advantage share for Medicare, predict held-out county values better than PLACES alone. Adding PLACES increases cross-validated R², the share of county variation predicted, by at most 0.8 percentage points in the full available samples shown below and 2.3 percentage points in the restricted samples.

Each comparison includes counties with all required data. The restricted samples require at least 5,000 Original Medicare beneficiaries, 20,000 adults for Census outcomes, or 100 expected deaths for mortality outcomes.

Chart data

What the atlas shows

Differences are descriptive comparisons with a model, not estimates of preventable disease or county performance.

Using PLACES with its limitations

Suited to

  • Estimating likely burden where no local data exist (CDC's intended use)
  • Planning and needs assessments
  • Broad geographic patterns

Not suited to, on its own

  • Judging whether a county does better or worse than similar counties
  • Evaluating a local program or policy
  • Treating links with demographics, or between two PLACES measures, as independent evidence
  • Treating modeled differences as precise rankings of county health

Prefer direct local estimates when they measure the same outcome, population and period with enough precision. Claims and death records add complementary evidence, but they do not measure the same thing as adult prevalence surveys.

How it was checked

Independent reviews by Codex and Agy reconstructed specified analyses and checked their methods. Their findings led to corrections in county matching, suppression, prediction and smoothing. Statistical replication and website verification are separate checks; each review report, kept with the project's code, states the data, calculations and interface behavior it examined.

  1. Build

    Analysis written by Claude (Opus 5.5) under the author's direction, from public raw files.

  2. Replicate

    Codex (GPT-6 Astra) reconstructed the analyses within each review's stated scope, then audited the code.

  3. Challenge

    Agy (Gemini 4 Argon) reviewed the method and recomputed headline numbers. Its findings were verified before use.

  4. Correct

    The reviews led to the corrections illustrated below. Remaining implementation and methodological limitations are documented in the review reports.

Problems the reviews caught

  • Correlations between two measures computed on mismatched sets of counties
  • A summary figure that could not exceed 1.9% even if two measures moved together perfectly
  • Zero-death counties kept in the data, against the CDC rule on counts of 9 or fewer
  • A renamed Alaska county dropped from the mortality data
  • Kentucky and Pennsylvania missing from every atlas layer
  • Negative expected death rates in four counties
  • A small-county smoothing formula missing a centering step
  • Wording that claimed more than the analysis showed, on charts and on this page

Method, data and limits

Data

Method

  • Expected values: predicted from county demographics (plus Medicare Advantage share for claims, and optionally state) by models fit on other counties, so no county predicts itself; deaths use a rate model (Poisson regression)
  • Deaths: compared with the deaths expected at national age-specific rates (standardized mortality ratio, 100 = national rate)
  • Smoothing: pulls noisy small-county differences toward zero according to each source's uncertainty (empirical Bayes); Medicare denominators are approximate, and age-standardized Medicare is not smoothed

Limits

  • Measured is not the same as true disease: claims reflect diagnosis and coding, deaths reflect certificates
  • Connecticut has no mortality match; Medicare smoothing uses an approximate denominator
  • No test yet on other regions or years

Builds on Kong and Zhang, "The Use of Small Area Estimates in Place-Based Health Research", American Journal of Public Health, 2020.