Exploring Nonresponse Bias in a Health Survey Using Neighborhood Characteristics

Exploring Nonresponse Bias in a Health Survey Using Neighborhood Characteristics (DOI: https://doi.org/10.2105/AJPH.2008.154161) was written by Sunghee Lee, E. Richard Brown, David Grant, Thomas R. Belin, and Michael Brick in 2009. It was published in the American Journal of Public Health (vol. 99, no. 10).

The authors examine whether neighborhood characteristics predict nonresponse to the screener of the 2005 California Health Interview Survey (CHIS).

• RDD sample

  • Interviewers attempted to reach households 15 times
  • screener selects adults for the main survey
• telephone numbers are geocoded
  • if an exact address could not be determined, then
    • most probable ZIP code assigned
    • Census tract of the centroid of the most probable ZIP code's area assigned

• Aggregated statistics at the Census tract level attached to CHIS data

  • from the 2000 Census...
    • "proportions of non-Hispanic Whites, urban dwellers, never-married persons, linguistically isolated persons, those living in the same house as in 1995, those with less than a high school education, unemployed persons, median household income"

  • also the HTC score

  • 10 cases from tracts formed after 2000 Census are excluded

First, nonrespondents were classified as one of refusals, noncontacts, and other. Authors compare neighborhood characteristics across these groups.

Second, the mean of each neighborhood characteristic was calculated, and tracts were categorized according to being above or below that threshold. Response rates were compared between the above/below groups.

Finally, the authors regressed response on CHIS variables and the above neighborhood characteristics to obtain a model-predicted response propensity.

• CHIS data available only for those who were selected for main survey and responded completely.
  • "current health insurance coverage, self-report of fair or poor health, overweight or obese status, disability status, binge drinking in the last 12 months, and current smoking status"

• Multilevel design

Reading notes

The authors find some correlation between neighborhood characteristics and response, but importantly do not find any correlation between those same covariates and key metrics from the CHIS. Therefore, they say no evidence of nonresponse bias.

The authors purposefully do not report significance tests because they expect significance due to large sample sizes no matter what. I call BS. In any case, it means there's only so much I can do to interpret the results without replicating.

Much smarter people than I have argued against binning cases based on mean thresholds, I don't think the second analysis holds up.

CategoryRicottone CategoryReadingNotes