Survey Statistics
There are a number of considerations for analyzing survey data.
Contents
Statistics with Survey Data
There are key differences between the statistics employed in models and in surveys.
Model-based inference
- Build a mathematical model that describes a population.
- Generate a random sample from that population to generate estimates.
- Estimate how the error terms of those estimates would vary if repeated samples were drawn.
In other words, the key is how well the model describes the population.
Design-based inference
- Identify a population with fixed descriptives.
- Draw a sample from that population to collect measures from.
- Estimate how the measures would vary if repeated samples were drawn.
In other words, the key is how well the sample fits the population. If the full population were contacted, measures would be perfect.
Inferential statistics from complex survey data
Using model-based inference while accounting for survey design.
Survey Populations
A sample is contacted, rather than the full population, for reasons of cost and administration. There are costs paid for this decision.
Limitations of Survey Sampling
If a sample is a poor fit for the population, then it will be difficult/impossible to estimate population descriptives. This is why samples are randomly drawn.
But for random sampling to succeed, the target populations needs to be completely identified. Incompleteness of frame data can skew samples.
See here for a details on survey sampling.
Limitations of Surveying
Non-random non-response is an additional roadblock to estimating population descriptives.
Some populations are inherently difficult to contact, due to:
- political, legal, and ethical considerations (i.e. minors in the population)
- language and cultural barriers
- remoteness of location (i.e. rural Alaska)
The availability of contact data often dictates the mode of survey instrument. Some populations are not easily contacted by specific modes, due to:
- socioeconomic conditions (i.e. lack of telephone connectivity)
- low accuracy contact data (i.e. outdated addresses)