Differences between revisions 2 and 35 (spanning 33 versions)
⇤ ← Revision 2 as of 2020-10-22 19:26:39 →
Size: 887
Editor: DominicRicottone
Comment:
← Revision 35 as of 2026-07-21 01:33:54 → ⇥
Size: 6210
Editor: DominicRicottone
Comment: Relink
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
Survey weights account for the design of a survey sample and other biases/errors introduced by a survey instrument. '''Survey weights''' account for the [[Statistics/SurveySampling|survey design]], [[Statistics/SurveyInference#Sampling_Error|sampling error]], and [[Statistics/SurveyInference#Non-sampling_Error|non-sampling error]].
Line 11: Line 11:
== The Basic Process == == Description ==
Line 13: Line 13:
 1. Set survey dispositions
 2. Set base weights
 3. Apply non-response adjustments to base weights
 4. Calibrate the weights		 Survey data is collected through a mechanism which can be specified statistically. If it is not specified, bias can be introduced and [[Analysis/Estimation|estimates]] can be over-confident.
Line 18: Line 15:
See [[SurveyDisposition|here]] for details about survey dispositions. [[Statistics/InverseVarianceWeights|Inverse variance weights]] are related, but not the same.

Survey weights begin with [[Statistics/DesignWeights|design weights]] reflecting [[Statistics/SurveySampling|probability of selection]]. Generally this is simply the inverse of the sampling probability: ''n,,k,,/N'' for all strata ''k''.

All real surveys feature [[Statistics/SurveyInference#Non-sampling_Error|non-sampling error]], especially [[Statistics/SurveyNonresponse|nonresponse]]. If nonresponse is uncorrelated with key metrics, it is negligible. Otherwise there is potential for [[Statistics/NonresponseBias|nonresponse bias]]. This bias can be corrected through survey weights in a few ways:
 * [[Statistics/InverseProbabilityWeights|inverse propensity adjustments]]
 * [[Statistics/WeightingClassAdjustment|weighting class adjustments]]

Modeling on insignificant or uncorrelated attributes does not introduce bias, but it does inflate [[Statistics/Variance|variance]].

[[Statistics/Calibration|Calibration]] can be used to:
 * make estimates be consistent with known true population proportions
 * correct [[Statistics/SurveyInference#Sampling_Error|sampling error]] like undercoverage or overcoverage
 * further correct for non-sampling error like nonresponse bias

The methods here include:
 * raking
 * iterative proportional fitting
 * RIM weighting
 * [[Statistics/GeneralizedRegressionEstimator|GREG estimators]]

----
Line 22: Line 40:
== Weighted Estimators ==

Survey weights ''w'' are designed such that a population proportion ''μ'' can be calculated using the weighted estimator ''Σ(wx) / Σw''.

In the case that all cases have equal weight, the [[Statistics/Variance|variance]] of that estimator is ''w^2^σ^2^''.

In any other case, the variance is given by ''Σ(w^2^σ^2^) / (Σw)^2^''. This ratio must then be linearized or simulated to arrive at an approximate variance. [[Calculus/TaylorSeries|Taylor expansion]] is a common strategy for linearization.
Line 24: Line 50:
== Calculating Weights ==
Line 26: Line 51:
The base weight is the inverse of the probability of being sampled. Think ''desired over actual''. As such, the sum of base weights should equal the population size.
Line 28: Line 52:
For a SRS design, this is calculated as a simple rate. Given a population of 20,000 and a sample size of 667, the propbability of being sampled is 20,000/667 = '''29.99'''. == Reading Notes ==
Line 30: Line 54:
For a STSRS design, the same process is applied per stratum.  * [[AGeneralizationOfSamplingWithoutReplacementFromAFiniteUniverse|A Generalization of Sampling Without Replacement from a Finite Universe]], D. G. Horvitz and D. J. Thompson, 1952
 * [[CalibrationEstimatorsInSurveySampling|Calibration estimators in survey sampling]], Jean-Claude Deville and Carl-Erik Särndal, 1992
 * [[TheEffectOfWeightTrimmingOnNonlinearSurveyEstimates|The Effect of Weight Trimming on Nonlinear Survey Estimates]], Frank J. Potter, 1993
 * [[PostStratificationAModelersPerspective|Post-Stratification: A Modeler's Perspective]], Roderick J.A. Little, 1993
 * [[SamplingWeightsAndRegressionAnalysis|Sampling Weights and Regression Analysis]], Christopher Winship and Larry Radbill, 1994
 * [[ImprovingOnProbabilityWeightingForHouseholdSize|Improving on Probability Weighting for Household Size]], Andrew Gelman and Thomas C. Little, 1998
 * [[RandomEffectsModelsForSmoothingPoststratificationWeights|Random-effects Models for Smoothing Poststratitication Weights]], Laura C. Lazzeroni and Roderick J.A. Little, 1998
 * [[TheGeneralizedExponentialModelForSamplingWeightCalibrationForExtremeValuesNonresponseAndPostStratification|The generalized exponential model for sampling weight calibration for extreme values, nonresponse, and poststratification]], R.E. Folsom and A.C. Singh, 2000
 * [[UsingCalibrationWeightingToAdjustForNonresponseAndCoverageErrors|Using Calibration Weighting to Adjust for Nonresponse and Coverage Errors]], Phillip S. Kott, 2006
 * [[EstimationInMultipleFrameSurveys|Estimation in Multiple-Frame Surveys]], Sharon Lohr and J. N. K. Rao, 2006
 * [[StrugglesWithSurveyWeightingAndRegressionModeling|Struggles with Survey Weighting and Regression Modeling]], Andrew Gelman, 2007
 * [[TheCalibrationApproachInSurveyTheoryAndPractice|The calibration approach in survey theory and practice]], Carl-Erik Särndal, 2007
 * [[ASingleFrameMultiplicityEstimatorForMultipleFrameSurveys|A single frame multiplicity estimator for multiple frame surveys]], Fulvia Mecatti, 2007
 * [[WeightTrimmingInTheNationalImmunizationSurvey|Weight Trimming in the National Immunization Survey]]; Sadeq Chowdhury, Meena Khare, and Kirk Wolter; 2007
 * [[PracticalConsiderationsInRakingSurveyData|Practical Considerations in Raking Survey Data]]; Michael P Battaglia, David C Hoaglin, and Martin R Frankel (and sometimes David Izrael); 2009
 * [[MethodsForAdjustingSurveyWeightsWhenEstimatingATotal|Methods for Adjusting Survey Weights When Estimating a Total]], Kimberly Henry and Richard Valliant; 2012
 * [[StatisticalParadisesAndParadoxesInBigData|Statistical Paradises and Paradoxes in Big Data]], Xiao-Li Meng, 2018
 * [[ANewParadigmForPolling|A New Paradigm for Polling]], Michael A. Bailey, 2023
 * [[TheLawOfLargePopulationsDoesNotHeraldAParadigmShiftInSurveySampling|The “Law of Large Populations” Does Not Herald a Paradigm Shift in Survey Sampling]], Roderick J. Little, 2023
 * [[SurveysOfConsumersTechnicalReport|Surveys of Consumers Technical Report: Technical Documentation for the 2024 Methodological Transition to Web Surveys]], 2024
 * [[TheEffectOfOnlineInterviewsOnTheUniversityOfMichiganSurveyOfConsumerSentiment|The effect of online interviews on the University of Michigan Survey of Consumer Sentiment]], Ryan Cummings and Ernie Tedeschi, 2024
 * [[LeveragingPredictiveModellingFromMultipleSourcesOfBigDataToImproveSampleEfficiencyAndReduceSurveyNonresponseError|Leveraging Predictive Modelling from Multiple Sources of Big Data to Improve Sample Efficiency and Reduce Survey Nonresponse Error]]; David Dutwin, Patrick Coyle, Joshua Lerner, Ipek Bilgen, and Ned English; 2024

Survey Weights

Survey weights account for the survey design, sampling error, and non-sampling error.

Contents

  1. Survey Weights
    1. Description
    2. Weighted Estimators
    3. Reading Notes

Description

Survey data is collected through a mechanism which can be specified statistically. If it is not specified, bias can be introduced and estimates can be over-confident.

Inverse variance weights are related, but not the same.

Survey weights begin with design weights reflecting probability of selection. Generally this is simply the inverse of the sampling probability: nk/N for all strata k.

All real surveys feature non-sampling error, especially nonresponse. If nonresponse is uncorrelated with key metrics, it is negligible. Otherwise there is potential for nonresponse bias. This bias can be corrected through survey weights in a few ways:

Modeling on insignificant or uncorrelated attributes does not introduce bias, but it does inflate variance.

Calibration can be used to:

  • make estimates be consistent with known true population proportions

  • correct sampling error like undercoverage or overcoverage

  • further correct for non-sampling error like nonresponse bias

The methods here include:

Weighted Estimators

Survey weights w are designed such that a population proportion μ can be calculated using the weighted estimator Σ(wx) / Σw.

In the case that all cases have equal weight, the variance of that estimator is w2σ2.

In any other case, the variance is given by Σ(w2σ2) / (Σw)2. This ratio must then be linearized or simulated to arrive at an approximate variance. Taylor expansion is a common strategy for linearization.

Reading Notes

CategoryRicottone

Statistics/SurveyWeights (last edited 2026-07-21 01:40:36 by DominicRicottone)