= Estimating Measurement Error in Annual Job Earnings: A Comparison of Survey and Administrative Data =

'''Estimating Measurement Error in Annual Job Earnings: A Comparison of Survey and Administrative Data''' (DOI: https://doi.org/10.1162/REST_a_00352) was written by John M. Abowd and Martha H. Stinson. It was published in the Review of Economics and Statistics (2013). (My copy was actually published in the 2012 Social, Economic, and Housing Statistics Division (SEHSD) working papers series.)

Job-level annual earnings are collected from 5 [[UnitedStates/CensusBureau/SurveyOfIncomeAndProgramParticipation|SIPP]] panels and from the [[UnitedStates/WelfarePolicy/SocialSecurityAct|SSA]] Detailed Earnings Record (DER) (which itself is sourced from [[UnitedStates/InternalRevenueService|IRS]] W-2 forms). (The authors use confidential, unanonymized data.)

Errors are typically measured with reference to some external, true-er data source. This is a prior. The authors estimate a mixed effects model and observe changes in the levels of error as priors are varied. "The advantage of the MLMM framework is that it shows with full generality how to accommodate two or more matched observations of earnings on the same job, how to vary the prior assumptions about which measure is 'true' systematically, and how to use external audit information, if available, to update the posterior distribution over which value is 'true'."



== Preface to the model ==

Building off of [[MeasurementErrorModels|Fuller]] and [[EvidenceOnTheValidityOfCrossSectionalAndLongitudinalLaborMarketData|Bound et al]], the authors specify a measurement ''Y,,t,,'' in terms of a true value ''y,,t,,'' and uncorrelated measurement error ''u,,t,,''. The reliability ratio is characterized by:

{{attachment:reliability1.svg}}

When a variable is a function of the true value (like ''A,,t,, = βy,,t,, + ε,,t,,'') but is regressed on the measurement (like ''A,,t,, = βY,,t,, = βˆ(y,,t,, + u,,t,,)''), then the estimated coefficient is biased to zero.

{{attachment:reliability2.svg}}



== The model ==

There are ''I'' individuals, ''S'' sequential job spells, and ''T'' time periods. Job earnings are measured per individual ''i'', per sequential job spell ''s'', and per time period ''t'': ''y,,ist,,''. For every level ''s'' by ''t'', not every ''i'' has a measurement. All stacked up, there ''N'' measurements.

There are also ''J'' employers. All job earnings measurements (i.e., ''y,,ist,,'') have a corresponding employer ''j''.

Fixed effects are estimated for a ''K'' parameters. The fixed effect measurements and fixed effect coefficients are stacked into ''x,,ist,,'' and ''B'', both vectors of size ''K''

Random effects are estimated at individual- and employer-levels. Per individual ''i'', there is a random effect ''θ,,i,,''. These are stacked into ''Θ'', a vector of size ''I''. Per employer ''j'', there is a random effect ''ψ,,j,,''. These are stacked into ''Ψ'', a vector of size ''J''.

The model ''could'' be specified as:

''y,,ist,, = x,,ist,,B + d,,i,,Θ + f,,ist,,Ψ + η,,ist,,''

where:

 * ''η,,ist,,'' is the residual
 * ''d,,i,,'' (individual-level random effects design) is of size ''I''
 * ''f,,ist,,'' (employer-level random effects design) is of size ''J''

This model ''could'' then be estimated as:

''Y = '''X'''B + '''Z'''U + H''

 * both ''Y'' (stacked ''y,,ist,,'') and ''H'' (stacked ''η,,ist,,'') are of size ''N''
 * '''''X''''' (stacked ''x,,ist,,'') is of shape ''N'' by ''K''
 * '''''Z''' ≡ ['''D''' '''F''']''
   * '''''D''''' (stacked ''d,,i,,'') was of shape ''N'' by ''I''
   * '''''F''''' (stacked ''f,,ist,,'') was of shape ''N'' by ''I''
   * the combination of these two, '''''Z''''', is of shape ''N'' by ''(I + J)''
 * ''U ≡ [Θ Ψ]'' (combined random effects) is of size ''(I + J)''

But the intention of this model is to estimate the outcome from multiple sources of job earnings. The number of sources is ''Q''.

The fixed effect coefficients are set in '''''B''''' of shape ''K'' by ''Q'' (per source).

Per individual ''i'', there is a random effect vector ''θ,,i,,'' of size ''Q'' (per source). These are stacked into '''''Θ''''', a matrix of shape ''I'' by ''Q''. Per employer ''j'', there is a random effect vector ''ψ,,j,,'' of size ''Q'' (per source). These are stacked into '''''Ψ''''', a matrix of shape ''J'' by ''Q''.

The model is now specified as:

''y,,ist,, = x,,ist,,'''B''' + d,,i,,Θ + f,,ist,,Ψ + η,,ist,,''

where:

 * ''η,,ist,,'' is the residual vector of size ''Q'' (per source)

And the model is now estimated as:

'''''Y''' = '''XB''' + '''ZU''' + '''H'''''

where:

 * both ''Y'' (stacked ''y,,ist,,'') and ''H'' (stacked ''η,,ist,,'') are of shape ''N'' by ''Q''
 * '''''U''' ≡ ['''Θ'''^T^ '''Ψ'''^T^]^T^'' (combined random effects) is of shape ''(I + J)'' by ''Q''
   * the repeated transpositions just massage the combination into the correct shape

The model is fit using restricted maximum likelihood (REML) (via ASReml).



== Varying priors of the model ==

The outcome estimations are combined with a positive orthogonal weight vector ''ω''; each ''ω,,q,,'' is between 0 and 1 and they sum to 1.

The signal of the model is calculated as ''Sig(y,,ist,,) = ωy,,ist,,'' and the measurement error is the deviation of outcome estimations from the signal as ''ME(y,,ist,,) = y,,ist,, - Sig(y,,ist,,)''.

As an example, ''ω = [0 1]'' effectively is the statement that the second source is certain truth. This is how most measurement error evaluation studies are formalized: the secondary data source is the external, true-er one. The authors proceed with estimating the signal and measurement error using ''ω = [0.5 0.5]'' and ''ω = [0.1 0.9]''.

The authors note that a full Bayesian approach would be to model the priors per measurement, as ''ω,,ist,,'', using some other predictors (e.g., data quality measures ''g,,ist,,''). The model would then update the priors into a posterior.

Authors find that "reliability statistics for SIPP and DER earnings measures were quite comparable except for the sub-sample of SIPP person-jobs where at least one year of SIPP earnings contained a Census Bureau imputation". Also very little meaningful measurement error in fixed effects. The administrative data had greater variance, even when conditioned on the predictors.



----
CategoryRicottone