Survey Disposition

Survey disposition is the description of how a respondent interacted with a survey instrument.

Contents

Survey Disposition

Non-Response Bias

Non-response bias is introduced when an individual's likelihood to respond to a survey instrument is correlated to a key measurement.

Note that non-response bias is not the same as low response rates. Bias can be introduced if a subgroup's responsivity increases.

AAPOR Disposition Codes

The rough rubric for dispositioning are:

Eligible Respondent
Eligible Non-Respondent
Unknown Eligibility
Ineligible

Web Surveys

The common codes for web surveys are:

Code	Explanation
1.1	Complete
1.2	Partial with sufficient information (< NN% unanswered)
2.111	Explicit Refusal
2.112	Implicit Refusal (i.e. logged in but did not respond)
2.12	Partial with insufficient information (> NN% unanswered)
2.3	Data quality issues
3.19	Unknown eligibility (i.e. did not log in)
4.11	Screened Out of Sample - Reason #1
4.12	Screened Out of Sample - Reason #2
4.8	Quota Filled

The difference between treating low-quality data as an eligible non-respondent or ineligible is murky at best.

Paper Surveys

Paper surveys are in many ways simpler. Responses are generally higher quality. Whereas web interviews are often shown interactively, paper questionnaire are seen entirely and at once. Questions cannot be conditionally skipped on a paper questionnaire.

The common codes for paper surveys are:

Code	Explanation
1.1	Complete
1.2	Partial with sufficient information (< NN% unanswered)
2.111	Unknown Individual Refusal
2.112	Known Respondent Refusal
2.113	Implicit Refusal (i.e. known eligible but returned blank)
2.12	Partial with insufficient information (> NN% unanswered)
4.11	Screened Out of Sample - Reason #1
4.12	Screened Out of Sample - Reason #2

Completion

In a basic survey, use codes 1.1 and 2.112.

if (COMP_01 eq 1) DISPO=1.1.
if (COMP_01 eq 0) DISPO=2.112.

where COMP_01 indicates completion of any single question.

If a survey requires NN% completion, use codes 1.1, 1.2, and 2.12.

if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1.
if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2.
if (COMP_NN eq 0) DISPO=2.12.

where...

STATUS indicates the respondents access to a web survey, and COMPLETE means that the respondent does not have access resulting from reaching the end of the survey.
COMP_NN indicates the completion of NN%, or a completion rate >= 0.NN.

If a survey requires consent, use codes 1.1, 2.111, and 2.112.

if (COMP_01 eq 1) DISPO=1.1.
if (CONSENT ne 1) DISPO=2.111.
if (COMP_01 eq 0) DISPO=2.112.

where CONSENT is the consent question.

If a survey requires consent and NN% completion, use codes 1.1, 1.2, 2.111, 2.112, and 2.12.

if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1.
if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2.
if (COMP_NN eq 0) DISPO=2.12.
if (CONSENT ne 1) DISPO=2.111.
if (COMP_01 eq 0) DISPO=2.112.

Eligibility

If a survey requires some conditions to be met, use codes 1.1, 2.112, and 4.11 (and so on).

if (COMP_01 eq 1) DISPO=1.1.
if (COMP_01 eq 0) DISPO=2.112.
if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11.

where...

OVERQUOTA means that the respondent does not have access to the web survey resulting from failure of a required condition.
CONDITION_1 indicates success of the required condition.

If a survey requires some conditions to be met as well as requiring consent, use codes 1.1, 2.111, 2.112, and 4.11.

if (COMP_01 eq 1) DISPO=1.1.
if (CONSENT ne 1) DISPO=2.111.
if (COMP_01 eq 0) DISPO=2.112.
if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11.

If a survey requires some conditions to be met as well as NN% completion, use codes 1.1, 1.2, 2.12, 2.112, and 4.11.

if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1.
if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2.
if (COMP_NN eq 0) DISPO=2.12.
if (COMP_01 eq 0) DISPO=2.112.
if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11.

If a survey requires some conditions to be met, consent, and NN% completion, use codes 1.1, 1.2, 2.111, 2.12, 2.112, and 4.11.

if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1.
if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2.
if (COMP_NN eq 0) DISPO=2.12.
if (CONSENT ne 1) DISPO=2.111.
if (COMP_01 eq 0) DISPO=2.112.
if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11.

Panel Quotas

If a panel survey has quotas, add code 4.8.

if (STATUS eq OVERQUOTA) DISPO=4.8.

where OVERQUOTA means that the respondent does not have access to the web survey resulting from a closed quota.

CategoryRicottone

Statistics/SurveyDisposition