|
Size: 5224
Comment:
|
Size: 3200
Comment: Removing nonresponse bias section
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| Survey disposition is the description of how a respondent interacted with a survey instrument. | A '''survey disposition''' encodes the outcome of an interview. |
| Line 10: | Line 10: |
| == Disposition Codes == | |
| Line 11: | Line 12: |
| == Non-Response Bias == | The four overarching categories for interview outcomes are: |
| Line 13: | Line 14: |
| Non-response bias is introduced when an individual's likelihood to respond to a survey instrument is correlated to a key measurement. | 1. Complete eligible repondents 2. Eligible nonrespondents 3. Unknown eligible nonrespondents 4. Ineligible respondents |
| Line 15: | Line 19: |
| Note that non-response bias is not the same as ''low'' response rates. Bias can be introduced if a subgroup's responsivity ''increases''. | The codes then have subcategories, i.e. 1.1 for a truly complete respondent and 1.2 for a partial respondent with sufficient information. There may also be codes for completion by a proxy at 1.11 and 1.21. A disposition code can then be truncated at certain levels of precision to simplify categories, up to and including the collapse of all precise categories into the four overarchive categories. |
| Line 17: | Line 21: |
| ---- | AAPOR has published guidance to standardize the encoding of dispositions. |
| Line 21: | Line 25: |
| == AAPOR Disposition Codes == The rough rubric for dispositioning are: 1. Eligible Respondent 2. Eligible Non-Respondent 3. Unknown Eligibility 4. Ineligible ---- == Web Surveys == |
=== Web Surveys === |
| Line 41: | Line 32: |
| ||2.111 ||Explicit Refusal|| ||2.112 ||Implicit Refusal (i.e. logged in but did not respond)|| |
||2.111 ||Explicit refusal by other individual|| ||2.112 ||Explicit refusal by addressee|| |
| Line 44: | Line 35: |
| ||2.3 ||Data quality issues|| ||3.19 ||Unknown eligibility (i.e. did not log in)|| ||4.11 ||Screened Out of Sample - Reason #1|| ||4.12 ||Screened Out of Sample - Reason #2|| |
||2.27 ||Too late response|| ||2.35 ||Technological issues|| ||3.19 ||Unknown eligibility (i.e. did not login)|| ||4.11 ||Screenout - Reason #1|| ||4.12 ||Screenout - Reason #2|| |
| Line 50: | Line 42: |
| The difference between treating low-quality data as an eligible non-respondent or ineligible is murky at best. ---- |
If respondents were invited as [[Statistics/SurveyFrame#List_Frames|specifically-named individuals]], nonrespondents may be 2.113 (i.e. Implicit Refusal). Otherwise they are typically 3.19 (i.e. Did not login). |
| Line 55: | Line 45: |
== Paper Surveys == |
=== Paper Surveys === |
| Line 60: | Line 49: |
| The common codes for paper surveys are: | On the other hand, some information gathered from nixie codes ought to be incorporated. |
| Line 65: | Line 54: |
| ||2.111 ||Unknown Individual Refusal|| ||2.112 ||Known Respondent Refusal|| ||2.113 ||Implicit Refusal (i.e. known eligible but returned blank)|| |
||2.111 ||Explicit refusal by other individual|| ||2.112 ||Explicit refusal by addressee|| |
| Line 69: | Line 57: |
| ||4.11 ||Screened Out of Sample - Reason #1|| ||4.12 ||Screened Out of Sample - Reason #2|| |
||2.27 ||Too late response|| ||2.30 ||Deceased|| ||3.19 ||Unknown eligiblity (i.e. returned blank)|| ||3.31 ||Mail returned undelivered|| ||3.32 ||Mail refused by addressee|| ||4.11 ||Screenout - Reason #1|| ||4.12 ||Screenout - Reason #2|| If respondents were invited as [[Statistics/SurveyFrame#List_Frames|specifically-named individuals]], nonrespondents may be 2.113 (i.e. Implicit Refusal). Otherwise they are typically 3.19 (i.e. Nothing returned). |
| Line 74: | Line 69: |
| === Completion === In a basic survey, use codes 1.1 and 2.112. {{{ if (COMP_01 eq 1) DISPO=1.1. if (COMP_01 eq 0) DISPO=2.112. }}} where ''COMP_01'' indicates completion of any single question. If a survey requires NN% completion, use codes 1.1, 1.2, and 2.12. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. }}} where... * ''STATUS'' indicates the respondents access to a web survey, and ''COMPLETE'' means that the respondent does not have access resulting from reaching the end of the survey. * ''COMP_NN'' indicates the completion of NN%, or a completion rate >= 0.NN. |
=== Telephone Surveys === |
| Line 99: | Line 73: |
| === Consent === | === Multi-modal Surveys === |
| Line 101: | Line 75: |
| If a survey requires consent, use codes 1.1, 2.111, and 2.112. | A multi-modal survey has to incorporate components from each of the above. |
| Line 103: | Line 77: |
| {{{ if (COMP_01 eq 1) DISPO=1.1. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=2.112. }}} where ''CONSENT'' is the consent question. If a survey requires consent and NN% completion, use codes 1.1, 1.2, 2.111, 2.112, and 2.12. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=2.112. }}} === Eligibility === If a survey requires some conditions to be met, use codes 1.1, 2.112, and 4.11 (and so on). {{{ if (COMP_01 eq 1) DISPO=1.1. if (COMP_01 eq 0) DISPO=2.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} where... * ''OVERQUOTA'' means that the respondent does not have access to the web survey resulting from failure of a required condition. * ''CONDITION_1'' indicates success of the required condition. If a survey requires some conditions to be met as well as requiring consent, use codes 1.1, 2.111, 2.112, and 4.11. {{{ if (COMP_01 eq 1) DISPO=1.1. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=2.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} If a survey requires some conditions to be met as well as NN% completion, use codes 1.1, 1.2, 2.12, 2.112, and 4.11. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. if (COMP_01 eq 0) DISPO=2.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} If a survey requires some conditions to be met, consent, and NN% completion, use codes 1.1, 1.2, 2.111, 2.12, 2.112, and 4.11. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=2.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} === Panel Quotas === If a panel survey has quotas, add code 4.8. {{{ if (STATUS eq OVERQUOTA) DISPO=4.8. }}} where ''OVERQUOTA'' means that the respondent does not have access to the web survey resulting from a closed quota. |
A push-to-web survey, for example, typically can start with web survey dispositioning as a template. But nixie information reflected in paper survey dispositioning must be inserted. |
Survey Disposition
A survey disposition encodes the outcome of an interview.
Contents
Disposition Codes
The four overarching categories for interview outcomes are:
- Complete eligible repondents
- Eligible nonrespondents
- Unknown eligible nonrespondents
- Ineligible respondents
The codes then have subcategories, i.e. 1.1 for a truly complete respondent and 1.2 for a partial respondent with sufficient information. There may also be codes for completion by a proxy at 1.11 and 1.21. A disposition code can then be truncated at certain levels of precision to simplify categories, up to and including the collapse of all precise categories into the four overarchive categories.
AAPOR has published guidance to standardize the encoding of dispositions.
Web Surveys
The common codes for web surveys are:
Code |
Explanation |
1.1 |
Complete |
1.2 |
Partial with sufficient information (< NN% unanswered) |
2.111 |
Explicit refusal by other individual |
2.112 |
Explicit refusal by addressee |
2.12 |
Partial with insufficient information (> NN% unanswered) |
2.27 |
Too late response |
2.35 |
Technological issues |
3.19 |
Unknown eligibility (i.e. did not login) |
4.11 |
Screenout - Reason #1 |
4.12 |
Screenout - Reason #2 |
4.8 |
Quota Filled |
If respondents were invited as specifically-named individuals, nonrespondents may be 2.113 (i.e. Implicit Refusal). Otherwise they are typically 3.19 (i.e. Did not login).
Paper Surveys
Paper surveys are in many ways simpler. Responses are generally higher quality. Whereas web interviews are often shown interactively, paper questionnaire are seen entirely and at once. Questions cannot be conditionally skipped on a paper questionnaire.
On the other hand, some information gathered from nixie codes ought to be incorporated.
Code |
Explanation |
1.1 |
Complete |
1.2 |
Partial with sufficient information (< NN% unanswered) |
2.111 |
Explicit refusal by other individual |
2.112 |
Explicit refusal by addressee |
2.12 |
Partial with insufficient information (> NN% unanswered) |
2.27 |
Too late response |
2.30 |
Deceased |
3.19 |
Unknown eligiblity (i.e. returned blank) |
3.31 |
Mail returned undelivered |
3.32 |
Mail refused by addressee |
4.11 |
Screenout - Reason #1 |
4.12 |
Screenout - Reason #2 |
If respondents were invited as specifically-named individuals, nonrespondents may be 2.113 (i.e. Implicit Refusal). Otherwise they are typically 3.19 (i.e. Nothing returned).
Telephone Surveys
Multi-modal Surveys
A multi-modal survey has to incorporate components from each of the above.
A push-to-web survey, for example, typically can start with web survey dispositioning as a template. But nixie information reflected in paper survey dispositioning must be inserted.