|
Size: 4548
Comment:
|
← Revision 19 as of 2026-02-06 20:41:41 ⇥
Size: 2977
Comment: Rewrite
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Disposition Codes = | = Survey Disposition = |
| Line 3: | Line 3: |
| '''Disposition codes''' are a means for classifying survey response and non-response, devised as an attempt to understand why a respondent did or did not complete a survey. | A '''survey disposition''' encodes the outcome of an interview. |
| Line 10: | Line 10: |
| == Description == | |
| Line 11: | Line 12: |
| == AAPOR Standard == | A survey disposition encodes the most important information about how an individual interacted with a survey instrument. The major categories are: |
| Line 13: | Line 14: |
| The rough rubric for disposition codes are: | 1. Eligible respondents 2. Eligible nonrespondents 3. Sample members with unknown eligibility 4. Ineligible sample members |
| Line 15: | Line 19: |
| 1. Eligible Respondent 2. Eligible Non-Respondent 3. Unknown Eligibility 4. Ineligible |
There are then subcategories with finer detail, for example the reason a sample member is ineligible. |
| Line 24: | Line 25: |
| == Web Surveys in Practice == | == Coding == |
| Line 26: | Line 27: |
| The common codes for web surveys are: | AAPOR has published guidance to standardize the encoding of dispositions. |
| Line 28: | Line 29: |
| ||'''Code'''||'''Explanation'''|| ||1.1 ||Complete|| ||1.2 ||Partial with sufficient information (< NN% unanswered)|| ||2.111 ||Explicit Refusal|| ||2.112 ||Implicit Refusal|| ||2.12 ||Partial with insufficient information (> NN% unanswered)|| ||2.3 ||Data quality issues|| ||3.112 ||Logged on to survey, did not complete any items|| ||4.11 ||Screened Out of Sample - Reason #1|| ||4.12 ||Screened Out of Sample - Reason #2|| ||4.8 ||Quota Filled|| |
Major categories are an integer part, while minor categories are a decimal part. These codes can be truncated at any level to collapse subcategories into parent categories. For example, codes 4.11 and 4.12 may convey different survey items whereby sample members have indicated their ineligibility. If any analyst only need to know which cases self-identified as ineligible through any survey item, the codes can be truncated to the tenths decimal place; 4.11 and 4.12 become 4.1. |
| Line 40: | Line 31: |
| If there is only one way to screen out, 4.11 may be recoded to 4.1. If there is no explicit refusal, 2.112 may be recoded to 2.11. | When working with a sample of specifically-named individuals, there is generally a presumption of eligibility. It follows that: * An explicit refusal to participate in a survey, either as declining to give consent or as requesting to be removed from a contact list, is generally coded as eligible nonresponse. * An incomplete survey response, either due to item nonresponse or due to technical issues, is generally coded as eligible nonresponse. |
| Line 42: | Line 35: |
| The difference between treating low-quality data as an eligible non-respondent or ineligible is murky at best. | The exception is when contact is attempted but not necessarily successful. This leaves open the possibility that the specifically-named individual has e.g. moved, and is no longer member to the eligible population. Mailed surveys will often have many subcategories of unknown eligibility, corresponding to the various '''nixie codes''' returned by the postal service: * Undeliverable as addresses * Refused by addressee * Vacant * Deceased * and so on |
| Line 44: | Line 42: |
| ---- | Similarly, a survey link that is shared by email can lead to a variety of email bouncebacks. |
| Line 46: | Line 44: |
| If responses are collected from a nonprobability panel, individuals often must be considered as having unknown eligibility until they provide survey responses indicating otherwise. | |
| Line 47: | Line 46: |
| Nonprobability panel surveys also often incorporate live or rebalancing quotas which cause 'surplus' respondents to be terminated as 'over-quota'. In most cases these are treated as eligible nonrespondents. | |
| Line 48: | Line 48: |
| == Paper Surveys in Practice == Paper surveys are in many ways simpler. Responses are generally higher quality. Whereas web interviews are often shown interactively, paper questionnaire are seen entirely and at once. Questions cannot be conditionally skipped on a paper questionnaire. === Completion === In a basic survey, use codes 1.1 and 2.112. {{{ if (COMP_01 eq 1) DISPO=1.1. if (COMP_01 eq 0) DISPO=2.112. }}} where ''COMP_01'' indicates completion of any single question. If a survey requires NN% completion, use codes 1.1, 1.2, and 2.12. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. }}} where... * ''STATUS'' indicates the respondents access to a web survey, and ''COMPLETE'' means that the respondent does not have access resulting from reaching the end of the survey. * ''COMP_NN'' indicates the completion of NN%, or a completion rate >= 0.NN. === Consent === If a survey requires consent, use codes 1.1, 2.111, and 2.112. {{{ if (COMP_01 eq 1) DISPO=1.1. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=2.112. }}} where ''CONSENT'' is the consent question. If a survey requires consent and NN% completion, use codes 1.1, 1.2, 2.111, 2.112, and 2.12. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=2.112. }}} === Eligibility === If a survey requires some conditions to be met, use codes 1.1, 3.112, and 4.11 (and so on). {{{ if (COMP_01 eq 1) DISPO=1.1. if (COMP_01 eq 0) DISPO=3.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} where... * ''OVERQUOTA'' means that the respondent does not have access to the web survey resulting from failure of a required condition. * ''CONDITION_1'' indicates success of the required condition. If a survey requires some conditions to be met as well as requiring consent, use codes 1.1, 2.111, 3.112, and 4.11. {{{ if (COMP_01 eq 1) DISPO=1.1. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=3.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} If a survey requires some conditions to be met as well as NN% completion, use codes 1.1, 1.2, 2.12, 3.112, and 4.11. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. if (COMP_01 eq 0) DISPO=3.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} If a survey requires some conditions to be met, consent, and NN% completion, use codes 1.1, 1.2, 2.111, 2.12, 3.112, and 4.11. {{{ if (STATUS eq COMPLETE and COMP_NN eq 1) DISPO=1.1. if (STATUS ne COMPLETE and COMP_NN eq 1) DISPO=1.2. if (COMP_NN eq 0) DISPO=2.12. if (CONSENT ne 1) DISPO=2.111. if (COMP_01 eq 0) DISPO=3.112. if (STATUS eq TERMINATE and CONDITION_1 eq 0) DISPO=4.11. }}} === Panel Quotas === If a panel survey has quotas, add code 4.8. {{{ if (STATUS eq OVERQUOTA) DISPO=4.8. }}} where ''OVERQUOTA'' means that the respondent does not have access to the web survey resulting from a closed quota. |
A multi-modal survey has to incorporate components from each of the above, including a hierarchy to reconcile contradictory dispositioning information. For example, consider an individual that was sent a survey link in both a mailed postcard and an email. If a response is collected and one of those invites is returned with information, a decision must be made with regard to which information is more accurate. |
Survey Disposition
A survey disposition encodes the outcome of an interview.
Contents
Description
A survey disposition encodes the most important information about how an individual interacted with a survey instrument. The major categories are:
- Eligible respondents
- Eligible nonrespondents
- Sample members with unknown eligibility
- Ineligible sample members
There are then subcategories with finer detail, for example the reason a sample member is ineligible.
Coding
AAPOR has published guidance to standardize the encoding of dispositions.
Major categories are an integer part, while minor categories are a decimal part. These codes can be truncated at any level to collapse subcategories into parent categories. For example, codes 4.11 and 4.12 may convey different survey items whereby sample members have indicated their ineligibility. If any analyst only need to know which cases self-identified as ineligible through any survey item, the codes can be truncated to the tenths decimal place; 4.11 and 4.12 become 4.1.
When working with a sample of specifically-named individuals, there is generally a presumption of eligibility. It follows that:
- An explicit refusal to participate in a survey, either as declining to give consent or as requesting to be removed from a contact list, is generally coded as eligible nonresponse.
- An incomplete survey response, either due to item nonresponse or due to technical issues, is generally coded as eligible nonresponse.
The exception is when contact is attempted but not necessarily successful. This leaves open the possibility that the specifically-named individual has e.g. moved, and is no longer member to the eligible population. Mailed surveys will often have many subcategories of unknown eligibility, corresponding to the various nixie codes returned by the postal service:
- Undeliverable as addresses
- Refused by addressee
- Vacant
- Deceased
- and so on
Similarly, a survey link that is shared by email can lead to a variety of email bouncebacks.
If responses are collected from a nonprobability panel, individuals often must be considered as having unknown eligibility until they provide survey responses indicating otherwise.
Nonprobability panel surveys also often incorporate live or rebalancing quotas which cause 'surplus' respondents to be terminated as 'over-quota'. In most cases these are treated as eligible nonrespondents.
A multi-modal survey has to incorporate components from each of the above, including a hierarchy to reconcile contradictory dispositioning information. For example, consider an individual that was sent a survey link in both a mailed postcard and an email. If a response is collected and one of those invites is returned with information, a decision must be made with regard to which information is more accurate.