|
Size: 3523
Comment: Killing SurveyStatistics page
|
← Revision 19 as of 2026-02-06 20:41:41 ⇥
Size: 2977
Comment: Rewrite
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 10: | Line 10: |
| == Disposition Codes == | == Description == |
| Line 12: | Line 12: |
| The four overarching categories for interview outcomes are: | A survey disposition encodes the most important information about how an individual interacted with a survey instrument. The major categories are: |
| Line 14: | Line 14: |
| 1. Complete eligible repondents 2. Eligible non-respondents 3. Unknown eligible non-respondents 4. Ineligible respondents |
1. Eligible respondents 2. Eligible nonrespondents 3. Sample members with unknown eligibility 4. Ineligible sample members |
| Line 19: | Line 19: |
| The codes then have subcategories, i.e. 1.1 for a truly complete respondent and 1.2 for a partial respondent with sufficient information. There may also be codes for completion by a proxy at 1.11 and 1.21. A disposition code can then be truncated at certain levels of precision to simplify categories, up to and including the collapse of all precise categories into the four overarchive categories. AAPOR has published guidance to standardize the encoding of dispositions. === Web Surveys === The common codes for web surveys are: ||'''Code'''||'''Explanation'''|| ||1.1 ||Complete|| ||1.2 ||Partial with sufficient information (< NN% unanswered)|| ||2.111 ||Explicit refusal by other individual|| ||2.112 ||Explicit refusal by addressee|| ||2.12 ||Partial with insufficient information (> NN% unanswered)|| ||2.27 ||Too late response|| ||2.35 ||Technological issues|| ||3.19 ||Unknown eligibility (i.e. did not login)|| ||4.11 ||Screenout - Reason #1|| ||4.12 ||Screenout - Reason #2|| ||4.8 ||Quota Filled|| If respondents were invited as [[Statistics/SurveyFrames#List_Frames|specifically-named individuals]], non-respondents may be 2.113 (i.e. Implicit Refusal). Otherwise they are typically 3.19 (i.e. Did not login). === Paper Surveys === Paper surveys are in many ways simpler. Responses are generally higher quality. Whereas web interviews are often shown interactively, paper questionnaire are seen entirely and at once. Questions cannot be conditionally skipped on a paper questionnaire. On the other hand, some information gathered from nixie codes ought to be incorporated. ||'''Code'''||'''Explanation'''|| ||1.1 ||Complete|| ||1.2 ||Partial with sufficient information (< NN% unanswered)|| ||2.111 ||Explicit refusal by other individual|| ||2.112 ||Explicit refusal by addressee|| ||2.12 ||Partial with insufficient information (> NN% unanswered)|| ||2.27 ||Too late response|| ||2.30 ||Deceased|| ||3.19 ||Unknown eligiblity (i.e. returned blank)|| ||3.31 ||Mail returned undelivered|| ||3.32 ||Mail refused by addressee|| ||4.11 ||Screenout - Reason #1|| ||4.12 ||Screenout - Reason #2|| If respondents were invited as [[Statistics/SurveyFrames#List_Frames|specifically-named individuals]], non-respondents may be 2.113 (i.e. Implicit Refusal). Otherwise they are typically 3.19 (i.e. Nothing returned). === Telephone Surveys === === Multi-modal Surveys === A multi-modal survey has to incorporate components from each of the above. A push-to-web survey, for example, typically can start with web survey dispositioning as a template. But nixie information reflected in paper survey dispositioning must be inserted. |
There are then subcategories with finer detail, for example the reason a sample member is ineligible. |
| Line 83: | Line 25: |
| == Non-Response Bias == | == Coding == |
| Line 85: | Line 27: |
| Non-response bias is introduced when an individual's likelihood to respond to a survey instrument is correlated to a key measurement. | AAPOR has published guidance to standardize the encoding of dispositions. |
| Line 87: | Line 29: |
| Note that non-response bias is not the same as ''low'' response rates. Bias can be introduced if a subgroup's responsivity ''increases''. | Major categories are an integer part, while minor categories are a decimal part. These codes can be truncated at any level to collapse subcategories into parent categories. For example, codes 4.11 and 4.12 may convey different survey items whereby sample members have indicated their ineligibility. If any analyst only need to know which cases self-identified as ineligible through any survey item, the codes can be truncated to the tenths decimal place; 4.11 and 4.12 become 4.1. When working with a sample of specifically-named individuals, there is generally a presumption of eligibility. It follows that: * An explicit refusal to participate in a survey, either as declining to give consent or as requesting to be removed from a contact list, is generally coded as eligible nonresponse. * An incomplete survey response, either due to item nonresponse or due to technical issues, is generally coded as eligible nonresponse. The exception is when contact is attempted but not necessarily successful. This leaves open the possibility that the specifically-named individual has e.g. moved, and is no longer member to the eligible population. Mailed surveys will often have many subcategories of unknown eligibility, corresponding to the various '''nixie codes''' returned by the postal service: * Undeliverable as addresses * Refused by addressee * Vacant * Deceased * and so on Similarly, a survey link that is shared by email can lead to a variety of email bouncebacks. If responses are collected from a nonprobability panel, individuals often must be considered as having unknown eligibility until they provide survey responses indicating otherwise. Nonprobability panel surveys also often incorporate live or rebalancing quotas which cause 'surplus' respondents to be terminated as 'over-quota'. In most cases these are treated as eligible nonrespondents. A multi-modal survey has to incorporate components from each of the above, including a hierarchy to reconcile contradictory dispositioning information. For example, consider an individual that was sent a survey link in both a mailed postcard and an email. If a response is collected and one of those invites is returned with information, a decision must be made with regard to which information is more accurate. |
Survey Disposition
A survey disposition encodes the outcome of an interview.
Contents
Description
A survey disposition encodes the most important information about how an individual interacted with a survey instrument. The major categories are:
- Eligible respondents
- Eligible nonrespondents
- Sample members with unknown eligibility
- Ineligible sample members
There are then subcategories with finer detail, for example the reason a sample member is ineligible.
Coding
AAPOR has published guidance to standardize the encoding of dispositions.
Major categories are an integer part, while minor categories are a decimal part. These codes can be truncated at any level to collapse subcategories into parent categories. For example, codes 4.11 and 4.12 may convey different survey items whereby sample members have indicated their ineligibility. If any analyst only need to know which cases self-identified as ineligible through any survey item, the codes can be truncated to the tenths decimal place; 4.11 and 4.12 become 4.1.
When working with a sample of specifically-named individuals, there is generally a presumption of eligibility. It follows that:
- An explicit refusal to participate in a survey, either as declining to give consent or as requesting to be removed from a contact list, is generally coded as eligible nonresponse.
- An incomplete survey response, either due to item nonresponse or due to technical issues, is generally coded as eligible nonresponse.
The exception is when contact is attempted but not necessarily successful. This leaves open the possibility that the specifically-named individual has e.g. moved, and is no longer member to the eligible population. Mailed surveys will often have many subcategories of unknown eligibility, corresponding to the various nixie codes returned by the postal service:
- Undeliverable as addresses
- Refused by addressee
- Vacant
- Deceased
- and so on
Similarly, a survey link that is shared by email can lead to a variety of email bouncebacks.
If responses are collected from a nonprobability panel, individuals often must be considered as having unknown eligibility until they provide survey responses indicating otherwise.
Nonprobability panel surveys also often incorporate live or rebalancing quotas which cause 'surplus' respondents to be terminated as 'over-quota'. In most cases these are treated as eligible nonrespondents.
A multi-modal survey has to incorporate components from each of the above, including a hierarchy to reconcile contradictory dispositioning information. For example, consider an individual that was sent a survey link in both a mailed postcard and an email. If a response is collected and one of those invites is returned with information, a decision must be made with regard to which information is more accurate.