Survey measures of democratic attitudes and social desirability bias
Survey measures of democratic attitudes and social desirability bias (DOI: https://doi.org/10.1017/psrm.2025.10044) was written by Pedro C. Magalhães and Laurits Florang Aarslew in 2025. It was published in Political Science Research and Methods.
The authors analyze the ESS for social desirability bias in their measures for support of democracy. In the context of the ESS, they look for a mode effect between the baseline (face-to-face interviews) and 'self-completion' (web or paper surveys).
There are several questions relating to support for democracy fielded in both round 6 and 10:
- one question asks about specific support, i.e. support for democracy in their country
- one question asks about diffuse support, i.e. how important it is to live in a democracy
- set of questions asks about conceptions of democracy, i.e, how important something is for democracy
- e.g., free and fair elections
- all questions are 11-point scales
- fielded only in round 10: support for autocracy, i.e. how acceptable is having a strong leader above the law
First, they exploit a mode change that was forced by the COVID pandemic in the 10th round of the ESS. This occurred in Cyprus, Germany, Israel, Poland, Spain, and Sweden. They use a difference-in-differences framework to examine the potential 'treatment' effect, repeated for each of the measures common across rounds 6 and 10. They used weighted regressions, with country and round fixed effects. Individual controls were gender, age, age squared, educational attainment, income perception. Country controls were GDP per capita, control of corruption, and the Gini coefficient. GDP per capita is measured in 2017 values; i.e., adjusted for purchasing power parity. Standard errors are clustered by country.
The authors do find a statistically significant effect, although it is small.
Second, the authors use a multi-modal experiment also in the 10th round. This was conducted in Great Britain and Finland. The samples for each mode were selected with comparable methods. Main differences were:
- target audience of self-completed survey is 18+
- sample of self-completed survey is not clustered by address
- self-completed survey not fielded in Northern Ireland
Here the authors directly estimate a mode effect as difference in predicted means. They use a weighted regression model repeated for each measure, including the one unique to round 10. (They expect any measured bias to be reversed on the autocracy question.)
The results are inconsistent; not all measures are found to feature a significant effect. In Finland, the estimated (insignificant) effects are not even found to be of the same sign. The estimated (significant) effect on the autocracy measure is not oppositely signed.
Third, the authors embed original data collection in wave 6 of the Portuguese CROss-National Online Survey (CRONOS). They utilize a double list experiment as an indirect measure of support for autocracy. The sensitive statement is: "the government should be able to ignore court rulings that are regarded as politically biased". To estimate the bias that this experiment should have avoided, they also directly ask for agreement with the statement.
The authors do not find a statistically significant effect.