https://orcid.org/0000-0002-7934-9832
![Sample our Politics & International Relations journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5947/TOC-Subject-Sample-2021-Politics-International-Relations.jpg"})
ABSTRACT
Scholars of conflict often rely on fieldwork to study behaviours of civilians and combatants on the ground. A list experiment is a potentially useful tool for conflict scholars, as this survey methodology is designed to indirectly obtain truthful self-reports of behaviours while preserving the respondents’ anonymity. Acknowledging its advantages, this article also reviews the often overlooked shortcomings of list experiments as a survey method in conflict zones, including those limitations that cannot be corrected with better design or implementation. As an illustration, we discuss the list experiment employed to measure civilian assistance to the insurgents in the Donbas War.
Acknowledgments
We thank Josh M. Ryan for his helpful comments on the earlier version of this manuscript. We are also grateful to all the scholars who have shared with us their experiences regarding the use of list experiments.
Disclosure statement
No potential conflict of interest was reported by the authors.
Notes
1. We are grateful to all the scholars who have generously shared with us their experiences regarding the use of list experiments. We thank the Kyiv International Institute of Sociology (KIIS) of Kyiv, Ukraine, and especially Liana Novikova, Victoria Zakhozha, and the interviewers who conducted the survey for their excellent work and helpful feedback. We thank the editor and the anonymous reviewers for very helpful comments. We are also grateful to Josh M. Ryan for his valuable feedback on the earlier version of this manuscript. The online appendix and the replication materials are available upon request from the corresponding author. This research was approved by Carnegie Mellon's Office of Research Integrity and Compliance.
2. For an overview of alternative methods that may help elicit truth in response to sensitive questions, see Corstange (Citation2009, p.47–48) and Rosenfeld et al. (Citation2015, p.783–784).
3. See Blair et al. (Citation2015) for a review of other variants of the randomised response technique.
4. Droitcour et al. (Citation1991) also report that their respondents left notes on the paper surveys which revealed that respondents often circled the answer that corresponded to the order of the applicable item, while the survey asked them to indicate the sum of applicable items.
5. Another major downside of the list is stating the sensitive item directly so that respondents know the topic of interest to researchers, see Section 2.2 for a detailed discussion of this point.
6. Glynn (Citation2013) offers a similar test of estimating proportions, but his version does not account for multiple testing.
7. In many cases respondents were unable to attribute which side of the conflict damaged their property. The most common types of reported damage were broken windows, blown out doors and other damage to housing.
8. The appendix details the randomisation procedure.
9. The standard errors were .039 and .038 in the control and treatment groups, respectively.
10. The standard errors were .040 and .040 in the control and treatment groups, respectively.
11. This approach is similar to Glynn’s (Citation2013) diagnostics test, but it has an advantage of correcting for multiple testing.
12. See also Chou (Citation2018) for combining the list with direct questioning applied only to the control group.
13. We have also investigated the impact of covariates on respondents’ probability of misreporting based on Eady’s (Citation2017) method, which leverages the discrepancies between the list and direct answers. No statistically discernible effects of covariates on one’s propensity to misreport have been detected.
14. CEM matches a subject with a treatment with a subject (or a few subjects) without a treatment with identical (or the most similar) covariates. In all models, all covariates are better balanced after matching. Unlike the propensity score matching which approximates the means of the covariates’ distributions, CEM accounts for the entire empirical distributions of covariates (CitationKing and Nielsen N.d.).
15. For a two-tailed test, it would have been necessary to survey 40,048 or 8,724 individuals.
16. See Zhukov (Citation2016) for an explanation of how strong economic ties to Russia explain this region’s strong pro-Russian political affiliation.
17. The latter two types of offences are based on part 1 of article 258–2 and part 1 of article 258–5 respectively.
18. We relied on the local news the RBC-Ukrayina website. This news portal publishes stories directly from the National Security Service’s website, but has a more convenient archive to search compared to the National Security Service’s website.
Additional information
Funding
Notes on contributors
Anna O. Pechenkina
Anna O. Pechenkina is an assistant professor of Political Science at Utah State University. Her research focuses on conflict and political violence.
Andrew W. Bausch
Andrew W. Bausch is a data scientist at Murmuration. His research looks at experimental design and survey methods.
Kiron K. Skinner
Kiron K. Skinner is the Taube Professor of International Relations and Politics at Carnegie Mellon University and a Research Fellow at Stanford University’s Hoover Institution. Her research focuses on foreign policy and presidential politics.