![Sample our Politics & International Relations journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5947/TOC-Subject-Sample-2021-Politics-International-Relations.jpg"})
Abstract
Vladimir Putin has managed to achieve strikingly high public approval ratings throughout his time as president and prime minister of Russia. But is his popularity real, or are respondents lying to pollsters? We conducted a series of list experiments in early 2015 to estimate support for Putin while allowing respondents to maintain ambiguity about whether they personally do so. Our estimates suggest support for Putin of approximately 80%, which is within 10 percentage points of that implied by direct questioning. We find little evidence that these estimates are positively biased due to the presence of floor effects. In contrast, our analysis of placebo experiments suggests that there may be a small negative bias due to artificial deflation. We conclude that Putin’s approval ratings largely reflect the attitudes of Russian citizens.
Acknowledgements
We gratefully acknowledge financial support from the National Science Foundation grant number SES 13424291, “Voter Mobilization and Electoral Subversion in the Workplace.” We also thank members of the Experimental Politics Workshop at the University of Wisconsin-Madison and the 2015 PONARS Policy Conference for comments. Replication materials are available at the Harvard Dataverse, http://dx.doi.org/10.7910/DVN/ZJQZH5.
Notes
1. It is worth noting that public attitudes toward President Putin along other dimensions are somewhat less rosy. For example, in our March 2015 survey, when asked to name the five or six politicians in Russia whom they most trust, only 62% of respondents included President Putin in this list. Similarly, in a June 2015 survey by the Levada Center, 66% of respondents indicated that they would like to see President Putin retain his post after the next round of elections (http://www.levada.ru/24-06-2015/vybory-gotovnost-golosovat-perenos-elektoralnye-predpochteniya). We see no a priori evidence in the responses to these questions – which measure attitudes about trust and reelection, not approval – that Putin is less popular than suggested by opinion polls.
2. Treisman (Citation2014) finds that Putin increasingly lost the confidence of two groups during this period: those dissatisfied with the state of the Russian economy and those with a negative attitude toward the West.
3. See Colton and Hale (Citation2013) and “Public Opinion in Russia: Russians’ Attitudes on Economic and Domestic Issues,” Associated Press-NORC Center for Public Affairs Research, accessed at http://www.apnorc.org/projects/Pages/public-opinion-in-russia-russians-attitudes-on-the-economic-and-domestic-issues.aspx.
4. The Levada Center polls cited here use face-to-face interviews at the home of the respondent rather than phone interviews.
5. The exact phrasing was “Do you think that people who are critical of the authorities in public opinion polls can be persecuted by the authorities for these opinions?” See http://www.levada.ru/18-08-2014/oprosy-obshchestvennogo-mneniya-interes-doverie-i-strakhi.
6. Author-commissioned survey. Note, however, that this question did not ask specifically about opinions expressed in public opinion surveys. The exact phrasing was “What do you think about the opinion that people in Russia try to hide their political views in order to avoid troubles with the authorities?”.
9. For consistency with the direct questions for other political figures, we omitted the phrase “as president of Russia” from the standard Levada question that gauges support for Putin. In practice, the two versions of the question (which both appear on each survey instrument) produce nearly identical responses.
10. In the January survey instrument, the direct questions follow the list experiments, whereas in March the direct questions come first. As we demonstrate below, our results are quite robust to this change in question ordering.
11. In the January survey, in addition to the seven political figures mentioned above, we directly asked about support for former Finance Minister Alexei Kudrin, Russian oligarch Mikhail Prokhorov, Patriarch Kirill, Belorussian President Aleksandr Lukashenko, Nelson Mandela, and Fidel Castro. The only negative correlations, all very small in magnitude, were between Stalin and Prokhorov (r = −0.02), Brezhnev and Prokhorov (r = −0.01), and Yel’tsin and Putin (−0.02).
12. In the January survey, roughly 20% of respondents indicate that they support Putin but not Stalin, Brezhnev, or Yel’tsin, whereas 27% of respondents indicate that they support Putin but not Zyuganov, Zhirinovsky, or Mironov.
13. Adopting the framework of Imai (Citation2011), the two scenarios described in this paragraph involve violations of the assumptions of “no design effect” and “no liars,” respectively.
14. A more analytic description of this form of deflation is as follows. Assume that the probability that any individual indicates support for a political figure is reduced by p in a list, relative to the individual’s actual support. The expected count is thus “too low” by Np, where N is the length of the list. Estimating support by subtracting the mean response for the control group from the mean response for the treatment group therefore results in an underestimate of support (in expectation) of (J + 1)p − Jp = p, where J is the number of items in the control group. It is worth noting that this is not an idle concern; as Tsuchiya and Hirai (Citation2010) report, this type of bias has been observed in a number of published studies that use the item-count technique.
15. As an additional strategy, we followed Tsuchiya and Hirai (Citation2010) in randomly assigning half the respondents to the January survey to receive a version of each list experiment in which they were asked not only how many political figures they support, but also how many they do not support. For both the historical and contemporary lists, estimates of support for Putin and the underlying mean responses were nearly identical in the two versions of the experiment. As we are unable to distinguish between an absence of artificial deflation and a failure of Tsuchiya and Hirai’s strategy to correct for artificial deflation in our setting, we employed only the standard version of the list experiment in the March 2015 survey. In our analyses, we pool results from the two versions of the January 2015 experiments.
16. Appendix Table A1 provides the full distribution of responses for each of the four experiments.
17. Our estimates of support for Putin are substantially higher than those of Kalinin (Citation2015), who also uses the item-count technique in nationally representative surveys of the Russian population. Three distinctions in our respective experimental designs are relevant in explaining the different results. First, our list experiment mentions Vladimir Putin by name, as in the direct question used by the Levada Center. In contrast, Kalinin asks about approval “of the job of the President of the Russian Federation,” which may also capture approval of the government – an institution far less popular than Putin. Second, our list includes only three nonsensitive items, whereas Kalinin’s includes four. Longer lists may be harder to remember, potentially biasing results. Third, the nonsensitive items in Kalinin’s list experiment include a heterogeneous mix of opinions and factual statements (medical care should be free, in our family we have a car, environmental issues are a priority for me, I am satisfied with the level of my income) alongside a potentially sensitive political attitude. Our research design adopts the more typical practice of including nonsensitive items that are similar to the construct being measured.
18. We calculate confidence intervals for social desirability bias using the list package (Blair and Imai Citation2011; Citation2012).
19. If we assume that this is true of all treatment-group respondents who indicated support for precisely one political figure, then estimated support for Putin in the January experiments drops to 47% in the historical experiment and 44% in the contemporary experiment. These sharp lower bounds are derived by recalculating the mean count for members of the treatment group under the proposed assumption and subtracting the mean count for members of the control group. Similar results apply to the March experiment, although in that round of the survey the proportion of treatment-group respondents indicating support for precisely one political figure was greater in the historical than contemporary experiment (40 vs. 37%, respectively).
20. Graphics created with the package ggplot2 (Wickham Citation2009).
21. In a simple linear regression of the number of figures supported in the list experiment on (a) treatment status, (b) the number of control-group figures supported in the direct questions, and (c) their interaction, we find no significant interaction effect for any of the four experiments illustrated in Figure . The effect of including Putin in the list is largely limited to raising the intercept of the regression line. Full results are available in Appendix Table A2.
22. As discussed above, one form of artificial deflation involves a violation of the assumption of “no design effect,” that is, the assumption that responses to control items are unaffected by the inclusion of the sensitive item (Imai Citation2011). Blair and Imai (Citation2012) provide a test for design effects, the essence of which is to check that, on average, (1) scores in the treatment condition are not lower than those in the control condition, and (2) scores in the treatment condition are not more than one plus the scores in the control condition. Using this test, we reject the null of no design effect for the January historical experiment but not the other three experiments. Clearly, however, design effects may be present without these two conditions having been violated. Given the conservative nature of Blair and Imai’s test, we proceed to examine other evidence for artificial deflation.
23. For the Castro experiment, the mean number of political figures supported among Lukashenko, Merkel, and Mandela is 1.32 among control-group respondents when they are asked directly, vs. 1.14 when they are presented with a list of the same individuals. In the March contemporary Putin experiment, the mean number of political figures among Zhirinovsky, Zyuganov, and Mironov that control-group respondents directly support is 1.28, vs. 1.13 when they are presented with a list of the same individuals.