Policy Evaluation in Polarized Polities: The Case of Randomized Controlled Trials

Figures & data

Figure 1. Issue salience: a media content analysis.

Figure 2. Issue polarization and the timing of the RCT: a media content analysis.

Table 1. Experiments and the government’s vote share margin

	3ie data – Binary DV		J-PAL data – Binary DV
	All RCTs		All RCTs		State-sponsored RCTs
	LPM	LPM	LMP	LPM	LPM	LPM
	(1)	(2)	(3)	(4)	(5)	(6)
margin of votes ${}_{t-1}$	0.000	0.005***	−0.004***	0.013***	−0.002**	0.006*
	(0.00)	(0.00)	(0.00)	(0.00)	(0.00)	(0.00)
ethnic fractionalization	0.163**		0.071***		0.052***
	(0.08)		(0.02)		(0.02)
vote margin ${}_{t-1}$ × fractionalization		−0.011***		−0.034***		−0.017**
		(0.00)		(0.01)		(0.01)
ln(gdppc) ${}_{t-1}$	−0.030***	0.226***	−0.003	−0.002	0.002	−0.013
	(0.01)	(0.08)	(0.00)	(0.02)	(0.00)	(0.01)
ln(population) ${}_{t-1}$	0.041***	0.323*	0.012***	0.207***	0.010***	0.109*
	(0.01)	(0.19)	(0.00)	(0.08)	(0.00)	(0.06)
lagged dependent variable	0.404***	0.096**	0.807***	0.614***	0.806***	0.623***
	(0.04)	(0.04)	(0.03)	(0.04)	(0.04)	(0.06)
unconditional mean of DV	0.188	0.188	0.098	0.098	0.051	0.051
Country FE	no	yes	no	yes	no	yes
Year FE’s	yes	yes	yes	yes	yes	yes
joint F-stat p-value	–	0.002	–	0.001	–	0.092
within R^2	–	0.118	–	0.418	–	0.414
N	2177	2177	1649	1649	1649	1649
Countries	156	156	128	128	128	128

Notes: Robust standard errors clustered by country are in parentheses. Columns 1–2 consider a binary DV from the 3ie data that takes value one if the 3ie repository recorded any kind of RCT in a country/year. Columns 3–4 consider a binary DV from the JPAL data that takes value one if the JPAL repository recorded an kind of RCT in a country/year. Columns 5–6 consider a binary DV that takes value one if the JPAL repository recorded a state-sponsored RCT in a country/year.

***/**/* represent significance at the 1/5/10 percent level.

Figure 3. Marginal effect of an increase in the margin of votes on the probability of having an experiment as a function of ethnic fractionalization.

Table 2. Experiments and the government’s vote share margin

	3ie data – Binary DV
	LPM		logit
	low polarization	high polarization	low polarization	high polarization
	(1)	(2)	(3)	(4)
margin of votes ${}_{t-1}$	0.003***	−0.002**	0.025**	−0.021***
	(0.00)	(0.00)	(0.01)	(0.01)
ln(gdppc) ${}_{t-1}$	0.320**	0.129	1.797*	3.223**
	(0.14)	(0.09)	(1.06)	(1.27)
ln(population) ${}_{t-1}$	0.333	0.262	1.225	2.507
	(0.33)	(0.22)	(3.15)	(2.86)
lagged dependent variable	0.105*	0.092	−0.135	−0.036
	(0.06)	(0.06)	(0.32)	(0.24)
Country FE	yes	yes	yes	yes
Year FE’s	yes	yes	yes	yes
within R^2	0.143	0.112	–	–
N	1085	1127	546	707
Countries	71	88	35	54

Notes: Robust standard errors clustered by country are in parentheses.

***/**/* represent significance at the 1/5/10 percent level.

Appendix. Summary statistics for the cross-country panel

variable	N	mean	std. dev.	min	max
randomized controlled trial (count, 3ie)	2,177	0.557	2.096	0	35
randomized controlled trial (binary, 3ie)	2,177	0.188	0.391	0	1
randomized controlled trial (binary, J-PAL)	1,699	0.098	0.298	0	1
state-sponsored RCT (binary, J-PAL)	1,699	0.051	0.220	0	1
margin of votes (DPI)	2,177	32.577	27.010	−20.380	100
ethnic fractionalization index (Alesina)	2,177	0.413	0.243	0	0.908
gross domestic production per capita (logged, WDI)	2,177	8.611	1.556	4.751	11.626
population (logged, WDI)	2,177	15.785	1.727	11.515	21.028

Notes on sources. Alesina: Alesina, Baqir, and Easterly (1999), DPI: Cruz et al. (2016), WDI: Hollweg et al. (2019), J-PAL: Corduneanu-Huci et al. (2021).

Alesina, A., Baqir, R., & Easterly, W. (1999). Public goods and ethnic divisions. The Quarterly Journal of Economics, 114(4), 1243–1284. doi:10.1162/003355399556269

 Web of Science ®Google Scholar

Cruz, C., Keefer, P., & Scartascini, C. (2016). Database of political institutions codebook, 2015 update (dpi2015) (pp. 165–176). Washington, DC: Inter-American Development Bank.

 Google Scholar

Hollweg, C. H., Sáez, S., Aguiar, A., Walmsley, T., Narayanan, G.,B., Aguiar, A., … Mattoo, A., et al. (2019). World Development Indicators (database). Washington, DC: World Bank.

 Google Scholar

Corduneanu-Huci, C., Dorsch, M. T., & Maarek, P. (2021). The politics of experimentation: Political competition and randomized controlled trials. Journal of Comparative Economics, 49(1), 1–21. doi:10.1016/j.jce.2020.09.002

 Web of Science ®Google Scholar

Data availability statement

The data that support the findings of this study are available from the corresponding author, MTD, upon reasonable request.