When are experiments corrupt?

ABSTRACT

Between 2010 and 2011, the UK’s Department for International Development (DFID) funded a large-scale experiment on Kenyan schools. The policy experiment tested the effects of cutting teachers’ salaries and hiring them on short-term contracts, but the intervention failed after sustained opposition from teacher unions and parent associations. This article critically revisits the narrative of how this experiment was designed, implemented and interpreted, finding evidence that the experiment violated empirical logic. It examines whether the theory of neopatrimonialism can explain the ways in which vested interests may have undermined the empirical logic of the experiment. By doing so, the analysis tests the explanatory utility of neopatrimonialism, and casts light on the conditions under which policy experiments in Africa may be anti-empirical.

KEYWORDS:

• Randomized controlled trials
• impact evaluation
• social policy
• corruption
• neopatrimonialism

Disclosure statement

No potential conflict of interest was reported by the author.

Notes on contributor

Nimi Hoffmann is a postdoctoral fellow at the Institute of Social and Economic Research, Rhodes University, where she works onsocial policy and intellectual history.

ORCID

Nimi Hoffmann http://orcid.org/0000-0001-9527-3324

Notes

1 This figure should be treated with some caution and is likely an upper-bound estimate. Only 46% of the studies discussed the question of informed consent. And of these, 19% were unclear as to whether the participants gave consent to be part of an experiment, rather than gave consent to be part of a social programme.

2 This data was gleaned from the appendix, which contained a list of all the papers reviewed. I identified the current institutional location of first authors by conducting a Google search, and then coded the institution by country and then region.

3 The references in this article are from the authors’ working paper from 2013, which was then published in 2018.

4 Calculated using the World Bank’s estimate of a PPP private consumption conversion factor of 35.43 for Kenya in 2011, accessed at: https://data.worldbank.org/indicator/PA.NUS.PRVT.PP?locations=KE.

5 Put in international perspective, Kenya’s GDP per capita is US$2.71 per day in PPP dollars. A teacher would therefore earn 0.7 times the per capita GDP. In contrast, teachers earn 1.8 times per capita GDP in OECD countries, and 2.5 times per capita GDP in middle-income countries (Mulkeen 2010, 152).

6 One might respond that lower wages for black people or women is a case of otherwise irrelevant identity markers being used as proxies of competence, such that they lead to inefficient hiring outcomes. On this view, labour market discrimination is defined by the misapplication of identity markers. Thus, differences in wages between black and white people count as a form of discrimination, but differences in wages between permanent and contract teachers do not. Instead, these differences are a form of efficiency improving measures, since they seek to bring down government wages to private sector levels. But consider a situation in which black workers in the public sector receive a higher wage than black workers in the private sector (as is the case in South Africa, see Bhorat et al. 2015). One might argue that black civil servants receive a higher wage because the government is compelled to do so by a largely black electorate and strong public sector unions, whereas black workers in the private sector continue to be subject to discrimination in the largely white-controlled private sector. In this scenario, setting private sector wages for black civil servants would therefore undermine democratic gains and reintroduce white racism into the state sector. Similarly, teachers in the Kenyan public system might receive higher wages because the state is compelled to pay salaries that meet minimum livelihood thresholds by democratic institutions, such as unions and courts, whereas the private sector is less encumbered by democratic oversight. The point is not that higher civil service wages are necessarily more just, but that further contextual evidence and theoretical argument is required to adjudicate between these different interpretations.

7 Since the precise nature of this anti-empiricism is difficult to convey, the literature on neopatrimonialism often relies on ‘detailed descriptions and … imaginative portrayals of a local scene, replete with a kind of anthropological knowingness … [it] is full of vibrant metaphors and characterized by unbridled use of anecdotes, pejorative vocabulary … and vivid vignettes of the all-too-frequent cases of egregious abuse of state resources and power' (Mkandawire 2015, 564). While this may strengthen the rhetorical force of the argument, it does nothing to justify its empirical validity.

8 The concept of a ‘living laboratory’ has been used to understand the kinds of scientific knowledge used in British colonial administration (Tilley 2011). The term is drawn from Lord Hailey’s opening remarks from the African Research Survey (1929–1938) which was conducted in an effort to better understand, govern and ‘develop’ African societies in the tropics. ‘Africa presents itself as a living laboratory’, Hailey wrote, ‘in which the reward of study may prove to be not merely the satisfaction of an intellectual impulse, but an effective addition to the welfare of the people.’ (1938, xxiv–xv) However, the allied concept of champs d’experience, or experimental terrains, has also been applied in the context of French colonial rule, especially in the Maghreb (Celik 1997).