Researching with impact in the Global South? Impact-evaluation practices and the reproduction of ‘development knowledge’

Abstract

Long-standing questions about the production and control of knowledge about ‘the developing world’ have been given new urgency through the deployment of impact-evaluation practices within UK universities, highlighting the need for careful ethical reflection on the role of Northern researchers in both academia and practice. In this context, this article takes up the three underlying themes of this special issue – the conceptualisation, evaluation and methods of knowledge mobilisation – to ask what ‘researching with impact’ might mean for academics whose work focuses on the Global South. With regard to the conceptualisation of knowledge, it argues that the Research Councils UK's definition of ‘high impact’ research sits uncomfortably with both critical scholarship on the power of ‘development knowledge’ and with ‘alternative development’ practices that call for knowledge co-production. With regard to the evaluation of knowledge mobilisation, it uses Northern researchers' reflections on their practice to argue that impact-evaluation practices are ‘nudging’ academia in directions that require our attention. Finally, with regard to methods of knowledge mobilisation, it investigates what an ethically engaged response to these pressures might look like, arguing for scholars working on the Global South to defend the production of ‘development knowledge’ that is both practically engaged and critically distant from policy-makers.

Keywords:

• impact evaluation
• Global South
• international development
• knowledge exchange (politics of)
• research practice

Acknowledgements

I would like to thank all interviewees for their participation in the interview process and their frank and open engagement with the subject matter. Their reflections on their own experiences, and discussion with participants of the NCRM ‘Dancing with New Partners’ network, have been important in developing the arguments presented here. Thanks are also due to the referees for their swift and constructive comments, and to Tariq Jazeel and Paula Meth, who originally encouraged me to write about the impacts of impact evaluation practices on development research.

Notes on contributor

Glyn Williams is Senior Lecturer in the Department of Town and Regional Planning, University of Sheffield, where he is also a co-director of the Sheffield Institute for International Development. His main research interests are in the field of international development, and more specifically in the interaction between development programmes, governance practices, and citizenship in the Global South. He has conducted most of his research in India, and is co-author of Geographies of developing areas: The global south in a changing world (Routledge, 2009) with Paula Meth and Katie Willis, and Seeing the state: Governance and governmentality in rural India (Cambridge University Press, 2005) with Stuart Corbridge, Manoj Srivastava and René Véron.

Notes

1. There are perhaps particularly strong parallels between development studies and other academic disciplines that directly link to a related field of professional practice. For example, Leonie Sandercock's deconstruction of planning history mirrors critical histories of development outlined above, and her call for the recognition of multiple epistemologies of planning knowledge (Sandercock, 2003) has similarities to ‘alternative development’ struggles to establish knowledge co-production as the aim of critically-reflexive practitioners.

2. This happens, not least, through university research training, where staple elements of teaching from undergraduate dissertations onwards would include critical reflection on dominant framings of development questions within policy, the positionality of the researcher, and the ethics of using and sharing research findings.

3. The take-up of research ideas by peers within or beyond one's own discipline, or in teaching, does not count as ‘user engagement’ unless a special case can be made that this is part of a ‘critical pathway towards economic and societal impact’ (RCUK, 2010).

4. More fundamental critiques from Geographers have also argued that the REF's separation of research (as knowledge production) from education (as its consumption) represents a deepening commodification of academia (Slater, 2012), and that a state-regulated notion of impact closes down of the space for critical or creative scholarship (Jazeel, 2010).

5. All interviewees had been in academia for more than 10 years since completing their PhDs, and all had experience of submitting grant applications, of directing funded research in the Global South, and of project evaluation (for all, this included reviewing grant applications and/or end-of-project reports for the ESRC). Care was taken to ensure that the group included academics from a range of UK universities (both pre- and post-1992). This selection process allowed me to discuss within a short series of interviews the effects of impact evaluation practices on all aspects of the research process. As I have suggested elsewhere (Williams, 2012), less established researchers may find the requirements of demonstrating ‘impact’ more difficult to accommodate within their research practice, and it would therefore be useful to extend further interviews to early career academics. To protect their anonymity, details of interviewees' UK institutions and places of study within the Global South have been removed, and as have indications of their gender (although grammatically awkward, I use ‘their’ in preference to ‘his’/‘her’).

6. Interviewees did not necessarily reject ‘influencing policy’ as an aspiration – rather they objected to its (assumed) centrality to the definition of ‘world-leading’ impact, and the assumption that there was a casual or linear relationship that led from high-quality research insights, via effective communication with policy communities, to policy change. Gilbert (2012) provides an interesting counter-example of how ideological consonance – rather than theoretical or empirical quality – has lead to Hernando de Soto's research becoming ‘world-leading’ in its influence within the World Bank.

7. Financial and/or reputational benefits of association with Northern universities may be difficult to resist, particularly for Southern institutions whose long term future is insecure. Because of the potential power imbalances involved, a Southern institution's agreement to enter an international partnership may not therefore equate with that research supporting their own agenda or developing their capacity.

8. Summarised by interviewee D as ‘throw it out there, say that this is what we found in our research, have an open discussion about it, and … leave people to make of it what they will’.

9. An interesting example for further study would be the Dutch model of encouraging knowledge mobilisation by sharing good practice – this was reviewed and yet rejected within background research for REF's own approach to impact (Grant, Brutscheer, Kirk, Butler, & Wooding, 2009).