Researching impact of mobile phones for development: concepts, methods and lessons for practice

Abstract

The use of mobile phones is an increasingly important part of development efforts particularly in low income countries. Investment in mobiles for development (m-development) is being led by the mobile phone industry together with donors and development partners across a wide range of development sectors. As adoption of mobile phones increases it becomes important to research and understand their impact, and to assess to what extent the expansion of m-development is assisting broader development efforts. This paper provides guidance for conducting research on impact assessment for m-development by reviewing 18 published impact studies, and suggests four key building blocks for good practice in impact assessment. These building blocks encompass: the extent to which the studies address the needs of defined audiences or beneficiaries, the choice of types of impact to assess and units of assessment, the application of suitable conceptual frameworks, and the rigor of the methodology followed. The findings suggest that m-development impact assessment to date falls into three main categories: purely quantitative approaches rooted in information economics, and purely qualitative approaches rooted in social impact assessment. A further mixed method category provides cost effective and timely findings, but with the result of diluting the rigor evident at the methodological poles. Overall, the paper concludes there is a need for more and better guidance for the conduct of such studies, in relation to identifying the rationale and objectives, and linking the choice of concepts and methods with the requirements of key audiences and beneficiaries. The paper identifies m-development impact research as a contested area, and by taking stock of experience thus far, seeks to raise the level of debate concerning the relative merit of alternative methodological and conceptual approaches.

Keywords:

• mobile phones
• impact assessment
• research methodology
• developing countries

Notes

Doug Vogel is the accepting Associate Editor for this article.

For a recent overview of m-finance literature refer to Duncombe (2009).

Refer to the Grameen Village Phone website for further information: http://www.grameenfoundation.org/what_we_do/technology_programs/village_phone/heritage/.

Details of a wide range of text-based m-development projects can be found at Kiwanja.net. http://www.kiwanja.net/database/kiwanja_search.php. Alternatively lists of projects can also be accessed via MobileActive.org. http://mobileactive.org/directory.

For a summary of mobile phone projects in international and developing country NGOs see Kinkade and Verclas (2008).

For an analysis of the reasons for IS/ICT project failure in developing countries, see Heeks (2002).

The review was conducted by the single author. This means that the interpretation and categorization of the methodological and conceptual approaches of the studies is in line with the author's perception of the field. In this sense, the paper is, to some extent, reflective of the author's point of view, and should not be read as a completely objective view of the field of study.

On-line searches were conducted accessing a broad range of databases from within the social sciences – incorporating a broad range of disciplines – Economics, Banking and Finance, Development Studies, Business and Management Studies, as well as more specialized disciplines – Information Systems and Information and Communication Technologies for Development (ICT4D). Databases searched were: ABI-Inform (ProQuest), EBSCO Business Source Premier, Emerald Fulltext and Science Direct as well as more general searches using both Google and GoogleScholar. Additionally, a number of websites specializing in the dissemination of research concerning mobile phones and development were searched (kiwanja.net/dgroups.org/mobileactive.org). Studies were included in the review on the basis of the author's perception of their importance to the field of study, and it is acknowledged that some relevant studies may have been omitted, although it is the author's belief that all major studies up until December 2008 have been included.

This example is drawn from a case study of m-development application for the cattle farming sector in Kenya. Full details are available from Kithuka, Mutemi, and Mohamed (2007).

Provision of “detailed” method guidance (typically covering two to three pages of notes) suggests that there was sufficient detail to be able to replicate the study. “Some detail” (typically less than one page of notes) indicates that while some areas of methodology were covered sufficiently, there were also gaps. “Limited” (typically one paragraph or less) suggests lack of guidance.

For example, on Google Scholar, Jensen's article had received 154 citations (recorded on 21 June 2010).

A detailed exposition of the assessment of ICT pilot projects incorporating an approach that emphasizes the early integration of stakeholder views is laid out by Batchelor and Norrish (2005).