Building theory in ICT4D evaluation: a comprehensive approach to assessing outcome and impactFootnote^*

* Mathias Hatakka is the accepting Associate Editor for this article.

References

• Adam, L., & Wood, F. (1999). An investigation of the impact of information and communication technologies in sub-Saharan Africa. Journal of Information Science, 25(4), 307–318. doi:10.1177/016555159902500407
   Web of Science ®Google Scholar
• Ahsan, A., & Raihan, A. (2013). Understanding mHealth impact among Aponjon (MAMA Bangladesh) subscribers through a Phone Survey in Bangladesh. Paper presented at proceedings of the sixth international conference on information and communications technologies and development: Notes – Volume 2 (ICTD ‘13), (Vol. 2, pp. 1–4). New York, NY: ACM. doi10.1145/2517899.2517920
   Google Scholar
• Alampay, E. (2006). Analysing socio-demographic differences in the access and use of ICTs in the Philippines using the capability approach. Electronic Journal of Information Systems in Developing Countries, 27(5), 1–39. Retrieved from http://www.ejisdc.org/ojs2/index.php/ejisdc/article/viewFile/204/182 doi: 10.1002/j.1681-4835.2006.tb00182.x
   Google Scholar
• Ali, M., & Bailur, S. (2007, May). The challenge of sustainability in ict4d- is Bricolage the answer? Paper presented at the proceedings of the 9th international conference on social implications of computers in developing countries. Sao Paulo, Brazil . Retrieved from http://www.ifipwg94.org.br/fullpapers/R0010-2.pdf
   Google Scholar
• Alkire, S., & Santos, E. A. (2010). Acute multidimensional poverty: A new index for developing countries (Human development research paper). United Nations Development Programme. Retrieved from http://hdr.undp.org/en/reports/global/hdr2010/papers/HDRP_2010_11.pdf
   Google Scholar
• Alsop, R., Bertelsen, M. F., & Holland, J. (2006). Empowerment in Practice: From analysis to implementation. Directions in Development. doi:10.1596/978-0-8213-6450-5
   Google Scholar
• Ashley, C., & Maxwell, S. (2001). Rethinking rural development. Development Policy Review, 19, 395–425. doi:10.1111/1467-7679.00141
   Google Scholar
• Bailur, S. (2007). Using stakeholder theory to analyze telecenter projects. Information Technologies and International Development, 3(3), 61–80. doi:org/10.1162/itid.2007.3.3.61
   Google Scholar
• Balancing Act. (2013). Rescue attempt on poorly performing telecentres in South Africa. Telecoms, Internet and Broadcast in Africa-Balancing Act, 340, Retrieved from http://www.balancingact-africa.com/news/en/issue-no-340/telecoms/rescue-attempt-on-po/en
   Google Scholar
• Baskerville, R., & Pries-Heje, J. (2010). Explanatory design theory. Business & Information Systems Engineering, 2, 271–282. Retrieved from http://aisel.aisnet.org/bise/vol2/iss5/2 doi: 10.1007/s12599-010-0118-4
   Web of Science ®Google Scholar
• Batchelor, S., & Norrish, P. (2005). Framework for assessment of ICT pilot projects: Beyond monitoring and evaluation to applied research. Information and Development Programme. Retrieved from InfoDev website: http://www.infodev.org/infodev-files/resource/InfodevDocuments_4.pdf
   Google Scholar
• Brundtland, G. (1987). United Nations world commission on environment and development: our common future. Oxford: Oxford University Press.
   Google Scholar
• Brunet, P., Tiemtoré, O., & Vettraino-Soulard, M. C. (2004). Ethics and the Internet in West Africa: Toward an ethical model of integration. Ottawa, CA: IDRC.
   Google Scholar
• Brydon, L. (2006). Ethical practices in doing development research. In V. Desai & R. Potter (Eds.), Doing development research (pp. 25–33). London: Sage.
   Google Scholar
• Chambers, R., & Conway, G. R. (1991). Sustainable rural livelihoods; practical concepts for the 21st century. (Paper 296). Retrieved from Institute of Development Studies (IDS) Discussion website: http://opendocs.ids.ac.uk/opendocs/bitstream/handle/123456789/775/Dp296.pdf?sequence = 1
   Google Scholar
• Clemens, M. A., & Demombynes, G. (2010). When does rigorous impact evaluation make a difference? The case of the Millennium Villages (Working Paper No. 225). Centre for Global Development. Retrieved from http://www.cgdev.org/files/1424496_file_Clemens_Demombynes_Evaluation_FINAL.pdf
   Google Scholar
• DeLone, W. L., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30. Retrieved from http://dl.acm.org/citation.cfm?id=1289765.1289767 doi: 10.1080/07421222.2003.11045748
   Web of Science ®Google Scholar
• Department of International Development. (1999). Sustainable livelihood guidance sheet. (Section 2). Retrieved from http://www.eldis.org/vfile/upload/1/document/0901/section2.pdf
   Google Scholar
• Department for International Development (DFID). (2005). Monitoring and evaluating Information and Communication for Development (ICD) programmes: Guidelines. Information and Communication for Development. Retrieved March 19, 2012 from http://www.webarchive.nationalarchives.gov.uk/http://www.dfid.gov.uk/pubs/files/icd-guidelines.pdf
   Google Scholar
• Duncombe, R. (2007). Using the livelihoods framework to analyze ICT applications for poverty reduction through microenterprise. Information Technologies and International Development, 3(3), 81–100. Retrieved from http://itidjournal.org/itid/article/viewFile/231/101 doi: 10.1162/itid.2007.3.3.81
   Google Scholar
• Ei Chew, H., Ilavarasan, V., & Levy, M. (2013). When there’s a will, there might be a way: The economic impact of mobile phones and entrepreneurial motivation on female-owned microenterprises. Paper presented at the proceedings of the sixth international conference on information and communication technologies and development: Full Papers – Volume 1 (ICTD ‘13) (Vol. 1, pp. 196–204). New York, NY: ACM. doi:10.1145/2516604.2516605
   Google Scholar
• Figueroa, M. E., Kincaid, D. L., Rani, M., & Lewis, G. (2002). Communication for social change: An integrated model for measuring the process and its outcomes. (Working Paper No.1). Retrieved from http://www.communicationforsocialchange.org/pdf/socialchange.pdf
   Google Scholar
• Gigler, B. S. (2004, September). Including the excluded: Can ICTs empower poor communities? Towards an alternative evaluation framework based on the capability approach. Paper presented at the 4th international conference on the capability approach. University of Pavia, Italy.
   Google Scholar
• Gigler, B. (2015). Development as freedom in a digital age. Experience of the rural poor in Bolivia. Washington: World Bank Group.
   Google Scholar
• Gomez, R., & Pather, S. (2012). ICT Evaluation: Are we asking the right questions? Electronic Journal of Information Systems in Developing Countries, 50(5), 1–14. Retrieved from http://www.ejisdc.org/ojs2/index.php/ejisdc/article/view/879/393
   Google Scholar
• Gomez, R., Reed, P., & Chae, H. Y. (2013, December). Assessment of community wellness outcomes to measure ICT impact. Work in progress. Proceedings of ICTD 2013, Cape Town, South Africa.
   Google Scholar
• Gregory, R. W., & Muntermann, J. (2011, January). Theorizing in design science research: Inductive versus deductive. Paper presented at the 32nd international conference on information systems. Shanghai, China.
   Google Scholar
• Grunfeld, H. (2007, December). Framework for evaluating contributions of ICT to capabilities, empowerment and sustainability in disadvantaged communities. Indian Institute of Technology (IIT). Paper presented at the CPRSouth2 (Communication Policy Research) conference, ‘Empowering rural communities through ICT policy and research’, Madras, Chennai.
   Google Scholar
• Hallikainen, P., & Chen, L. (2005). A holistic framework on information systems evaluation with a case analysis. The Electronic Journal Information systems Evaluation, 9(2), 57–64. Retrieved from file://C:/Users/g08m1052/Downloads/ejise-volume9-issue2-article764.pdf
   Google Scholar
• Hamel, J. (2010). ICT4D and the human development and capabilities approach: the potentials of information and communication technology (Report No. 37). Retrieved from http://mpra.ub.uni-muenchen.de/25561/
   Google Scholar
• Heeks, R. (2005). Foundation of ICTs in development: the information chain. eDevelopment Briefing, 3(1), 1–2.
   Google Scholar
• Heeks, R. (2006). Theorizing ICT4D research. Information Technology and International Development, 3(3), 1–4.
   Google Scholar
• Heeks, R. (2009). The ICT4D 2.0 manifesto: Where next for ICTs and international development (Working Paper 42)? Manchester Centre for Development Informatics. Development Informatics Group University of Manchester, UK.
   Google Scholar
• Heeks, R. (2014). ICT4D 2016: New priorities for ICT4D policy, practice and WSIS in a post-2015 world. Development Informatics Working Paper Series, No. 59/2014. Manchester: Institute for Development Policy and Management. Retrieved June 20, 2014. from http://www.seed.manchester.ac.uk/subjects/idpm/research/publications/wp/di/di-wp59/
   Google Scholar
• Heeks, R., & Molla, A. (2009). Impact assessment of ICT-for-development projects: A compendium of approaches (Working Paper No. 36). Retrieved from Manchester Development Informatics Group, Manchester, UK: Development Informatics Group Institute for Development Policy and Management: http://www.uq.edu.au/ccsc/impact-assessment-of-ict-for-development-projects-a-compendium-of-approaches
   Google Scholar
• Herrero, C., Martinez, R., & Villar, A. (2010). Improving the measurement of human development (Human Development Research Paper). United Nations Development Programme. Retrieved from http://hdr.undp.org/en/reports/global/hdr2010/papers/HDRP_2010_12.pdf
   Google Scholar
• Hilty, M., & Hercheui, D. M. (2010). ICT and sustainable development (Report No. 328). Queensland: IFIP International Federation for Information Processing. Retrieved from http://link.springer.com/chapter/10.1007%2F978-3-642-15479-9_22#
   Google Scholar
• Howitt, R. (2011). Theoretical foundations. In F. Vanclay & A. E. Esteves (Eds.), New directions in social impact assessment: Conceptual and methodological advances (pp. 3–19). Cheltenham: Edward Elgar.
   Google Scholar
• Ibrahim-Dasuki, S., & Abbott, P. (2011). ICT and empowerment to participate: A capability approach. Paper presented at the proceedings of Americas conference on information systems (AMCIS), Detroit, Michigan . Retrieved from http://aisel.aisnet.org/amcis2011_submissions/20
   Google Scholar
• Kozma, R. B. (2005). Monitoring and evaluation of ICT for education impact: A review. In D. A. Wagner, B. Day, T. James, R. B. Kozma, J. Miller, & T. Unwin (Eds.), Monitoring and evaluation of ICT in education projects. A handbook for developing countries (pp. 11–18). Washington, DC: infoDev/World Bank. Retrieved from http://www.infodev.org/en/Publication.9.html
   Google Scholar
• Krauss, K. (2009). Ethical research practice for community entry: Using ICT4D in a deep rural context. Paper presented at the 3rd International Development Informatics Association (IDIA) conference, Kruger National Park, South Africa, 28–30 October 2009, pp. 246–262. Retrieved from http://www.developmentinformatics.org/conferences/2009/papers/16-krauss-k.pdf
   Google Scholar
• Kreber, C., Brook, P., & Educational Policy. (2001). Impact evaluation of educational development programmes. International Journal for Academic Development, 6(2), 96–108. doi:10.1080/13601440110090749
   Google Scholar
• Kuechler, B., & Vaishnavi, V. (2008). On theory development in design science research: Anatomy of a research project. European Journal of Information Systems, 17, 489–504. doi:10.1057/ejis.2008.40
   Web of Science ®Google Scholar
• Kumar, R., & Best, M. (2006). Social impact and diffusion of telecentre use. Journal of Community Informatics, 2(3), Retrieved from http://ci-journal.net/index.php/ciej/article/view/328/267
   Google Scholar
• Loh, Y. (2013). Approaches to ICT for development (ICT4D): Vulnerabilities vs. capabilities. Information Development, 31, 1–10. doi:10.1177/0266666913513198
   Web of Science ®Google Scholar
• Marathe, M., O’Neill, J., Pain, P., & Thies, W. (2015). Revisiting CGNet Swara and its impact in rural India. Paper presented at proceedings of the seventh international conference on Information and Communication Technologies and Development (ICTD ‘15). Singapore, 15–18 May 2015. New York: ACM. doi:10.1145/2737856.2738026
   Google Scholar
• Mchombu, K. (1995). Impact of information on rural development: Background, methodology, and progress. In: P. McConnell (Ed.), Making a difference: Measuring the impact of information on development (pp. 87–102). Retrieved from http://www.idrc.ca/en/ev-9372-201-1-DO_TOPIC.html
   Google Scholar
• Melamed, C., Devlin, N., & Appleby. (2012). Valuing development: Could approaches to measuring outcomes in health help make development more accountable. Overseas Development Institute. Retrieved from http://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/7616.pdf
   Google Scholar
• Molla, A., & Al-Jaghoub, S. (2007). Evaluating digital inclusion projects: A livelihood approach. International Journal of Knowledge and Learning, 3, 592–611. doi:10.1504/IJKL.2007.016835
   Google Scholar
• Mthoko, H. (2016). Outcome and impact assessment in the comprehensive evaluation of rural ICT projects in developing countries (PhD). Rhodes University.
   Google Scholar
• Myers, M. (2005). Monitoring and evaluating Information and Communication for Development (ICD) Programme. Retrieved from http://www.oecd.org/dev/devcom/46388330.pdf.London:DFID
   Google Scholar
• Ochara, N. M. (2013, August). Linking reasoning to theoretical argument in information systems research. Paper presented at the 19th Americas Conference on Information Systems (AMCIS), Chicago, IL.
   Google Scholar
• O’Neil, D. (2002). Assessing community informatics: A review of methodological approaches for evaluating community networks and community technology centers. Internet Research, 12, 76–102. doi:10.1108/10662240210415844
   Web of Science ®Google Scholar
• Osah, J., Pade-Khene, C., & Foster, G. (2014). Critical themes of process assessment in rural ICT4D projects: An analysis of assessment approaches. Electronic Journal of Information Systems in Developing Countries, 60. Retrieved from http://www.ejisdc.org/ojs2/index.php/ejisdc/article/view/1144 doi: 10.1002/j.1681-4835.2014.tb00425.x
   Google Scholar
• Pade, C., & Sewry, D. (2009, October). The practice and need for rural ICT for development evaluation: An experience of the Siyakhula Living Lab Baseline Study. Paper presented at the 3rd International Development Informatics Association (IDIA) conference. Digitally Empowering Communities: Learning from Development Informatics Practice. Kruger National Park, South Africa.
   Google Scholar
• Pade-Khene, C. (2012). Assessing ICT4D project design: A programme theory assessment of the Siyakhula Living Lab. Paper presented at the 6th International Development Informatics Association (IDIA) conference. Istanbul, Turkey.
   Google Scholar
• Pade-Khene, C., Palmer, R., & Kavhai, M. (2010). A baseline study of a Dwesa rural community for the Siyakhula Information and Communication Technology for development project: Understanding the reality on the ground. Information Development, 26(4), 265–288. Retrieved from http://idv.sagepub.com/content/26/4/265.full.pdf doi: 10.1177/0266666910385374
   Web of Science ®Google Scholar
• Pade-Khene, C., & Sewry, D. (2011a). Towards a comprehensive evaluation framework for ICT for development evaluation – an analysis of evaluation frameworks. Paper presented at the 2nd international conference on information management and evaluation. Toronto, Canada.
   Google Scholar
• Pade-Khene, C., & Sewry, D. (2011b). Proposed stages of a Rural ICT Comprehensive Evaluation Framework in ICT for rural development projects. Paper presented at the proceedings of the South African Institute of Computer Scientists and Information Technologists conference on knowledge, innovation and leadership in a diverse, multidisciplinary environment (SAICSIT ‘11), Cape Town, South Africa, pp. 326–329. doi:10.1145/2072221.2072269
   Google Scholar
• Parkinson, S., & Ramirez, R. (2006). Using a sustainable livelihoods approach to assessing the impact of ICTs in development. Community Informatics, 2(3), 116–127. Retrieved from http://ci-journal.net/index.php/ciej/issue/view/15
   Google Scholar
• Rabaya, K. S. (2009). Economic and social empowerment of women through Information and Communication Technology: A case study of Palestine. The Journal of Community Informatics, 5, 3. Retrieved from 2013 from http://ci-journal.net/index.php/ciej/article/view/500
   Google Scholar
• Ramirez, R. (2011). Participatory monitoring and evaluation of ICTs for development. In J. Steyn & G. Johanson (Eds.), Sustainable solutions for the digital divide: Theory and perspectives (pp. 65–85). Hershey: Information Science Reference.
   Google Scholar
• Reilly, K., & Gomez, R. (2001). Comparing approaches: Telecentre evaluation experiences in Asia and Latin America. The Electronic Journal on Information System in Developing Countries, 4(3), 1–17. Retrieved from http://www.tandfonline.com/doi/pdf/10.1080/10572317.2002.10762563
   Google Scholar
• Rodriguez, P., Nussbaum, M., Lopez, X., & Sepulveda, M. (2010). A monitoring and evaluation scheme for an ICT-Supported Education Program in Schools. Educational Technology and Society, 13(2), 166–179. Retrieved from http://www.ifets.info/journals/13_2/14.pdf
   Web of Science ®Google Scholar
• Rogers, P., Khagram, S., Bonbright, D., Earl, S., Carden, F., Ofir, Z., & Macpherson, N. (2011). IE4D-impact evaluation for development: Principles for action. Retrieved from http://www.scalingimpact.net/files/Impact%20Evaluation%20for%20Development%20-%20Principles%20for%20Action.pdf
   Google Scholar
• Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (1999). Evaluation: A systematic approach (6th ed.). London: Sage.
   Google Scholar
• Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed). London: Sage.
   Google Scholar
• Sen, A. (2013). The ends and means of sustainability. Journal of Human Development and Capabilities, 14, 6–20. doi:10.1080/19452829.2012.747492
   Web of Science ®Google Scholar
• Shadish, W. R., Cook, T. D., & Leviton, C. L. (1991). Foundations of program evaluation: Theories of practice. New York, NY: Sage.
   Google Scholar
• Souter, D. (2012). ICTS, the internet and sustainability (Discussion Paper 28 May). Manitoba: International Institute for Sustainable Development.
   Google Scholar
• Tabassum, G., & Yeo, A. W. (2015). Measurement of tangible and intangible impacts of telecentres on rural communities. Paper presented at proceedings of the seventh international conference on information and communication technologies and development (ICTD ‘15). 15–18 May 2015, Singapore. New York, NY: ACM. doi:10.1145/2737856.2737882
   Google Scholar
• The World Bank. (2011). What is Monitoring and Evaluation (M&E)? Independent Evaluation Group. Retrieved from http://web.worldbank.org/WBSITE/EXTERNAL/EXTOED/EXTEVACAPDEV/0,,contentMDK:22293310~enableDHL:True~menuPK:4585753~pagePK:64829573~piPK:64829550~theSitePK:4585673,00.html
   Google Scholar
• Toyama, K. (2011, February 8–11). Technology as amplifier in international development. Paper presented at proceedings of the 2011 iConference, inspiration, integrity, and intrepidity. New York, USA, pp. 75–82. doi:10.1145/1940761.1940772
   Google Scholar
• Underwood, H., Sterling, R., & Bennet, J. (2013). The PartoPen in Practice: Evaluating the Impact of Digital Pen Technology on Maternal Health in Kenya. Paper presented at proceedings of the sixth international conference on information and communication technologies and development: full papers – Vol. 1 (ICTD ‘13), (Vol. 1, pp. 274–283). New York, NY: ACM. doi:10.1145/2516604.2516613
   Google Scholar
• United Nations Conference on Trade and Development. (2011). Measuring the impacts of information and communication technology for development (Report No.3). Current Studies on Science, Technology and Innovation. Geneva: United Nations.
   Google Scholar
• Unwin, T. (2009). ICT4D-information and communication technology for development. Cambridge: Cambridge University Press.
   Google Scholar
• Wagner, D. A., Day, B., James, T., Kozma, R. B., Miller, J., & Unwin, T. (2005). Monitoring and evaluation of ICT in education projects. A handbook for developing countries. Washington, DC: infoDev. Retrieved from http://www.infodev.org/infodev-files/resource/InfodevDocuments_9.pdf
   Google Scholar
• Walsham, G., & Sahay, S. (2006). Research on information systems in developing countries: Current landscape and future prospects. Information Technology for Development, 12(1), 7–24. doi:10.1002/itdj.20020
   Google Scholar
• Weber, R. (2009). Research on ICT for development: Some reflections on rhetoric, rigor, reality, and relevance. Paper presented at the 3rd International IDIA Development Informatics conference. Kruger National Park, South Africa. Retrieved from http://www.developmentinformatics.org/conferences/2009/papers/weber.html
   Google Scholar
• Weick, K. E. (1989). Theory construction as discipline imagination. The Academy of Management Review, 14(4), 516–531. Retrieved from http://www.jstor.org/stable/258556
   Web of Science ®Google Scholar
• Weick, K. E. (1995). What theory is not, theorizing is. Administrative Science Quarterly, 40(3), 385–390. Retrieved from http://www.commerce.nccu.edu.tw/iacmr/file/Weick%20(1995)%20What%20Theory%20is%20Not%20Theorizing%20Is-9b.pdf doi: 10.2307/2393789
   Web of Science ®Google Scholar
• White, H., Killick, T., & Kayizzi-Mugerwa, S. (2001). African poverty at the millennium: Causes, complexities and challenges. Washington, DC: World Bank Publications.
   Google Scholar
• Wieman, A., Gast, L., Hagen, I., & Van Der Krogt, S. (2001). Monitoring and evaluation at IICD: An instrument to assess development impact and effectiveness (Report No. 5). The Hague: International Institute for Communication and Development (IICD).
   Google Scholar
• World Bank Group. (2012). ICT for greater development impact-information and communication technology. World Bank Group Strategy. Retrieved from http://siteresources.worldbank.org/extinformationandcommunicationandtechnologies/Resources/WBG_ICT_Strategy-2012.pdf
   Google Scholar
• Yafi, E., Nasser, R., & Tawileh, A. (2015). ICT’s impact on youth and local communities in Syria. Paper presented at proceedings of the seventh international conference on Information and Communication Technologies and Development (ICTD ‘15). Singapore, 15–18 May 2015. New York: ACM. doi:10.1145/2737856.2737907
   Google Scholar