How ICT4D Research Fails the Poor

Abstract

Research can improve development policies and practices and funders increasingly require evidence of such socioeconomic impact from their investments. This article questions whether information and communication technologies for development (ICT4D) research conforms to the requirements for achieving socioeconomic impact. We report on a literature review of the impact of research in international development and a survey of ICT4D researchers who assessed the extent to which they follow practices for achieving socioeconomic impact. The findings suggest that while ICT4D researchers are interested in influencing both practice and policy, they are less inclined toward the activities that would make this happen, especially engaging with users of their research and communicating their findings to a wider audience. Their institutions do not provide incentives for researchers to adopt these practices. ICT4D researchers and their institutions should engage more closely with the users of their research through more and better communications with the public, especially through the use of information and communication technologies.

Keywords:

• ICT4D
• research
• policy
• practice
• impact

1. Introduction

The UK's Department for International Development (DFID) has acknowledged that research can have powerful influences on both policies and institutions in support of development objectives and is therefore likely to be an essential element in meeting the Millennium Development Goals and reducing poverty. Research has a crucial role to play, it says, in helping to develop evidence-based and innovative approaches to international development. However, while journal articles remained the predominant output of DFID research, this form of output, says the report, inevitably constrains the impact that the research will have on the problems being studied, as it relies on a trickle-down mechanism from readers of journal papers which is hard to justify. Accordingly, DFID needs to invest in uptake pathways in which there is a need to go beyond research and dissemination (Surr et al., 2002). By 2013, the guide for DFID-funded research had announced that research programs are expected to plan and implement a research uptake strategy, which should encompass stakeholder engagement, capacity-building, communication and monitoring and evaluating uptake (2013).

Alongside this development, the UK's Economic and Social Research Council (ESRC), which invests around £3 billion annually in research, has announced that it expects that the researchers it funds will have considered the potential scientific, societal and economic impact of their research. In recent years, says ESRC, the government has placed increasing emphasis on the need for evidence of economic and social returns from its investment in research. By ensuring that ESRC-funded research makes the biggest possible impact on policy and practice, it says, and improving how we measure and capture this, we are better able to support the case for research funding. Impact helps to demonstrate that social science is important – that it is worth investing in and worth using.¹ On its website, the Research Councils UK outlines the common characteristics of high-quality pathways to impact that research proposals should contain. These include good consideration of the relevant beneficiaries and user needs with specific targeting and tailored activities; clear description of how the applicant intends to reach and engage with the beneficiaries of the research, including clear deliverables and milestones; co-production and involvement of beneficiaries and users from the outset (including research design); and clearly demonstrated commitment for realizing both academic and non-academic research impacts.

This article questions the extent to which academic research in information and communication technologies for development (ICT4D) conforms to the requirements for achieving economic and societal impact. There seems to be an implicit acceptance that ICT4D research should achieve such impact. For example, the Declaration of Principles of the World Summit on the Information Society makes repeated reference to the use of information and communication technologies (ICTs) for alleviating poverty,² which it reaffirms for the Post-2015 Development Agenda (WSIS, 2014). Accordingly, as Heeks (2014) points out, “if the post-2015 agenda reflects what matters and what works in delivering socioeconomic development, then there can be no better guide for prioritizing research that makes a difference, and that has a real-world impact.” It therefore becomes valid to question whether research that is prioritized by an agenda that includes poverty alleviation is capable of leading to such an outcome, whether or not it does in fact serve the poor.

The article is structured as follows: first, after briefly recalling debates around theory-building in the role of research, we report on a review of the literature on research into the impact of research in international development; next, we report on a survey of ICT4D researchers, which is structured around the results of the literature review. After a discussion of the survey findings, we offer some recommendations for consideration by ICT4D researchers and their institutions, as well as proposals for further enquiries.

2. What is the purpose of research?

Debates around the impact of research within policy-making processes and for the shaping of professional practices have continued for some time. However, recent trends relating to calls for greater use of evidence in policy-making as well as the demands of major research funding agencies have brought the issue into sharper focus. It is not the intention here to discuss the relative merits of research that emphasizes either theory development or problem-solving, or both, but in introducing a review of the literature on the impact of research in international development it seems worth pointing out that the problems posed by the difficulty in transferring scientific information or research results to decision-making have become a subject for academic thinking (Almeida & Báscolo, 2006). Accordingly, theory-building around the role of research in knowledge transfer constitutes a legitimate academic pursuit, dating back at least to the seminal work of Carol Weiss who categorized the different meanings associated with the concept of the use of social science research in the sphere of public policy. She concluded that social scientists can at least improve the contribution that research makes to the wisdom of social policy (Weiss, 1979). More recently, Lavis, Ross, McLeod, and Gildner (2003) have highlighted three ways of improving research utilization: producer push; user pull and knowledge exchange. Based on these theoretical perspectives, the concept of knowledge mobilization has emerged, characterized as a suite of services that enhances the two-way connection between researchers and research users so that research and evidence can inform decisions about public policy and professional practice (Phipps & Shapson, 2009).

The relatively recent field of ICT4D research is yet to address the issue of transferring or mobilizing its knowledge into the spheres of policy formulation or professional practice. Research impact, as in most academic disciplines, actually refers to the impact that research has on other research, thus perpetuating a closed loop of intellectual pursuit. The currently dominant discourse around the impact of ICT4D research reveals its obsession with citation rates (Heeks, 2010a, 2010b, 2014). Despite claims in published accounts that research findings will be useful in guiding governments, aid agencies, NGOs and communities toward desirable outcomes with ICT4D projects, little evidence that this actually happens is offered. Although published journal papers in ICT4D often close with the author's reflection that their findings can inform professional practice and policy reform, rarely is it reported how it is intended to bring such changes about; the apparent assumption being that saying it can be so will be sufficient for it to happen. Is it not time for an end to such lip-serving naivety?

Academic research in general receives continued criticism in the media over its apparent disconnection from real-world issues and the people who might make good use of its findings. One observer, Anne-Marie Slaughter, who is currently the president and CEO of the New America Foundation and Professor Emerita of Politics and International Affairs at Princeton University has noted:

all the disciplines have become more and more specialized and more and more quantitative, making them less and less accessible to the general public; a basic challenge is that PhD programs have fostered a culture that glorifies arcane unintelligibility while disdaining impact and audience. This culture of exclusivity is then transmitted to the next generation through the publish-or-perish tenure process. (Kristof, 2014)

The online publication Inside Higher Ed reported that the International Studies Association, with 6200 members worldwide and which labels itself as “the most respected and widely known scholarly association dedicated to international studies,” unveiled a proposal to bar members affiliated with its scholarly journal from blogging. It quotes Stephen M. Saideman, a professor at Carleton University in Canada, saying “there's still a segment of academia that doesn't engage in any kind of social media” (Straumsheim, 2014).

In order to examine issues such as these in the field of ICT4D research, and in the absence of an existing body of research into the socioeconomic impact of ICT4D research, we turn to the wider field of international development, of which ICT4D is arguably a part, in order to learn more about how the knowledge that research generates is translated into policy formulations and professional practices that can actually benefit the poor.

3. The impact of research in international development

In a review of the literature on the impact of research on development practice and policy formulation, Harris (2015) highlights 2 overriding observations and 10 themes. In this section, we summarize the review and introduce the survey research that stems from it.

The first observation relates to stark differences between researchers and practitioners in international development. According to de Vibe, Hovland, and Young (2002), there is a clear divide between development researchers and policy-makers, which underpins the traditional view of the link between research and policy. It seems that researchers, practitioners and policy-makers live in parallel worlds with different values, languages, time frames, reward systems and professional ties (Stone, 2009). Others also attribute a failure to transfer knowledge to cultural differences between researchers and users, which act as barriers to engagement (Jacobson, Butterill, & Goering, 2004). Moreover, researchers in any one field tend not to speak with one voice, and not all researchers see policy engagement as part of their role (Datta, 2012).

The second observation concerns the different perspectives that development researchers and practitioners have of impact. For academics, the impact of their research is usually reflected by the impact factor that is assigned to the journal in which the research report is published. For practitioners, if research is to have any impact, the results must inform and shape policies and programs, and it should be adopted into practice (Young, 2008). Shanley and López (2009) argue that appropriation of the word “impact” to designate a journal's ranking constitutes a potential misrepresentation of what impact really is and that research institutions discourage impact-oriented research by prioritizing the number and frequency of publications in peer-reviewed journals. The excessive spread of performance measurement practices in academia is argued to have led to an identity representation of academics as performers who are assessed on the basis of their journal publication “hits” instead of on the substance of their work (Gendron, 2008). As a reflection of the shortcomings in the academic systems of rankings and ratings, Hovland (2007) notes that policy research programs will not usually use conventional academic citations in peer-reviewed journals as a primary monitoring and evaluation tool.

Turning to the 10 themes identified by Harris (2015), we note first that for research to have an influence on policy and practice, researchers need to have the intent for it to do so (Carden, 2009; Sen, 2005; Wheeler, 2007). However, even if such intent exists, without other conditions, such as leadership and capacity within the user community, the impact of high-quality policy-relevant research will be limited. Communication is by far the most cited factor in the literature on the impact of research on development policy and practice. Newman, Fisher, and Shaxson (2012) point to the recent interest in supporting evidence-informed policy-making in developing countries through building the capacity of researchers and research intermediaries to supply appropriately packaged research information to policy-makers. Greijn (2008) describes the ability to communicate in a language that policy-makers can understand as a crucial capacity for researchers. However, Jacobson et al. (2004) found that plain language communication with the public is not widely accepted as a legitimate form of scholarship.

Of particular relevance in the current context, the increasing use of ICTs is seen as blurring the line dividing academic researchers and other observers such as op-ed writers and bloggers (Lewin & Patterson, 2012). Some highlight a more transformative role for ICTs for practitioners and researchers who are engaged in iterative and participatory communication processes using ICTs for announcing research findings as they unfold (Harvey, Lewin, & Fisher, 2012). DFID describes the “echo-chamber” effect of social media, referring to the overlap between individuals and organizations working in allied or similar fields, which works to amplify its content, giving rise to enormous reach. For example, the 50 biggest followers of the Twitter account @DFID_Research have a combined reach of 2.4 million users, of @IDS_UK number 3.6 million and of @odi_development have a combined reach of 4.3 million.³ Despite DFID's argument that online media plays a central role in all areas of knowledge and research, a study by Brown (2012) found that UK researchers are discouraged from publishing online by the policy of having international peer-reviewed journal citations, rather than online citations, count toward academic promotion.

With the cultural differences that exist between academics and practitioners, compounded by systemic inadequacies in institutional incentive structures and weaknesses in academic communication skills and processes, the separate role of intermediaries as translators and communicators has emerged (Harvey et al., 2012). These are individuals or institutions that act as research communication specialists, not necessarily undertaking research themselves, but who strengthen the use of research within change processes (Court & Maxwell, 2005; Harvey et al., 2012). Shaxson (2010) describes the contribution that knowledge intermediary organizations make in synthesizing, interpreting and communicating research results to individuals and organizations in policy and practice. She also highlights their role in understanding the demand for such knowledge. For intermediaries, measures of impact move away from content analysis issues such as hit rates, downloads and citations and more toward measures of inclusivity and stakeholder involvement in project and program plans and institutional strategies (Shaxson, 2010).

Going one step further than the intermediary, the role of policy entrepreneur has emerged for researchers wishing to influence policy. A policy entrepreneur is an individual who invests time and resources to advance a position or policy, with one of their most important functions being to change people's beliefs and attitudes about a particular issue (Stone, 2009). Masset, Mulmi, and Sumner (2011) specify four essential skills for policy entrepreneurs: understanding politics; synthesizing research into simple compelling stories; good networking and building programs that bring these factors together.

Although research may not have a direct influence on specific policies, the production of research may still exert a powerful influence through shaping the policy discourse (Weiss, 1977). Weiss describes this as a process of percolation, in which research findings and concepts circulate and are gradually filtered through various policy networks. Various types of networks feature in the development research-policy link literature, such as policy streams, policy communities, epistemic communities, think tank networks and advocacy coalitions. Networks and inter-organizational linkages are among the determining influences as to why some ideas are picked up and acted on, while others are ignored and disappear (de Vibe, Hovland, & Young, 2002).

The next requirement for research to become influential in practice and policy circles, ensuring that science is shared with those who need it, is a shift in academic incentive structures so that they reward actual impact rather than only publication in so-called high-impact journals (Datta, 2012). Cultural changes around the impact agenda are required and they need to be led with strong leadership by senior management and academics at research institutions (Stevens, Dean, & Wykes, 2013). Apparently, incentives for researchers to produce outputs that reach a broader swath of society are so low that if engaged in at all, this occurs as an afterthought once results are published (Shanley & López, 2009). Additionally, researchers must know the key stakeholders in the policy-making process, and they should understand how the door can be opened to politicians and the public interest (Taylor, 2005). Academics and think tanks have a far greater chance of being heard when there are like-minded influential politicians in the dominant advocacy coalition (Hovland, 2007). Understanding the political context will also contribute to an awareness of the demand for research, as Mulgan and Puttick (2013) put it, one of the most striking factors impeding the effective use of evidence is the absence of organizations tasked with linking the supply and demand of evidence. Shaxson (2010) observes that we know more about how to improve the supply of evidence than we do about how to improve the demand for it. Adolph, Jones, and Proctor (2010) argue for the development of new user participation models in research design and implementation.

Several of the themes in the literature converge around the final theme, that of engagement; the need for closer relationships between researchers and research users (O'Neil, 2005). DFID regards engagement as individuals moving from simply accessing or consuming the content and services offered by an online platform to becoming more involved in the platform, recommending or promoting it and actively co-creating the content. However, many researchers still regard engaging in knowledge transfer as a low priority activity (Jacobson et al., 2004) and fewer than 5% of academics responding to Shanley and López's (2009) survey regarded engagement with the media as an outlet for scientific findings as having any consequence for measuring scientific performance at their institutions.

4. Method

In order to assess the extent to which ICT4D research conforms to the requirements for development research to have any socioeconomic impact, especially on professional practice and policy formulations, a survey was conducted among ICT4D researchers. The survey was structured around the findings of the literature review, a model of which is shown in Figure 1.

Figure 1. The impact of research on policy and practice.

A questionnaire was circulated to ICT4D researchers in November 2013 through an online survey tool.⁴ Survey respondents were drawn from 1884 individuals who have registered themselves on the website of the Electronic Journal on Information Systems in Developing Countries (EJISDC) which has published 61 volumes since 2000.⁵ This sample was supplemented with a further 61 respondents drawn from the author's network of colleagues. A total of 272 replies were received, giving a response rate of 14%. The questionnaire contained 48 questions answerable on a Likert-type scale from 1 to 5, with 1 indicating strongly disagree/never and 5 indicating strongly agree/very often. The questions and the variables they represent are shown in the Appendix. Respondents’ demographic data are shown in Table 1.

Table 1. Demographic data of respondents.

		N
Gender
Female	23.60%	62	266
Male	76.40%	204
Age
Under 21	0%	0	266
21–29	12.03%	32
30–39	34.96%	93
40–49	33.08%	88
50–59	12.03%	32
60–69	7.52%	20
70 or older	0%	0
Rather not say	0.38%	1
Education
High school	0.37%	1	267
Diploma	0.37%	1
Graduate	6.37%	17
Masters	49.81%	133
Ph.D.	43.07%	115
Location of current research
Global	11.70%	31	265
North Africa	1.13%	3
Sub-Saharan Africa	29.43%	78
Oceania	2.26%	6
North America	1.51%	4
Central America and the Caribbean	3.40%	9
South America	4.91%	13
Europe	7.55%	20
Southwest Asia (Middle East)	7.55%	20
South Asia	12.45%	33
Southeast Asia	17.74%	47
East Asia	0.38%	1
Current academic position
Student	13.62%	35	257
Research assistant	5.84%	15
Lecturer	20.23%	52
Senior lecturer	11.28%	29
Assistant professor	8.56%	22
Associate professor	10.89%	28
Research fellow	7.78%	20
Professor	7.39%	19
Dean	0%	0
Other	12.45%	32
Rather not say	1.95%	5
Length of time carrying out research in ICT4D
Less than a year	5.26%	14	266
1–3 years	24.06%	64
3–5 years	15.41%	41
5–7 years	19.92%	53
7–9 years	9.02%	24
9–11 years	10.90%	29
11–13 years	3.38%	9
13–15 years	4.14%	11
More than 15 years	7.89%	21

A summary of the scores for each variable is shown in Table 2 and depicted graphically in Figure 2. Statistically significant correlations between variables that were found to depict strong relationships are shown in Table 3.

Figure 2. Variable summary values.

Table 2. A summary of the scores for each variable.

Intent to influence	4.10
Understanding of communication with the public	3.91
Understanding of research impact as policy and practice influence	3.81
Cultural affinity with research users	3.69
Extent of engagement with research users	3.67
Understanding of research impact as publications	3.61
Influence of incentives	3.38
Level of consideration of the political context	3.36
Involvement of intermediaries	3.27
Use of ICTs	3.20
Extent of demand for research	3.03
Degree to which considered to be a policy entrepreneur	3.00
Degree of participation in networks	2.93

Table 3. Statistically significant correlations between variables that depict strong positive relationships.

	Cultural affinity	Intent	Communication	Use of ICTs	Intermediaries	Network participation	Political context	Research demand	User engagement
Communication	.430**
	.000
Use of ICTs			.423**
			.000
Network participation				.454**
				.000
Political context						.520**
						.000
Research demand						.455**	.497**
						.000	.000
User engagement		.403**	.421**	.547**	.422**	.466**	.564**	.475**
		.000	.000	.000	.000	.000	.000	.000
Policy entrepreneur	.411**	.372**	.416**	.528**	.416**	.550**	.642**	.476**	.601**
	.000	.000	.000	.000	.000	.000	.000	.000	.000

**Correlation is significant at the .01 level (two-tailed).

The data reveal that respondents are generally attracted by the prospect of their research having influence on policy and practice. About 91.1% agree that research-based evidence is important for shaping policy and 93.8% agree that it is important for shaping professional practice. Moreover, 85.8% of respondents agreed that when they start an ICT4D research project, they have the intention to use the results to influence professional practice, and 74.6% agree that they intend to influence government policy. Similar proportions agree that the impact of their research is reflected by its influence on professional practice (88.8%) and policy formulation (79.0%). At the same time, however, 70.7% agree that their institutions emphasize peer-reviewed publications over communications with the public, and this is despite the fact that 82.3% of respondents agree that informing the public of their research findings is a legitimate form of scholarship.

Notwithstanding this apparent interest in practice and policy influence, only 51.4% of respondents include communicating the results of their research to the public in their research design, even though 92% claim they are willing to do so; in clear, non-technical language. Furthermore, although 92.9% of respondents agree that being an effective communicator is an important skill for an ICT4D researcher, only 50.0% use ICTs to communicate their findings, 33–34% use social media to communicate their research activities and findings and 32.7% use Web 2 tools within their work. Only half of respondents agree that their institutions either encourage them to produce research outputs that reach a broad section of society (51.0%) or to effectively engage with users of their research findings (50.7%). Roughly the same proportion agrees that their institutions assess their performance based on their influence on professional practice (55.0%) or on policy formulation (43.4%).

In contrast to the apparent belief and interest in the influence that their research should have on practice and policy, only 34.3% of respondents claim to spend much time synthesizing their findings into compelling stories, and 27.8% in communication programs targeting policy-makers. Even fewer participate in either national or international networks that include practitioners and/or policy-makers (27.3–35.0%); 51.5% try to stay familiar with the demand for their research findings among policy-makers and 36.7% claim to be familiar with the methods for accessing policy-makers. Only 25.7% invest time in advancing a political position or policy.

5. Discussion

The survey results have revealed the relative strengths of the factors relating to ICT4D research that are thought to influence professional practice and policy formulations, as derived from the review of the literature on the impact of research in international development. It is encouraging to see that ICT4D researchers generally show an interest in influencing both practice and policy. It is less encouraging to see that such interest is less likely to manifest itself in actual influence as a result of a number of weaknesses.

Firstly, ICT4D researchers do not engage closely with the users of their research findings. Few seem to interact with research users in a way that would either stimulate demand for their research or help them understand how research could contribute to solving their problems. Likewise, few researchers engage in advancing policy positions or working with others who might do so. Similarly, while communicating with the public is quite well regarded as a research output by ICT4D researchers, they seem less inclined to translate this into tangible activity within formal communication programs that might involve ICTs or other forms of networking. Finally the incentives of the institutions within which researchers work militate against the activities that are deemed necessary for research to have practice and/or policy influence. Cited publications are more highly regarded than other forms of communication and on achieving any influence on practice or policy. It is arguably because of this that researchers engage less in activities that might yield practice or policy influence.

Beyond these shortcomings in ICT4D research, the study has revealed evidence in support of the research-related factors that are postulated to influence practice and policy. Several strong positive relationships between the variables of interest emerged from the study, as depicted in Table 3. They indicate the importance of ICTs and participation in networks for engaging and communicating with research users in order to understand the demand for research that is suited to the political context within which the findings will be inserted.

The survey suggests that ICT4D researchers are at least interested in achieving socioeconomic impact, but they pay lip service to the processes that might make this possible. One plausible explanation for this is that their institutions have less interest than they do in achieving socioeconomic impact with research, being more focused on the traditional perspective of publications and citation counts for assessing their version of impact. Previous findings conform to this view. According to Phipps and Shapson (2009), almost all academic institutions still lack the capacity to support research utilization to inform decisions about public policy and professional practice. Another observer has noted that at major research institutions, publication in leading academic journals is a necessary condition for tenure and promotion, whereas influencing practitioners is not a goal of these journals and university reward systems are not designed to facilitate practitioner-oriented research (Bolton & Stolcis, 2003).

Assuming that respondents were sincere in stating their intentions to influence policy and practice and not just conforming to some perceived political correctness, there is the possibility that they are either unaware of and/or untrained in the processes that are required to induce socioeconomic influence from their findings. This is unsurprising, given that the pathway from research to policy has been described as overwhelmingly complex (Jones et al., 2013). Moreover, as Weiss (1977) pointed out, government decision-makers tend to use research indirectly, as a source of ideas, information and orientations to the world and outsiders cannot often trace the effect of a particular finding or a specific study on a public decision. Accordingly, as reasoned by Harvey et al. (2012), research communication has evolved away from solely linear and top-down models of influencing (e.g. getting research onto the desks of the most senior decision-makers), toward more complex and multi-sited theories of change. They see a proliferation in roles and actors for communicating research in development which push the boundaries of conventional ideas of research and challenge how research agendas are set, and how knowledge is generated and shared. It appears, therefore, that the activities and skills required to achieve policy influence with research are quite different from those that are necessary for conducting it. Yet they are rarely included in training and education programs in research methods for young and early-career researchers.

6. Conclusions

The study has sought to understand the extent to which ICT4D academic research conforms to the requirements for achieving economic and societal impact, as revealed by a review of the literature on the impact of research in international development. Not only is such impact regarded as a powerful component in support of development objectives, it is also increasingly being demanded by research funders. The findings illustrate how ICT4D research fails the poor. By performing weakly in the areas which have been identified as important for the findings of research to reach wider audiences beyond the research community, which would render them capable of influencing professional practice and policy formulations, ICT4D research falls short of its potential to benefit the recipients of development programs.

More than 85% of the respondents in the current survey were qualified at the postgraduate level, and 43% had Ph.Ds. Eighty-six percent of respondents were in academic positions, ranging from student to professor. It is plausible to assume that most have received training in research methodologies and see the publication of their findings as the key to advancing their careers. While it is laudable that more than 90% declare their intention to influence practice and policy with their research findings, it is lamentable that so few engage with the processes that could lead to this happening.

This study concludes that ICT4D researchers and their institutions need to take practice and policy influence more seriously, by engaging more closely with the users of their research and by encouraging more and better communications with the public, especially through the use of ICTs. They need to do this for at least three reasons: (1) to continue receiving funding; (2) to realize their potential for influence on international development policy and practices and (3) because it is the right thing to do. They should pay more attention to how their research can be used outside academia and, where necessary, adopt positions for practice and policy that can be furthered through their research activities. The emphasis on academic publication should be matched with an equal prominence given to socioeconomic influence, despite the challenges that this implies in terms of both the skills required to achieve it and the measures required to recognize it when it happens. The commendable intentions of ICT4D researchers deserve to be matched by equal efforts at generating tangible benefits for the subjects of their work.

In the spirit of practicing that which one is teaching, the author is preparing capacity-building activities in support of achieving socioeconomic impact with research for members of his own institution. This includes engaging with research users, adopting specific policy positions and strategies for communicating findings to the right audiences, in the right language and using the most suitable medium of communication. Additionally, in recommending further research, the next stage of the current enquiry is being formulated to collect stories about instances where ICT4D research seems to have influenced practice and policy-making in developing countries in order to reveal the complexities of the process and how it can be possible to engage with them.

Notes on contributor

Dr Roger W. Harris has a Ph.D. in Information Systems from the City University of Hong Kong. He works as an independent consultant in the use of information and communication technologies (ICTs) for poverty reduction and rural development in Asia, providing services to Asian governments, NGOs and development agencies. Dr Harris is also Visiting Professor at the Institute of Social Informatics and Technological Innovation at Universiti Malaysia Sarawak, where he is involved with the deployment of ICTs for development for remote indigenous communities in Borneo.

Notes

1. http://www.esrc.ac.uk/funding-and-guidance/impact-toolkit/what-how-and-why/why-impact.aspx.

2. http://www.itu.int/wsis/docs/geneva/official/dop.html.

3. @DFID_Research R4D is the open access portal to DFID-funded research. It houses over 30,000 research documents on international development, UK: http://www.dfid.gov.uk/r4d; @IDS_UK. The Institute of Development Studies is a leading global charity for research, teaching and communications on international development, Brighton, UK: http://www.ids.ac.uk/; @odi_development is the UK's leading independent think-tank on international development and humanitarian issues, London, UK: http://www.odi.org.uk.

4. https://www.surveymonkey.com/.

5. The EJISDC strives to become the foremost international forum for practitioners, teachers, researchers and policy-makers to share their knowledge and experience in the design, development, implementation, management and evaluation of information systems and technologies in developing countries (https://www.ejisdc.org/).

Appendix. The questions and the variables they represent

Table

Variable	Question
Understanding of research impact as publications	10	The impact of my research is reflected by the impact factor of the journal in which it is reported
	11	The impact of my research is reflected by the number of times it is cited in peer-reviewed journals
	15	My institution emphasizes peer-reviewed publications over communications with the public
	16	My institution assesses my performance on the basis of my publications in peer-reviewed journals
Understanding of research impact as influence on policy and practice	12	The impact of my research is reflected by the influence it has on professional practice
	13	The impact of my research is reflected by the influence it has on policy formulation
	14	Informing the public of my research findings is a legitimate form of scholarship
	17	My institution assesses my performance on the basis of my influence on professional practice
	18	My institution assesses my performance on the basis of my influence on policy formulation
Cultural affinity with research users	19	I spend time interacting with people who can use the findings of my research
	20	My research findings are usually accessible in time for policy decisions
	21	My research findings are usually digestible in time for policy decisions
	22	Research-based evidence is important for shaping policy
	23	Research-based evidence is important for shaping professional practice
Intent to influence	24	When I start an ICT4D research project, I have the intention to use the results to influence professional practice
	25	When I start an ICT4D research project, I have the intention to use the results to influence government policy
Amount of communication with the public	26	I invest time communicating my ICT4D research results to the public in plain language
	27	Communicating the results of my ICT4D research to the public is included in the design of my research
	28	I am willing to communicate the results of my ICT4D research to the public in clear, non-technical language
	29	Being an effective communicator is an important skill for an ICT4D researcher
	30	Communication for development is a discipline that is closely associated with ICT4D
Use of ICTs	31	ICTs are blurring the line dividing academia and journalists/bloggers
	32	I use ICTs to communicate my ICT4D research findings as they unfold
	33	I use social media to communicate my ICT4D research activities
	34	I use social media to communicate my ICT4D research findings
	35	I use Web 2 tools within my ICT4D research work
	36	My institution encourages me to publish my ICT4D research findings online
Involvement of intermediaries	37	I would rather have other individuals or organizations communicate my ICT4D research findings to the public
	38	I try to stay aware of other individuals or organizations that could communicate my ICT4D research findings to the public
	39	When conducting my ICT4D research, I involve other individuals or organizations who can communicate my findings to the public
Degree to which considered to be a policy entrepreneur	40	In my ICT4D research, I invest time and resources in advancing a political position or policy
	41	I keep up to date with who the key political players are in my field of ICT4D research
	42	I spend time synthesizing my ICT4D research findings into simple compelling stories
	43	I put effort into communication programs targeted at policy-makers and/or their advisors
Degree of participation in networks	44	I participate in a national network of ICT4D researchers that also includes practitioners
	45	I participate in a national network of ICT4D researchers that also includes policy-makers
	46	I participate in an international network of ICT4D researchers that also includes practitioners
	47	I participate in an international network of ICT4D researchers that also includes policy-makers
Influence of incentives	48	My institution encourages me to produce ICT4D research outputs that reach a broad section of society
	49	My institution encourages me to effectively engage with users of my ICT4D research findings
Level of consideration of the political context	50	I understand the political context of my ICT4D research work
	51	I am familiar with the methods for accessing politicians
	52	I try to keep myself familiar with the demand for my ICT4D research among policy-makers
Extent of demand for research	53	I participate in activities that are aimed at stimulating the demand for my ICT4D research
	54	I have conducted research into the demand for my ICT4D research
Extent of engagement with research users	55	I make an effort to cultivate relationships with people who can use the findings of my ICT4D research
	56	I encourage the beneficiaries of my ICT4D research to communicate the research findings to the public
	57	I regard engagement with the media as an outlet for my ICT4D research findings as being important for my institution's measurement of my performance
	58	Policy engagement is part of my role as an ICT4D researcher