Interrogating the theory of change: evaluating impact investing where it matters most

Abstract

How is impact investing evaluated? How can and should it be evaluated? Over the past 5 years, there has been solid progress in developing social impact metrics at the industry-wide, firm and investment levels and the industry is becoming increasingly data-rich. Nevertheless, evaluation practices still tend to focus on counting inputs and outputs, and telling stories. Moreover, an important element is too often underdeveloped, invisible, not explicit or missing altogether. That element is theory of change, an approach and tool drawn from the field of program evaluation. This article reviews cases where theory of change has, in fact, been used to good effect at various levels of the impact investing industry. It also discusses a range of qualitative and quantitative methods which could be usefully blended with the theory of change approach, and affirms the equally important imperatives of accountability and learning across all combinations of methods. The article concludes that a more comprehensive application of theory of change to all levels of the field is required – and especially to the micro-level of individuals, households and communities, where the results of impact investments matter most. Such an approach can help build an impact investing industry that is adaptive, transparent and self-sustaining. To this end, creating an ongoing dialogue between the development evaluation field and the impact investing industry, and designing and launching new education and training initiatives, are key tasks in the years ahead.

Keywords:

• impact investing
• evaluation
• theory of change
• mixed methods
• accountability
• learning

Introduction

How can impact investing be evaluated? How should it be evaluated? In such a metrics-rich and increasingly data-driven industry, it could be argued that all stakeholders in the emerging field of impact investing are concerned with these questions. To a considerable degree, this is indeed the case.

However, an important element is often underdeveloped in the discourse and practice on performance assessment in the sector. That element is theory of change. A construct and tool originating in the field of program evaluation, theory of change can, and should be a core element in the evaluation of impact investing. This is especially important where such evaluation matters the most: at the point of social and economic impacts on poor and marginalized communities, households and individuals.1

Fortunately, theory of change is already a part of the practice of the impact investing industry at a number of levels and in various manifestations. In fact, there is much to build upon. Nevertheless, there are two problems. First, in some areas of the field's practice, theory of change is still invisible, not explicit or missing altogether. And second, there has not yet been an assessment of the overall state of play of this pivotal element in the field as a whole and how it can be applied to the maximum effect. This article argues that making the theory of change explicit enables all parties to better understand and strengthen the processes of change and to maximize their results, as well as to test the extent to which results and processes actually align with the expected theory of the intervention. The article draws upon the evaluation theory for its conceptual framework. It aims to address these issues and to point to future directions in evaluation that can contribute to building an impact investing industry that is robust, resilient and self-sustaining.

Even with the remarkable rise of the new economic powers of China, India, and Brazil, extreme poverty is still too widespread in the Global South. It is clear enough that funding from government and philanthropic grants is insufficient to solve the problems faced by the bottom billion, not to mention other wicked, complex global challenges such as the deadly effects of climate change and pandemics. Accordingly, for the past decade, the United Nations has explored ways and means of encouraging innovative forms of development finance. One of the breakthrough instruments that the international community has produced, for example, is the vaccine bond, in which private and public investors provide front-end research and development capital for new vaccines for HIV/AIDS and other diseases. The funds are repaid when the vaccines reach the market and generate business revenue. More recently, there has been a renewed call for the creation of an even wider range of innovative development-finance instruments, including, for example, a sovereign wealth fund for infrastructure, the global component of a fuel tax, a (Tobin) financial transactions tax, efficiency savings on the transfer of remittances and diaspora bonds (Gates 2011).

This article first presents an overview and definition of impact investing and its key actors, and reviews efforts to create a common set of standards and rating system for the industry globally. The contribution of the field of development evaluation is then discussed. Next, the concept and tool of theory of change is defined, and its advantages to the evaluation of impact investing are set out. The article then highlights examples of productive applications of theory of change at various levels of the impact investing field, and goes on to list a number of methods that can be combined with theory of change to produce valuable evaluation insights. The imperative of balancing accountability and learning cuts across all methods and levels. Finally, the article concludes with a discussion of new directions and next steps in advancing the integration of theory of change into the practice of the evaluation of impact investing.

Impact investing: an emerging industry

One of the most creative and promising areas of innovative development finance is a small, emerging industry that has come to be known as impact investing (see, for example, Bugg-Levine and Emerson 2011; Thornley et al. 2011; Freireich and Fulton 2009). Over the past half-decade, this industry has created networks, standards and metrics and, importantly, unlocked a substantial volume of private and public capital for social and environmental purposes around the globe. This initial stage of industry development, characterized by market organization, has been driven by a core group of proponents that has included foundations, high-net worth individuals, family offices, investment banks, development finance institutions and dedicated impact investment funds. The main organizing instrument of the industry is the Global Impact Investing Network (GIIN), which serves a membership of 50 institutions, firms and funds on its Investors' Council. Till date, most of the major players in this field have been based in the United States or Europe, but there is also evidence of a growing number of networks and actors based in Asia, Africa and the Americas (Jackson and Harji 2012; Harji and Jackson 2012).

There is an on-going (and healthy) debate on what, in fact, constitutes an ‘impact investment’. However, leading proponents of the industry generally would agree that they are mobilizing capital for ‘investments intended to create positive social impact beyond financial return’ (Brandenburg and Jackson 2012; Freireich and Fulton 2009). Two key components of this definition are, first, the intent of the investor to achieve such impacts, and, second, tangible evidence of the impacts themselves.

In addition, there should be a third component in the definition of an impact investment: theory of change. What is the theory of change of the investors? What outcomes and impacts do they expect will be catalysed by the capital they are channeling to enterprises or projects on the ground? Making the theory of change explicit enables all parties to better understand and strengthen the processes of change and to maximize their results, as well as to test the extent to which results and processes actually align with the expected theory of the intervention. Figure 1 depicts these three core elements of the definition of an impact investment.

Figure 1. Core components of the definition of an impact investment.

Actors in the impact investing industry

The stakeholders, or actors, in the impact investing industry can be divided into four broad categories: asset owners who actually own capital; asset managers who deploy capital; demand-side actors who receive and utilize the capital; and service providers who help make this market work. Figure 2 lists the various actors within these four categories.2 Recent research on the industry provides a picture of what impact investors do and how do they do it. Generally speaking, they design and execute private debt deals, providing loans, guarantees and other debt instruments, as well as equity and quasi-equity, to funds, enterprises and projects that aim to provide the poor and marginalized with employment, income and affordable products and services, including housing, food, health care, education, energy and environmental protection. Microfinance institutions in developing countries and affordable housing schemes in developed countries have been the favorite vehicles for these investments, though impact investors are also beginning to diversify across a wider range of sectors (see Bouri et al. 2011a; Saltuk, Bouri, and Leung 2011; Harji and Jackson 2012).

Figure 2. Actors in impact investing. Source: Harji and Jackson (2012).

Moreover, there is a better understanding of the sequencing, or the ‘layering’ of different types of capital, over time, in enterprises and projects designed to generate social as well as financial returns. One documented example where front-end philanthropy created the conditions for impact investments per se is that of Husk Power in India, a green-energy investee of the Acumen Fund (Trelstad and Katz 2011). Further, there is a growing understanding of how groups or syndicates of investors can pool their funds in a single investee enterprise or project while still maintaining different risk-adjusted financial return expectations.3

Common standards and measurement systems

Measuring the impact of such an investment is a key component of this approach for investors. While the capital of many investors has been unlocked for social and environmental impacts, a group of industry leaders has also worked hard to create, test and refine a recognized set of industry-wide standards for the field and a common system of social-performance metrics for funds and enterprises.

For its part, the Impact Reporting and Investment Standards (IRIS) project provides a common set of definitions and terms for the field. In addition, the Global Impact Investing Rating System (GIIRS), an analogue of the Standard and Poor's or Morningstar rating systems, uses a common set of indicators to measure the social performance of funds and companies that intend to create impact. These sector-wide or centralized systems are works-in-progress being driven and stewarded by the GIIN in the case of IRIS, and by the US-based non-profit B Lab in the case of GIIRS. What is not clear at this point are the business models that would enable these systems to sustain themselves in the long run. It is worth noting that the Consultative Group to Assist the Poor, which has set standards and assessed performance in the microfinance industry, has been subsidized by foundations and aid agencies for two decades.

While building sector-wide measurement systems for the impact investing industry is underway, there has also been a flourishing of customized or decentralized metrics and tools at the level of individual institutions or organizations. How the centralized and decentralized measurement systems can interact productively with each other is a key question for industry leaders.

Until recently, it could accurately be said that impact investing was metrics-rich but data poor. This is changing. New studies by the GIIN, J.P. Morgan, IRIS and GIIRS – plus the on-going reporting of funds dedicated to impact investing (notably, Acumen Fund, IGNIA, Root Capital and others) – have begun to mobilise and analyse larger and more granular datasets from microfinance institutions and small and growing businesses, in particular, to better understand the scope, nature and performance of impact investments in aggregate and by industry sector and investment type (among others, see Bouri et al. 2011a; Saltuk, Bouri, and Leung 2011).

This is certainly good progress. These efforts deserve continued, and even increased, support. Notwithstanding these gains, however, current practice in the evaluation of impact investing still tends to focus on counting inputs and outputs, and telling stories. Dollars invested, numbers of people reached or served and profiles of local entrepreneurs are all useful, of course. Indeed, they are necessary. But they are not sufficient. Most of the actors involved in the impact investing industry understand that the process of achieving meaningful social impact in poor communities is complex, nuanced, dynamic and, in fact, often uncertain. Evaluating these processes and their chains of results requires a detailed understanding of systems of causal factors and effects whose character is shaped by forces acting at all levels – global, national and local – and across the social, economic, political, cultural and environmental spheres. The field therefore needs frames and tools that are not only capable of rendering such systems understandable, but also of helping to locate and examine the granular effectiveness of specific investments and other interventions – and, ideally, of doing all this in a cost-effective manner.

The contribution of development evaluation

More than a decade ago, another professional field began its own growth trajectory. Spurred by a shared concern among donor agencies and governments for ensuring value for money in the high-risk business of international development interventions, the field of development evaluation grew rapidly. Evaluation and audit offices in developing-country governments, non-governmental organizations and bilateral and multilateral agencies grew and expanded their staffing complements. A body of knowledge for the field was built and mobilized through a range of education and training initiatives in the north and in the south. One such example is the World Bank–Carleton University International Program for Development Evaluation Training (see Morra-Imas and Rist 2009). Professional associations of development evaluators flourished at all levels: international, regional and national alike.

Against the backdrop of extensive activity on the measurement of impact, what can the field of development evaluation offer the impact investing industry? I suggest that the field of development evaluation can make three important contributions to the impact investing. First, development evaluation provides a more comprehensive frame for understanding the full potential of the evaluation function in the impact investing industry at all levels and among all stakeholders. Second, development evaluation brings a well-developed set of relevant data collection and analysis methods which can be adopted and adapted by the impact investing industry. Third, and perhaps most importantly, development evaluation reminds all actors in the impact investing field that what matters most is the extent to which the lives of the ultimate beneficiaries of impact investments actually undergo tangible, positive changes that can be attributed, at least in part, to these investments. In this sense, development evaluation can be seen as a force to hold impact investors accountable for their intentions and claims. Indeed, with well-designed training and support, the global network of development evaluators, especially those based in the Global South, can play a valuable role in promoting results, transparency and learning as impact investing evolves and matures (Jackson et al. 2012).

Theory of change

Development evaluation also offers impact investing the concept of theory of change. The term ‘theory of change’ originates in the field of program evaluation. Sometimes also called ‘program theory’, it refers to the construction of a model that specifies (usually visually) the underlying logic, assumptions, influences, causal linkages and expected outcomes of a development program or project. Through the collection and analysis of performance data, this model can be tested against the actual process experienced, and results attained, by the intervention (Funnell and Rogers 2011; Morra-Imas and Rist 2009; Rogers 2008).4

This exercise involves the interrogation of the theory of change: Is the program theory valid, appropriate, relevant and accurate? Does change actually occur in the ways the intervention proponents have expected? Are there other change dynamics or pathways at work? Are there unforeseen actors and factors who promote or constrain change? Are there obstacles that stymie or render ineffective the theory of change? How can those obstacles be minimized or eliminated altogether? These are just some of the questions that evaluators ask as they examine an intervention's theory of change. And the answers to these questions can usefully inform program managers and funders as to how they can modify the design of the intervention under review to improve outcomes, or whether the intervention should be terminated altogether.

Applying the theory of change approach is not as expensive as some other options for evaluating impact investing, such as large-scale experimental studies. Theory of change is a generally cost-effective way to frame and inform an evaluation. Furthermore, it can be used in conjunction with a wide range of other data collection and analysis methods. In this sense, it is a flexible tool but one that, at the same time, promotes analytic rigour, learning and value for money.

While there are no ‘silver bullets’ or panaceas in the practice of evaluation, the concept and tool of theory of change can and should be an integral and explicit element in the evaluation enterprise in the impact investing industry. Theory of change can, and should be interrogated at all levels: the field as a whole, multi-firm platforms, individual organizations, specific investments and the beneficiary level of communities, households and individuals.

To what extent do impact investments really make a difference in the lives of the poor and marginalized? This is the core question – the real bottom line – for any evaluation of impact investing. This is where the theory of change should be tested in greatest detail – and where it matters the most.

Multiple levels of application

The fact is that some actors in the impact investing field have already been using the theory of change, and to good effect. These cases illustrate the value of applying this approach at various levels of the industry. One such level is that of the sector or industry as a whole. In the evaluation of the Rockefeller Foundation's global Impact Investing Initiative in 2011–2012, theory of change was applied to both articulate and test the field-building strategy of the Initiative. Figure 3 reproduces that theory of change. With the objective of creating a full ecosystem for impact investing, the Initiative's strategy included the following components: building collective action platforms (notably, the GIIN), developing industry infrastructure (particularly IRIS and GIIRS), helping scale key intermediaries (e.g. through the provision of program-related investment to the Acumen Fund, Root Capital and IGNIA) and promoting policy reform (see Thornley et al. 2011). To operationalize these catalysing activities, the Initiative made use of a wide range of tactics, including: convening, grant-making, program-related investments, relationship-building, media engagement, co-production and co-dissemination of knowledge products, creating a core network of allies and establishing a legacy instrument – the Global Impact Investing Network – to continue this field-building work. Drawing on interviews with over one hundred industry leaders, the evaluation identified strengths and weaknesses in this strategy. In fact, the theory of change diagram itself was used to spark discussion with interviewees and served to clarify and even enrich these exchanges (see Jackson and Harji 2012).

Figure 3. Theory of change for the impact investing initiative.Source: Jackson and Harji (2012).

One important insight underscored by this theory of change at the sector-wide level was the many layers of activities, actors, factors, assumptions and results ‘standing between’ the catalysing activities of the Initiative, at the bottom end of the diagram and impacts expected to improve the lives of the poor and vulnerable, at the upper end of the figure. One implication of this, the evaluation suggested, is that the industry-building process will very likely take longer than the advocates' discourse of the early years had suggested. A second implication, the evaluation pointed out, is that industry-building efforts must shift away from market organization towards deal execution and investment implementation. The evaluation shows there is no better way for the field to make its case for further growth than by demonstrating the value created for beneficiaries of this investment approach (Jackson and Harji 2012).

For its part, the GIIRS platform is informed by a logic model that requires each impact investment manager it rates to make explicit the firm's own theory of change relating to social impact. The GIIRS report on each company includes a statement of intent by the firm, and details the policies and incentives designed by the company to encourage positive social and environmental performances, documents the actual practices of the company, reports on company outputs in the rating period (e.g. energy and water use, jobs created and volunteer hours), and tracks selected outcomes (e.g. carbon emissions, income increments for employees and communities). In other words, the company's theory of change is embedded in each ratings report (GIIRS 2012a). The GIIRS platform provides access by investors to a pool of companies with different theories of change. Investors can thus choose the investees that most closely match their own intent, theories and strategies (see the GIIRS website for more detail; GIIRS 2012b).

Another platform example involves Nexii, a social stock exchange based in South Africa. All companies seeking to be listed on the exchange must clearly state their impact intent and their theory of change. ‘An articulated theory of change illustrating the link between intention, action and outcome becomes the centerpiece of impact due diligence’, says Nexii (2012, 1). Prospective investors can test the logic of the firm's theory as a whole as well as interrogate its individual parts. Nexii argues that applying such due diligence processes can improve investor confidence and increase the impact of investments, as well as enabling the investor to hold its investees to account.

Theory of change is also used by individual organizations in the impact-investing industry. A good example is Root Capital, a US-based non-profit that provides loans, advice and training to rural small businesses, usually cooperatives, in Africa and the Americas. Drawing support from foundations, social investment banks, aid agencies and private investors, Root Capital functions as an intermediary between asset owners and managers, on the one hand, and demand-side actors, on the other hand. Root Capital provides working capital that is often secured against the inventories of the borrowing cooperatives. In 2010, average loan size ranged between $350,000 and $400,000, and average rates of return were 2.5–3% (Jackson and Harji 2012).

Figure 4 depicts Root Capital's theory of change. This theory holds that the provision of its capital and financial training enables rural small enterprises to flourish. In turn, these businesses generate steady, more attractive revenue streams to farmers and their households and communities. At the same time, Root Capital also helps its borrowers to access other sources of finance from its partner organizations, notably USAID and Rabobank. In late 2011, the strategic assessment team for the Impact Investing Initiative carried out fieldwork in Nicaragua that ‘confirmed that this theory of change is credible and can indeed deliver tangible and meaningful economic benefits to individual farmers and their households, which in turn, can be converted into social outcomes’ (Jackson and Harji 2012, 51).5

Figure 4. Root capital's theory of change. Source: Root Capital (2012).

At the level of individual investments, Rockefeller Philanthropy Advisors has published a widely distributed handbook on how to design, implement and evaluate impact investments. Using a logic model that details the ‘impact value chain’ for a given investment, this guide distinguishes clearly between outputs (defined as the ‘activity results’, such as the number of people served by a project), on the one hand, and outcomes (defined as the collection ‘of all results, intended and/or unintended’, minus what would have happened anyway without the intervention). The authors of the handbook observe that ‘most impact investors focus on the first order derivative output of their work. This not only makes the task of measurement and reporting more feasible, it also preempts speculative and contentious debate on real causes and effects’ (Godeke and Pomares 2009, 116–117). Aggregating outputs, therefore, is not the optimum way of measuring impact. Indeed, the guide's logic model, or the theory of change, shows that outcomes and impacts constitute very different kinds of results. The ideal systemic solution, the authors suggest, involves building and refining impact metrics that are standardized across the impact investing industry. They clearly support the efforts of IRIS, GIIRS and other actors working towards this goal (Godeke and Pomares 2009).

Why does theory of change matter to the evaluation of impact investing? Why should actors in the impact investing industry who are not using it now consider adopting this approach? What is its value added? As these examples suggest, there are at least five main reasons for industry actors to integrate this approach into their work: first, the complex and complicated processes of cause and effect in impact investing (and, indeed, in any development intervention) require systematic, disciplined and continuous analysis; theory of change is a cost-effective way of addressing this imperative. Second, the discipline of constructing – and, over time, refining – their theory of change helps investors to purposefully and clearly understand the change they are trying to create and to learn and adjust their strategies and instruments as they proceed forward.

Third, the act of investors publicly communicating their theory of change serves to engage other key stakeholders (partner investors, social enterprises, local organizations) in the implementation and learning process; in turn, this can help build commitment to the interventions financed by the investment. Fourth, an explicit theory of change can be used by governments, NGOs, employees and citizens at large to hold the investor accountable for their stated intentions. In a generally unregulated and emerging industry, checks and balances matter. And finally, for impact investing service providers and for evaluation professionals, theory of change is a tool that can be creatively and productively blended with other evaluation methods and applied at various levels to generate even more useful findings and insights.

Mixed methods for multiple levels

As useful as theory of change is, it is more of a framework and not a sufficient tool in and of itself to collect data on and understand the multiple levels and dimensions of the emergent field of impact investing. Fortunately, development evaluation offers a range of other strategies and methods that can and should be mobilized to evaluate impact investing – in concert with theory of change.

At the level of the field as a whole, it is important to assess the perspectives and experience of industry leaders – asset owners, asset managers, demand-side actors and service providers – from both the Global North and Global South. Purposeful and stratified sampling methods can be used, though new players are regularly identified, especially at the regional and country levels, as the field evolves and activities become more visible and connected. Open-ended qualitative interviews with leaders, as well as closed-ended surveys can be deployed. Over time, collecting useful data from this leadership cohort will increasingly depend on the ability of evaluators to operate in multiple languages and to understand, in detail, diverse economic, political and cultural contexts.

Further, as industry associations form at the global, regional and national levels, network analysis can assist in classifying and assessing the structure, decision-making processes and financing of these organizations (see Carden 2009). Participant observation at industry conferences, workshops and webinars can enrich such analysis. Tracking social media activity across the industry is also helpful in discerning trends, debates, achievements and obstacles, in real time.

Another dimension of field-building, particularly at the national level, is that of policy related to impact investing. The Impact Investing Policy Collaborative (IIPC), an international network, has developed a framework for understanding both government influence and direct government participation in developing the supply of capital for impact investing, actively directing capital, and developing the demand side for such capital (Thornley et al. 2011) discussed in a greater detail in the first article of this journal). At the same time, the International Development Research Centre (IDRC) offers a framework for understanding how development research, or knowledge more generally, is converted into policy change in developing countries (Carden 2009). For its part, the Overseas Development Institute has developed a guide to assessing policy influence by development actors and their interventions that provide evidence and advice to governments, conduct public campaigns and advocacy, or engage in the organized process of lobbying. One of the guide's evaluation methods is the analysis of change tactics, champions and results in specific, time-bound ‘policy episodes’ and comparing findings across episodes (Jones 2011).

At the level of the individual organization, organizational assessment tools are important. One set of such tools builds an overall assessment of organizational performance on the basis of three separate analyses: first, of the external environment (legal and administrative, political, cultural and economic) in which the organization operates; second, of its organizational ‘motivation’ (history, mission, rewards and incentives); and third, of its organizational capacity (strategic leadership, human resources, program management, financial management, inter-organizational linkages). These three analyses are then combined to generate an overall assessment of the organization's effectiveness, efficiency and financial viability (Canadian International Development Agency 2006; IDRC/Universalia, n.d.). This approach can be applied to organizations operating at any level across the industry's spectrum.

Such an approach to organizational assessment, and other approaches, can be used to create case studies for comparative analysis of organizational performance and change. The main purpose of such exercises, however, is usually to provide the organization under study with a private assessment of its progress, challenges and options for improvement. Indeed, the approach outlined here is often used in development evaluation for self-assessment by institutions and organizations. It can and should be used in this way in the impact investing field, as well.

Finally, there is the level of the individual impact investment. Building on the framework, terms, indicators and benchmarks of IRIS and GIIRS, evaluators can and should experiment with a wide range of methods for measuring the impacts of these investments on their intended beneficiaries: poor and marginalized individuals and households and their communities. Currently, in its uneven and fragmented state, and with most impact investments still in their early stages, qualitative methods, such as ‘most significant change’ stories, could be very useful (see Davies and Dart 2005). Moreover, qualitative tools for understanding cause and effect relationships in a given investment, like contribution analysis (Mayne 2008), are also appropriate.

Another general approach to assess micro-level impacts that is often qualitative in nature but can also involve quantitative methods is that of participatory evaluation. This approach aims to expand the breadth and depth of stakeholder participation in all stages of the evaluation process. These stages include identifying the issues to be assessed, developing data collection procedures and tools, collecting data, analyzing findings, reporting to key decision-making bodies and taking action on the findings. Often a working group or task force representing the key stakeholder groups oversees the evaluation and directly or indirectly engages the various evaluation tasks. Efforts must be made in this process to enhance the voice and choice of the most marginalized stakeholders, and to create a safe environment in which they can speak up (see, for example, Jackson 2005). Such participatory evaluation can be used to hold impact investors accountable for their claims of social impact, to interrogate the theory of change of the organizations involved, to promote learning among the key stakeholders and to strengthen the commitment of the key parties to improving the intervention under study.

It is well known that there is a cost to participation. Citizens incur an opportunity cost in attending meetings, responding to interviews, and in sitting on committees that oversee evaluation projects. Feminist participatory research (e.g. Aziz, Shams, and Khan 2011; Gouin, Cocq, and McGavin 2011) and feminist theory (e.g. Walby 2011) underscore that such opportunity costs are particularly problematic for women, who have multiple roles and responsibilities. Funders should recognize and find ways of moderating such opportunity costs. Funders should also support longer rather than shorter participatory evaluation processes, which are aligned with local seasonal, climatic and productive rhythms. In the end, there are two purposes of enlarging participation. The first purpose is to support community members in directly interrogating the relevant theories of change of impact investments that affect their lives. The second purpose is to enable community members themselves to construct new and better theories that advance their basic needs and strategic interests and help build an effective impact investing field.

Within individual investments, there are methods from social accounting and cost–benefit analysis that can also be mobilized. Social return on investment (Harji 2008a; SROI Network 2012) and the expanded value-added statement (Harji 2008b; Mook 2013; Mook, Quarter, and Richmond 2007) are two methods for monetizing the otherwise invisible and unaccounted for, social value created by an investment. These calculations can also provide an opportunity for cross-investment comparisons within and across impact investment portfolios. Not without their critics (Torjman 2012; Tuan 2008), these tools should be used carefully and transparently, with a clear understanding of the counterfactual in each instance. And there should be an acknowledgement that sometimes the impacts that are the most difficult to monetize may still be the most influential ones.

What about randomized clinical, or controlled, trials (RCTs) and other experimental methods? Do they have a role in the evaluation of impact investing? There is no doubt that RCTs can be applied to measuring the social impacts of impact investments. However, it is also true that RCTs are generally expensive to implement. In addition, they can sometimes involve serious political (and, some would say, ethical) challenges in interacting with control groups that have not received the same interventions as the experimental groups. Yet the work of Banerjee and Duflo (2011) and Karlan and Appel (2011), in particular, demonstrates the explanatory power of this method in understanding how the poor make decisions where they work and live, important factors for impact investing. In this regard, RCTs can usefully assist evaluators, investors and other stakeholders in understanding the causal factors at work in a given investment and its associated impacts. Still, it will take considerable time and effort for the impact investing industry in most parts of the world to build up a sufficient database of mature impact investments to permit effective randomized sampling.

In summary, impact investing leaders and evaluation practitioners can and should work together to test the blending of the theory of change approach with that of other evaluation methods. As the impact investing industry grows and matures over the next decade and beyond, such a creative orientation to ‘mixed methods’ is very likely to yield valuable insights at multiple levels, and at reasonable cost. Again, it is at the micro-level of households, enterprises and communities where efforts to blend theory of change with other evaluation methods should be primarily focused. That is the opportunity that lies ahead.

Balancing learning and accountability

Across all methodological approaches, the most effective evaluation practice balances accountability and learning. To be sure, the legal and financial underpinnings of most evaluations are derived from the accountability function. However, given the complex and rapidly changing character of today's social, economic and environmental problems, rapid, critical learning is essential for adaptation, innovation and solution-generation. Such learning is based on the knowledge produced by the widest range of stakeholders involved in an intervention (Rodin and MacPherson 2012).

This principle is especially appropriate in the dynamic, emergent field of impact investing. Evaluations in this industry can optimize learning in the following five ways:

•	Encouraging the negotiating of shared outcomes by the broadest range of stakeholders, including investors, investee enterprises and projects and the beneficiaries of investments.
•	Enabling evaluation partners to co-produce and share new evaluation knowledge as a public good, with the aim of improving performance at all levels of the industry.
•	Embedding evaluation processes that are transparent, ethical and culturally informed.
•	Ensuring that power asymmetries among stakeholders are moderated, and amplifying authentic voice and choice by the least powerful players.
•	Empowering communities of practice and organizing learning forums for the sharing of findings and substantive dialogue among key parties (Rodin and MacPherson 2012).

All of these ways and means of optimizing learning can and should be operationalized in the evaluation of impact investing.

New directions, next steps

Putting theory of change closer to the centre of the evaluation of impact investing is important and possible. And this task should be part of a broader effort to more generally strengthen evaluation capacity across the industry and test new combinations of methods and tools. Looking ahead, the following actions would advance this broader agenda.

First, increasing opportunities for dialogue and exchange: Leaders in the impact investing field and their counterparts in development evaluation should create new opportunities for the practitioners in their respective fields to meet each other, exchange perspectives and tools, and strengthen their practice in ways that will advance the growth and integrity of the impact investing field. Conferences, workshops, online communities and pilot projects could facilitate this process of on-going encounters and growing partnership.

Second, institutionalizing evaluation training and education: The development of impact investing to a mature, self-sustaining industry will take many more years. To effectively accompany this process, the practice of evaluation in the sector must be continuously improved and new cohorts of practitioners need to be equipped with the best knowledge and methods available. To this end, it is important to institutionalize and professionalize training and education in the evaluation of impact investing. University and college courses – both in degree programs and for professional development, online and on campus – should be designed, tested and refined. The CLEAR Centre at the University of Witwatersrand and ET Jackson and Associates, with funding support from the Rockefeller Foundation, are working together to develop and pilot such course offerings in Africa.

Third, mobilizing research and analytics: For evaluation to be conducted with ever more precision and utility, it must be informed by increasingly detailed research and analysis. Some impact investing funds and intermediaries are already undertaking this work for their own investment portfolios and target sectors. At the industry-wide level, the work of GIIN and IRIS is beginning to generate larger datasets of industry performance, as well as a series of case studies on collaborative impact investments. For its part, GIIRS continues to rate funds and companies, and now issues quarterly analytics reports on industry metrics. The GIIRS Research Group, coordinated by Duke University, is increasingly active in designing and undertaking systematic, independent research on the GIIRS dataset. As sites of relevant research also emerge in the Global South, these actors too will become valuable resources and partners for the evaluation enterprise.

Fourth, testing theory of change in the household: For the most part, the conversion of positive impact-investment outcomes into tangible benefits for the poor and marginalized occurs within the household. Such outcomes may include, for example, increased income, greater food security, improved housing, better access to affordable services (e.g. education, health, water, energy, finance) or reduced damage to the environment. How and in what form these benefits enter the household need to be fully understood. Perhaps even more important is to understand how decisions are made about the disposition of such additional income, or what services should be accessed – and for whom: which family members, what gender, which age groups? Who in the household does not benefit, and why? What strategies do households use to improve their quality of life and, in some cases, move out of poverty and vulnerability? What obstacles exist to actually implementing such strategies? Detailed study of these and other dynamics, actors and factors will enable evaluators and other practitioners to better test the theory of change of impact investors at the point of impact. Small-scale, qualitative studies and large-scale RCTs could both yield valuable insights. Foundations and development agencies should actively support this work.

Fifth, measuring social value creation: The enterprises that receive capital from impact investors create various forms of social and environmental value. It has usually proven complicated and time-consuming to quantify this value. However, there are some approaches and tools that appear to be capable of addressing this challenge, including social return on investment and the expanded value-added statement.

Finally, deepening stakeholder engagement: Long experience with participatory forms of evaluation attests to the utility of engaging stakeholders meaningfully in the evaluation process. The evaluation enterprise in impact investing should deepen and broaden opportunities for the primary stakeholders – that is, the ultimate beneficiaries of impact investments – to actively design, participate in and use the results of the assessment of impacts at the micro level. Such participation can not only contribute to a more accurate and nuanced evaluation exercise; it can also enable entrepreneurs, employees, villagers, women's organizations and many other groups to hold impact investors accountable for their statements, intentions and practices. In so doing, participants can generate essential insights to assist the impact investing industry in charting its path forward.

Conclusion

Continuing to build the capacity of the impact investing field to use theory of change in its evaluation practice should be a priority in the years ahead. Theory of change can be applied productively in conjunction with a wide range of methods, such as network analysis, policy episode analysis, organizational self-assessment, contribution analysis, participatory techniques, social return on investment, RCTs, and feminist research strategies, among others. Across all combinations of methods, balancing the equally important imperatives of accountability and learning will be essential. Testing various combinations of methods with theory of change will teach practitioners what works particularly well in the unique space of impact investing, and what does not. Furthermore, creating an ongoing dialogue between the development evaluation field and the impact investing industry, and designing and launching new education and training initiatives, should be key elements in this capacity building process.

Interrogating the theory of change across the field can help build an impact investing industry that is adaptive, transparent and self-sustaining. All levels of the industry can benefit from a more comprehensive application of theory of change, including specific investments, investment portfolios, individual organizations, multi-firm platforms and the industry as a whole. However, the application of this approach is especially important at the micro-level of individuals, households and communities, where the results of impact investments matter the most. And stronger micro-level performance enables impact investing to make an even more significant contribution to the macro-level search by the international community for effective innovations in development finance. Impact investing leaders should fully engage in this opportunity.

Acknowledgements

In addition to thanking Tessa Hebb, the editor of this special journal issue, and two thoughtful anonymous reviewers, the author recognizes the important leadership in evaluation and impact investing by Nancy MacPherson, Antony Bugg-Levine, Margot Brandenburg, Eme Essien, Laura Fishler, Brinda Ganguly, Sulley Gariba, Heather Grady, Karim Harji, Penny Hawkins, Zia Khan, Justina Lai, Zenda Ofir, Kelly Teevan and Ben Thornley, and also thanks Jennifer DeBien and Sonja Vanek for their assistance.

Notes

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The moral rights of the named author(s) have been asserted.

This article is especially concerned with micro-level results from impact investments that reduce poverty and improve the social and economic wellbeing of the poor and marginalized in other ways. Theory of change also applies to impacts related to environmental sustainability, often at a very large scale, but such impacts are generally beyond the scope of this article.

The assignment of these actors to these particular categories can, of course, be debated. For example, some analysts might place pension funds and sovereign wealth funds in the asset owners’ category. Others might include credit unions in the list of asset managers.

A case study of this kind of approach profiled by the GIIN is that of Mtanga Farms Limited in Tanzania, an agricultural business for small holders in which the Tony Elumelu Foundation, Calvert Foundation, Rockefeller Foundation and Lion's Head LLP have jointly invested, in the process also setting up a common social impact measurement framework (Bouri et al. 2011b).

It should be noted that there is an extensive set of methodologies that has been developed for evaluating the ‘sustainability’ outcomes of a range of environment-related programs (see, for instance, Bell and Morse 2012). Future research on theory of change and impact investing should tap this important pool of knowledge.

In one case, a Nicaraguan cooperative financed by Root Capital created a fund for health after two women coop members had died of cervical cancer. The new health fund paid for the screening of 140 local women, more than 90% of whom were found to have human papillomavirus. Complemented by better household incomes, the health fund is helping local women to address this health issue.