Using “theory of change” to improve agricultural research: recent experience from Tanzania

ABSTRACT

Demonstrating how agricultural research contributes to development outcomes is difficult but necessary given competing demands for scarce resources. This article summarises an adaptation of the “theory of change” approach and lessons derived from its application to improve the design and implementation of an agricultural research for development programme for greater impact. It was applied to Maziwa Zaidi, a programme that tested integrated interventions to catalyse the transformation of smallholder dairy value chains in Tanzania. Despite challenges, the approach was found useful for planning, communication, managing complexity, monitoring behavioural changes and deriving lessons to adapt future programme activities amid complexity and uncertainties.

KEYWORDS:

• Aid – Monitoring and evaluation
• Civil society – partnership
• Environment (built and natural) – agriculture
• Labour and livelihoods – poverty reduction
• Sub-Saharan Africa

Introduction

Evaluating contributions of research to policy outcomes and agricultural development is difficult and complex (Alston 2010; Barnett and Gregorowski 2013), but doing so is critical for deriving lessons for new research with greater impact. Efforts to make agricultural research for development (AR4D) more outcome-based in demonstrable ways has influenced experimentation with various approaches that complement and contextualise the commonly used logical framework or “logframe”. For example, outcome mapping (Earl, Carden, and Smutylo 2001) to link specific changes in behaviours to outcomes; the research and policy in development (RAPID) framework (ODI 2004) that combines episode studies to track backwards to identify key factors that influenced policy changes, and case studies to track forwards from specific research and related activities to assess their impact; and the RAPID outcome assessment (ROA) framework that combines all the above methods (Leksmono et al. 2006; ODI 2012). However, more appropriate methods that are less time consuming and simpler to apply yet able to satisfactorily measure impacts of AR4D addressing complex problems are needed.

The “theory of change” (ToC) has been a candidate for a more satisfactory approach in this endeavour. It has key elements of the above outcome-based approaches, such as a focus on influencing policy, but is considered more flexible. The concept has evolved since the 1960s but started to gain more acceptance in the 1990s (Vogel 2012a). Mostly because of donor requirements, ToC has increasingly been applied to design proposals and evaluate the impact of development projects. However, its utility in AR4D has not been widely explored beyond a few recent examples (e.g. Vogel 2012b; Barnett and Gregorowski 2013; Mayne and Johnson 2015; Thornton et al. 2017; Maru et al. 2018). To contribute to this body of evidence, the approach was adapted and applied in an AR4D context in Tanzania, the Maziwa Zaidi (Swahili for “more milk”) programme to test its practical application to contextualise smallholder dairy value chains with a view to generating appropriate evidence and evaluate progress from research outcomes towards development outcomes.¹

The design of Maziwa Zaidi (MZ) was informed by the need to provide a robust framework for channelling evidence to address persistent challenges in smallholder dairy systems in Tanzania in a sustainable manner. Testing of the use of ToC was to reinforce the robustness of the approach. It was an intentional strategy for the participation of stakeholders to internalise the programme design and evaluate progress against the design. The programme was implemented as one of nine country programmes under the CGIAR Research Program on Livestock and Fish led by ILRI.² Strategic research partners in Tanzania were Sokoine University of Agriculture and Tanzania Livestock Research Institute. MZ also worked closely with development organisations to test interventions, primarily with Heifer International, Tanzania Dairy Board (TDB) and Faida Market Link – a not-for-profit company. It sought to bring together the collective capacity of relevant CGIAR centres and a wide array of international and national public and private partners who, through their long-term involvement on the ground (at both local community and national levels) addressing technical, institutional and policy constraints, could generate evidence to stimulate more widespread research into use. The end goal was to catalyse inclusive smallholder value chains development towards the United Nations Sustainable Development Goals (SDGs) of reducing poverty, ending hunger and improving natural resource systems.

The umbrella global programme and, to a lesser extent, the MZ component and its cluster of integrated interventions reflects the various aspects of complication (lots of parts) and complexity (uncertain and emergent) of interventions that confront programme evaluators, as classified by Rogers (2008, 32). Complexity of the MZ is aptly reflected in this classification because in its design and over five years of implementation, it had multiple agencies in its governance, multiple simultaneous causal strands, different causal mechanisms, non-linear recursive causality and emergent outcomes. The programme comprised a cluster of 12 bilateral projects (Omore, Twine, et al. 2016) with varying lifespans, each with unique commitments to its donors, a situation that invariably causes shifts in context from time to time. Such complexity and inherent dynamism pose a challenge in applying ToC in ordinary circumstances. Applying ToC in a research context where determining the links between research and policy changes is important adds to the complexity (Barnett and Gregorowski 2013).

Articulating and learning about how change is expected to happen in such situations and contribute to it were important aims. Developed three years into the programme, we tested the application of an appropriately adapted ToC to periodically evaluate the programme’s contributions towards an envisaged transformation of the smallholder dairy value chains over the subsequent seven years. Key elements of the testing were its flexibility and robustness to generating appropriate learning. Primary learning objectives were how the ToC would support the implementation of the MZ in future and what lessons could be derived for similar programmes. Secondarily, using the process to self-evaluate progress towards realising more investment as a critical intermediate step towards achieving the programme’s goals was of interest.

This case study responds to the following questions: (1) How did adaptation and use of the ToC support the implementation of the MZ programme? (2) What were the lessons and their implications for effectively using the ToC in similar AR4D programmes? To answer these questions, in the following section, we describe the subject of the case study – the dairy sub-sector in Tanzania, its constraints and the envisaged role of MZ. We thereafter present a short review of literature related to ToC; methods showing how programme managers adapted and tested it; and results over a period of 18 months up to a critical reflection workshop in November 2016 to evaluate progress. The experiences are used to derive lessons and make concluding remarks on using the approach to improve AR4D in contexts like MZ.

Tanzania’s dairy sub-sector and role of the MZ programme

Contribution of the dairy sub-sector

The contributions of the dairy sub-sector to Tanzania economy, at only 1.5% to GDP, are much lower than those of neighbours in East Africa with fewer but more productive cattle numbers (Omore, Twine, et al. 2016); for example, Kenya and Rwanda record contributions of about 6–8% of national GDP. The main reason is that cattle keepers in Tanzania are mostly pre-commercial producers who, due to various challenges identified below, only occasionally sell small volumes to consumers or itinerant milk traders. Unmet demand for milk therefore persists given an increasing population, incomes and urbanisation.

Definition of the issues to be addressed

Previous investments in dairying relied heavily on promoting highly capitalised cold chains. The MZ programme considered these to be unsuitable where quantities produced by individual dairy farmers are small and farmers are dispersed, and where occasional and opportunistic marketing prevails, as pertains to many parts of Tanzania (Omore, Twine, et al. 2016). Installed milk processing capacity (about 420,000 litres per day) has remained grossly underutilised (generally below 30%) for many years. Overall, informal non-pasteurised milk markets offer about 97% of the domestic marketed supply in Tanzania. Therefore, the predominant small-scale pre-commercial producers and actors in the informal value chains were targeted for upgrading.

The value chain assessments identified the following inter-linked challenges that MZ needed to address. First is poor access to inputs (feeds, breeding, animal health), and services (lack of extension services, poor policies) leading to poor disease control, lack of knowledge and information, and poor animal husbandry. Second is the dominance of direct sales of small volumes of raw milk by smallholder producers that precludes economies of scale, therefore high costs of production and marketing and milk safety concerns. Third is the lack of appropriate organisational structures and financial arrangements to increase farmers’ access to basic inputs and services. This has discouraged investment in productivity improving innovations and thus perpetuated the low-input low-output vicious cycle. The assessments also highlighted the potential for value chain growth given the growing demand for milk. Farmers and other stakeholders recognised the need for efforts combining public, collective and private action to address these challenges.

Role of the MZ programme

Leveraging science to overcome value chain upgrading challenges became the subject of MZ. Building on past lessons in the region (e.g. Leksmono et al. 2006; Rao et al. 2016), MZ was designed to generate solutions that catalyse upgrading of the smallholder dairy value chains in Tanzania over a decade, with three major goals. The first is that smallholder farmers have reliable and consistent access to quality inputs and services that enable them to efficiently achieve high dairy productivity; second, for smallholder farmers to have access to an inclusive, reliable, well-coordinated, and efficient dairy products marketing arrangement with resultant improvement in household income and livelihoods. The third goal is directed at consumers and aims to ensure that poor consumers can access quality, safe, and nutritious dairy products at affordable prices, reflected by an increase in their per capita consumption of dairy products. An important indicator of change along the ToC impact pathway would be more private and public investors replicating best bet interventions that increase profitability and generate more household income in inclusive ways. A key dynamic here is how research and evaluation results can influence the behaviours of enterprises, government and donors towards increased investments.

Kidoido and Korir (2013) report how MZ had previously attempted to apply impact pathways to envision how to achieve its aims, but without describing causal assumptions behind the linkages that characterises ToCs. We hypothesised that, as a step towards generating more income, the pre-commercial producers could cost-effectively and sustainably improve the profitability of their dairy enterprises through collective action and participation in multi-stakeholder processes (MSPs) such as market hubs and innovation platforms (IPs). Further, that such processes are a means to enabling producers to overcome market failures occasioned by the common property nature of grazing land in pre-commercial areas, high transaction costs and information asymmetry, and as a result become more commercial. The hubs would allow pooling of a volume of milk attractive to small-scale traders, besides acting as a platform for channelling bundled inputs and services, including credit with milk delivery as collateral, adapting newly tested and existing innovations, and for undertaking further research. Complementing the hubs were IPs for co-creating solutions around common problems by actors including public (research and development) and private (e.g. input service providers, farmers/farmers groups, milk traders and milk processors) participants at local community (Kilelu et al. 2017) and higher levels (Omore, Bwana, and Ballantyne 2016), where the national Dairy Development Forum (DDF) plays a major role.³

Morogoro and Tanga regions were selected as pilot sites. Two districts were then selected per region, each a replicate for increasing level of intensification of production and commercialisation captured through proximity and access to rural versus urban milk markets. Seven to eight villages were selected per district (a total of 30 in four districts) where cattle producers were mobilised to form groups that were thereafter facilitated to develop site-specific plans. These plans informed capacity building and the creation of market linkages to address problems identified with community members. Further, these villages and groups offered opportunities as platforms for experimenting with relevant innovations, adapting technologies and conducting research to identify ways to spread the benefits of proven lessons. Generating evidence on the effects of producers’ association with dairy market hubs was considered key to investment decisions by farmers, firms, government and donors. These included evidence such as whether or not participation in the mostly pre-commercial dairy market hubs improves smallholder farmers’ income (Bayiyana et al. 2018), as a ladder towards more established and capitalised hubs where such benefits have been demonstrated (Rao et al. 2016); improves household nutrition (Mishkin et al. 2018); or is associated with more sustainable farms as determined through various economic, social and environmental indicators.

Investments across projects under MZ mostly covered field activities, complemented by funds from the CGIAR system that mostly covered staff time. These catalytic funds averaged US$2 million a year over the first three years, declining to about half that amount over the following two years. The latter years were also characterised by greater funding uncertainty. Though the declining trend of funding was in line with the envisaged catalytic role for the programme, with relatively more investment for development interventions in later years compared to spending on research (Figure 1), the decline happened sooner than anticipated and adversely affected the levels of engagement between CGIAR and local partners.

Figure 1. CGIAR Research Program on Livestock and Fish approach to a solution-driven AR4D to achieve impact. Source: Tom Randolph, ILRI.

ToC: origins and application

Literature on the origins and use of ToC indicate its evolution from the field of programme theory and evaluation in the 1960s to its more recent emergence as a way of analysing the theories motivating initiatives working for social and political change (e.g. Vogel 2012a). Having no single definition or set methodology, the approach thus allows flexibility according to the needs of a user across several disciplines. Different organisations have worked with ToCs in different ways: for scenario planning; impact assessment; or for a complete planning, monitoring and evaluation process. Whether applied ex-ante or ex-post, Mayne and Johnson (2015) list numerous uses for ToC in designing, managing and assessing interventions. They also proffer advice on the need to avoid complexity in a ToC, for example, through nesting given sub-components into a broader ToC.

The rationale for using ToCs is captured by Rogers, quoted in Vogel (2012a, 4) that “every programme is packed with beliefs, assumptions and hypotheses about how change happens – about the way humans work, or organisations, or political systems, or ecosystems; and ToC is about articulating these many underlying assumptions”. Vogel (2012b) visualises four stages of decreasing control and influence in a ToC. The sphere of control (activities, outputs, communications and networks); sphere of direct influence (immediate target groups where short-term changes can occur comprising partners, collaborators, stakeholders); sphere of indirect influence (e.g. policy shapers, knowledge networks, planners, practitioners); and long-term changes (sphere of interest) that lead to impact. Valters (2014) emphasises the need for a broader commitment to learning from ToC application by individuals and organisations, given complexity of social change across these spheres.

ToC versus the logframe

The relationship between ToCs and logframes is widely debated (e.g. Prinsen and Nijhof 2015). We agree with O’Flynn and Sonderskov (2015) that ToCs do not replace logframes; they are for different but mutually reinforcing purposes. They aver that a ToC embraces the complexity of social change and proposes a model of change that will be tested and improved over time. This requires regular critical reflection and adaptation of the model as contexts change, assumptions are tested, and progress towards desired change is assessed through a systematic process of analysing the feasibility and plausibility of planned pathways to change. This is essential for learning and improving organisational and/or programme effectiveness. They also compare ToC to a “helicopter view” given its focus on how and why change will happen, learning and adaptation across the spheres of influence, the signals of which need to be reflected in logframes. A logframe, on the other hand, is compared to a “road view” because it is designed for management and accountability, focusing on what is to be achieved. Ideally, logframe development should be informed by ToC thinking for it to be an objective management tool.

Approach to adapting and testing the ToC

The process of developing the ToC was a participatory one that required partners and stakeholders be involved in understanding the ToC cycle (Figure 2) and its change pathway diagram (Appendix). About 30 representatives from partner and stakeholder organisations participated in various stages of developing the change pathway, assumptions and reflections. The adaptation required that the key contextual issues around the programme be internalised into the change pathway. It also emphasised having a template that was not time consuming and less burdensome to fill than outcome mapping (Earl, Carden, and Smutylo 2001) to encourage its future use. The identified entry points (market hubs and IPs) informed the participatory prioritisation of initial change areas or theories and assumptions considered key to catalysing further changes.

Figure 2. Theory of change cycle. Source: O’Flynn and Sonderskov (2015).

Following the adaptation, “start point” (or baseline) data on the changes that the programme hoped to influence and monitor in the first year were collected by implementers (Omore, Twine, et al. 2016). The data included information such as context, entry points, main drivers, enabling environment, partnerships, projects, actors, and specific data related to the selected change areas. After one year, a critical reflection workshop was convened in November 2016 to generate new information; update contexts; review progress on planned changes; score the changes; and assess related assumptions, drivers and barriers following the concepts suggested by O’Flynn and Sonderskov (2015). Analysis of changes was conducted under the following sub-headings:

1. What has changed/is changing and for whom?

2. What exactly did MZ contribute to these changes?

3. Who/what else helped/ hindered change and how?

4. What are MZ’s most useful effective contributions and ways of working in relation to these changes and why?

5. Were there other, better ways the programme could have achieved changes and how?

Features of the adapted ToC

The resultant change pathway in the Appendix is summarised in Figure 3 with key assumptions included. The diagrams capture elements of the impact pathways (Kidoido and Korir 2013) developed earlier but with a clearer indication of causal inter-linkages between changes, partners and greater focus on what could be attributed to the programme in the short, medium and long-term at both local community and national levels. A total of 35 capacity and behavioural changes were defined. Out of which 12 were short-term changes (sphere of direct influence), 20 medium-term changes (sphere of indirect influence) and 3 long-term (sphere of interest), all contributing towards the main interest of realising more investments. Most changes (26/35) clearly targeted the local communities involving local value chain actors and local government authorities, while national level changes mostly targeted the DDF. The DDF was proposed by stakeholders for support by MZ as an informal national dairy industry mechanism for information sharing, coordination, dialoguing to co-create solutions and to marshal stakeholders for impact, with its secretariat at TDB. It involves a wide array of public and private partners who interact through physical meetings and social media. Six of the 35 changes (one on contextualisation, three at community, and two at national level) were selected for monitoring in the first year.

Figure 3. Summary of Maziwa Zaidi change pathway.

These initial engagements to develop the template and change pathway (Omore, Twine, et al. 2016) were important to the MZ programme in at least three ways. First, they graphically demonstrated the complexity of the change process and therefore a general acceptance of the importance of various interlocking contributions and the need for improved communication in the MZ partnership. Second, out of the complexity a clear intermediate goal of attracting more investments was re-affirmed and theories on areas where MZ had a realistic chance of making a difference generated. This entailed a critical evaluation of how research can influence the behaviours and investment decisions of farmers, private enterprises, government and donors, which in turn informed the prioritisation and selection of change areas for monitoring. At the same time, in contrast with more traditional approaches common in agricultural research institutions that target specific technical or biophysical areas to resolve value chain problems, the change pathway retained an awareness of the breadth of critical socio-political contexts. Third, there was a specific recognition of the importance of coordination for activities, ranging from smallholder farmers to research institutions and government instrumentalities, and for advocacy at local and national levels.

Applying the adapted ToC

The November 2016 critical reflection workshop attended by 30 stakeholders comprising public (research and development) and private sector actors mostly drawn from participants in the DDF provided the first opportunity to critically reflect on the adapted ToC and how it can guide adaptation of the programme. We present below a synthesis of highlights of this experience and examples from the workshop report by Omore and Kidoido (2016).

Review of context

Potential drivers: The reflection re-affirmed the main drivers for changes in the value chain, the key one being increasing demand for milk and the appropriateness of the selected entry points for interventions. An enabling environment was signalled by ongoing interest by the government to stimulate growth of the value chain, though this is yet to be reflected in well-articulated programmatic interventions and increased budget allocations. While these drivers could have been derived from any archetypal stakeholder consultation, what was unique about the ToC was describing linkages and causal assumptions behind them.

Initial barriers and challenges: A major ongoing challenge highlighted across all activities was limited, and often unpredictable, funding for operations among partners, especially by local government authorities (LGAs) where the MZ pilot sites were located. This was partly due to low prioritisation of dairying in some districts due to under-appreciation of its potential for improving livelihoods, generating a feedback cycle from low prioritisation and budgetary allocation which resulted in underinvestment and an overall low contribution of the sub-sector to the economy.

Assessing changes and assumptions

Monitoring by programme managers was an ongoing process across MZ projects based on household surveys, interviews with stakeholders and refection workshops, but the cumulative changes were examined in detail at the November 2016 workshop. Though the focus was changes over the previous 18 months, it is conceivable that some reflected on changes since the start of the programme. The questions that guided analysis of changes and MZ contributions were found useful for the reflection. The full analysis that considered changes across 162 categories is captured in the engagement data summarised in Table 1 (detailed in Omore and Kidoido 2016). We highlight below some reflections behind the scores and underlying assumptions.

Table 1. Theory, assumptions, key actors, status at start point and observed change at reflection.^a

Change area or theory #	Main level of change	Assumptions	Key actors^b	Status at start point	Change score at reflection^c
#3 Research and development partners contextualise approaches to meet needs of local men and women	Local community and national	Promoted approaches are appropriate for smallholder farmers. Partners engage concertedly to achieve their institutional and programme objectives and have flexibility to test alternative approaches	Local men and women; MZ implementers	R&D partners’ mandates are not exercised to address local needs	2.5 (emerging to established)
#5 Processors, traders and service providers respond to business opportunities created in the hub	Local community	Value chain agents are behaving competitively, are maximising profit and there are no information asymmetries	Value chain actors; MZ implementers	Low private investments; low volumes of transactions	2 (emerging)
#8 Smallholder farmers access and experiment with gender-sensitive and relevant innovations including groups	Local community	Groups not involved in collective bulking and marketing of milk are able to meet farmer needs	Farmers; MZ implementers	Limited opportunities to experiment and innovate	3 (established)
#10 DDF engages and mobilises stakeholders effectively	National	The DDF is the right modality to drive the chain	DDF Secretariat; DDF advisory committee	No DDF before MZ; few VC actors initially	2 (emerging)
#11 DDF members are aware of evolving opportunities for investment	National	DDF effectively communicates emerging investment opportunities	DDF Secretariat	No DDF before MZ; few VC actors initially	2 (emerging)
#13 Local government includes hub development and gender issues into their plans and budgets	Local government	Commercial milk production and gender issues are considered by LGAs as a priority for food and nutrition insecurity and in improving household incomes	LGAs	No “hubs” approach and gender is not planned and budgeted	2 (emerging)

^aFull range of scores for changes were: early stages = 1; emerging changes = 2; established change = 3; (very) significant changes have taken place = 4.

^bFull list of key actors available in Omore et al. (2016a).

^cFurther details in Omore and Kidoido (2016).

Assessing scores: Changes were scored as either “emerging” or “established” for each theory (Table 1). Established changes were in relation to contextualisation of approaches (#3) and in providing farmers with the opportunity to access and experiment with gender-sensitive and relevant innovations (#8). Women were a majority in the newly formed groups where they were able to articulate their demands through site-specific plans for extension support, besides engaging as milk traders in several villages and taking up leadership positions in some farmer groups. Also noted was increased uptake of new feed and improved forage varieties by women. However, gender awareness and integration in local government activities at the village level was still low, though partially mitigated by gender awareness training and deliberate inclusion as group leaders by MZ. These inadequacies require additional public investments for a longer period before private capital can yield positive returns (Dizyee, Baker, and Omore 2017; Kilelu et al. 2017).

The most effective overall contribution of MZ was deemed by stakeholders to be at the national level in relation to catalysing the emergence of the DDF (#10 and #11). The DDF was assessed as an effective mechanism for information sharing and communication for innovation among actors. Additional steps to further strengthen the DDF by formalising it as a private sector-driven organisation have been taken thereafter by the TDB and World Bank (through a project pursuing micro-reforms in agriculture) and endorsed by TDB’s Annual Council in September 2017. In May 2018, TDB ceded its role as secretariat of the DDF to interim private sector officials who would oversee steps towards its registration.

Assessing assumptions: A series of assumptions about behaviours and motivations underlies the ToC (Table 1). In re-examining the assumptions, participants judged those for theories #3, #10 and #13 to be true, while the assumptions for theories #5, #8 and #11 were judged to be false. This was an important finding. Going forward, the programme needs to assess how the assumptions that have been judged as “false” may have affected the programme and their implication for adapting the change pathway. So, revisiting the ToC in light of the falsity of the assumptions is a priority for subsequent monitoring. In addition, a correct underlying assumption does not necessarily translate into a completely successful result. For example, in relation to theory #10, the assumption that the DDF is the right modality to drive the chain was judged as valid but results so far are mixed. It was noted that the informal nature of the DDF may not be appropriate for advancing policy advocacy, though it has so far been useful in raising awareness. MZ’s most useful and effective contributions and ways of working were to be considered into the next round of priorities.

Lessons learnt from adapting and applying the ToC

The main thesis of this paper is the issue of how to appropriately use the ToC to improve an AR4D programme like MZ. The approach supported the MZ programme to better understand how research can more effectively contribute to alleviating persistent constraints in the value chains in several ways.

Visioning and designing interventions

We think the ToC process could have adequately replaced previous visioning at the beginning of the programme such as outcome mapping and impact pathways (Kidoido and Korir 2013) to prioritise interventions and guide implementation. Though the impact pathways were useful building blocks, the added value of the ToC was that it facilitated clearer definition and agreement on key changes, their linkages and assumptions along the impact pathway, while making explicit the various perspectives and drivers of change and expressing the underlying hypotheses. Developing the ToC at the start of the programme (rather than three and a half years into the programme as we did in this testing) would have better clarified respective roles of collaborating research and development (R&D) partners at the start and in the spheres of direct and indirect influence, thus pre-empting the difficulty experienced in defining the “we” in “what we do … ”, “with whom … ” (Figure 3 and Appendix). The earlier timing would also have improved communication from the onset and helped to better manage complexity.

Improving communication around a common purpose

The ToC process enhanced greater openness and collaboration among actors and partners as noted at reflections and evidenced by joint publications that capture lessons on linkages for value chain upgrading and innovation (e.g. Omore, Bwana, and Ballantyne 2016; Kilelu et al. 2017), including syntheses for a policy forum on exploiting evidence for inclusive investments (Kawuma et al. 2017); itself a product of ToC thinking. Materials for this forum showed the importance of thinking about outcomes at the onset (Vogel 2012b). The ToC process thus enhanced communication by facilitating mutual understanding of the complementary roles of collaborating R&D partners in testing interventions and how they lead to agreed goals.

Managing complexity

We think the ToC process helped partners to clarify interlinkages along impact pathways. As a result, they better appreciated the complexity of the programme and their respective roles. The fact that only six of proposed 35 changes were prioritised and monitored in the first year showed how ambitious the programme was and, at the same time, helped in managing partners’ contributions. The ambitious nature of the goals was confirmed at the reflection workshop where most changes were scored as “emerging” and therefore requiring both further time and monitoring before they could become established. It has been suggested that nesting complex ToCs (Mayne and Johnson 2015) can help manage this kind of complexity. A nested ToC for research within development component is recommended for MZ, where for example, distinguishing R&D roles could have better clarified complementary roles of partners, a difficulty alluded to above. Even then, the observation that a ToC is only a part of causality in complex programmes (Rogers 2008) should not be lost. Alternatively, fewer changes are recommended for a more manageable ToC and programme.

Flexibility, efficiency and sustainability

Despite challenges in clarifying linkages and causal assumptions behind them, the ToC was found by programme managers to be a flexible and holistic tool for planning, design and evaluation in MZ. The ToC helped both to identify critical gaps towards the goal and allowed flexibility that lends itself to addressing the most critical prioritised changes amid funding uncertainties – a scenario that prompted independent evaluators of the programme to conclude that though the value chain AR4D approach is promising, it needs to be fully implemented through adequate resourcing. Inherent in the approach of MZ was how positive changes can not only be catalysed but also sustained. The ToC process revealed how unpredictability in funding (and less so the budget amount) was the most severe constraint curtailing the effectiveness of partners, especially for LGAs, who could not, as a result, engage systematically in planning and executing prioritised upgrading activities. Severe funding unpredictability would certainly render use of any ToC challenging.

Applying the ToC at the beginning of a programme to generate and answer specific prioritised evaluation questions and address hypotheses mapped to each assumption can enhance efficiency for researchers. In doing this, it is important to be flexible by reflecting critically, on an ongoing basis, the research outcomes regarding what results can be used by who and for what in each period.

Efficiency at all levels can also be improved through more concerted, coherent and coordinated efforts over meaningful periods linking donor and national sources of financing for a programme geared towards achieving national goals and the SDGs. Despite high transactions costs often associated with collaboration and collaborative synthesis of lessons generated from complex programmes (Cochrane and Cundill 2018), greater collaboration over longer periods by CGIAR and national R&D organisations is still recommended for efficiency and sustainability. This also requires strong national leadership, at least in thought guidance if not institutionalised, to enhance efficiency in implementation and scaling of complex AR4D initiatives like MZ with ToC as an aid. Efficiency can also be enhanced if the ToC is institutionalised into regular planning and reporting within and between organisations and to donors.

Concluding remarks

While our experience in applying the ToC approach to improve its implementation is consistent with the widely documented difficulty of determining the precise links between R&D outcomes (e.g. Barnett and Gregorowski 2013), we nonetheless draw on the experience of applying the ToC to suggest the following.

First, the approach was found useful in helping to clarify, better contextualise and communicate the problems that needed to be addressed in a complex system involving a wide range of actors. It identified key areas for participatory approaches, stakeholder involvement and capacity development for value chain development. The approach was useful for factoring into the research process the broader socio-political context that is important for influencing research uptake, as well as clarifying linkages and causal assumptions behind the linkages. Improved communication between R&D partners resulting from engagement and feedback around the ToC potentially enhances the translation of research into use.

Second, the approach can be applied to improve the design and implementation of future AR4D efforts by focusing more explicitly and periodically reflecting on what it aims to change to achieve development outcomes and to provide feedback and engagement data that guides programme learning and improvement.

Third, R&D partnerships in a programme like MZ should have nested ToCs that clearly draws a distinction between causal and implementation theories and distinguishes respective roles that exploit identified synergies. In this case study, the envisaged stronger formalised DDF could potentially become the custodian of a development ToC for MZ or similar efforts to transform the smallholder dairy value chains. It could play a central role in continuing to catalyse the envisaged transformation in the sector, through what Bhola (2000) refers to as “impact by interaction and emergence”.

Finally, continued use of the ToC approach in programmes like MZ will require greater recognition of the need to be flexible (already inherent in the approach) given uncertainties such as funding and changing contexts.

Acknowledgements

We extend our appreciation to MZ implementing partners, collaborators, dairy value chain actors. We acknowledge the anonymous reviewers for their helpful comments.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Amos Omore is a Senior Scientist at International Livestock Research Institute (ILRI), Dar es Salaam, Tanzania.

Michael Kidoido was formerly a Scientist at ILRI, Nairobi, Kenya.

Edgar Twine was formerly a Scientist at ILRI in Dar es Salaam Tanzania. He is currently Associate Principal Agricultural Economist at AfricaRice, Bouaké, Côte d'Ivoire.

Lusato Kurwijila is a Professor of Dairy Science at Sokoine University of Agriculture, Morogoro, Tanzania.

Maureen O’Flynn is a consultant and Senior Associate for International NGO Training Research Centre (INTRAC), Oxford, UK.

Julius Githinji is a Research Assistant at ILRI, Nairobi, Kenya.

Additional information

Funding

This article is based on research conducted through the CGIAR Research Program (CRP) on Livestock (former Livestock and Fish). We thank all donors that globally support our work through their contributions to the CGIAR system. We particularly thank Irish Aid and International Fund for Agricultural Development for funding extensive field activities.

Notes

1 Details on the programme and its publications are available at https://maziwazaidi.org .

2 The other value chains where similar efforts were carried out by the global programme were dairy in India, dual purpose cattle in Nicaragua, pigs in Uganda and Vietnam, small ruminants in Ethiopia and Burkina Faso, and aquaculture in Egypt and Bangladesh. The proposal is available at http://hdl.handle.net/10568/3248 .

3 More information on the DDF is available at http://ddftz.ilriwikis.org/Home.

Appendix. Maziwa Zaidi change pathway diagram