Sleeping with the enemy? Capturing internal risks in the logical framework of a water management project

Abstract

Logframes are fundamental to contemporary development. However, there are ongoing debates about their efficacy. This paper pinpoints the limitations of the logframe approach in a water project in Central Asia. Issues surrounding logframes are identified. These include addressing internal risks; the use of baseline studies for the accuracy of assumptions; the ability to adapt under an inflexible budget; and linking the logframe and contract. Findings show that rigid planning may constrain effective project implementation. Greater flexibility through embedded learning and adaptation, adjustable budgets and meaningful mainstreaming of risks may equip projects to cope with uncertainties to achieve sustainability.

Keywords:

• transboundary water cooperation
• integrated water resources management (IWRM)
• Ferghana Valley
• Syr Darya basin
• Central Asia

Introduction

In the past Logical Frameworks (or logframes) have been widely credited with bringing more efficiency to development (Coleman, 1987; Cordingley, 1995; Wiggins & Shields, 1995) and today they are ubiquitous in modern-day development projects (Bakewell & Garbutt, 2005; Dearden & Kowalski, 2003; USAID, 2012). The utility of logframes has nonetheless been challenged. Some critique them as a mechanistic, controlling and rigid tool (Gasper, 2000; Mowles, 2010; Rowlands, 2003). Others note that logframes have limited ability to cope with external factors, unexpected events, contextual uncertainties and human related changes (Dale, 2003; Hummelbrunner, 2010; Ward & Chapman, 2008).

Analytical work in the context of donor-assisted transboundary water projects has not touched on the use of logframes specifically, but has nonetheless devoted attention to the importance of proper planning (Jägerskog & Zeitoun, 2009; Mostert, 2005; Wolf, 2001). Wolf (2001) acknowledged the significance of baseline knowledge, financial resources, process and third parties. Mostert (2005) recommended that donors evaluate possibilities and limitations with respect to capacity and financial resources, and suggests that in some social and political environments stakeholder involvement may not be desirable (Mostert, 2005, p. 29). However, Jägerskog & Zeitoun (2009, p. 8) asserted that the drawback in external assistance is ‘inappropriately programmed to finance processes without a clear outcome and timeline’. The implication is that without proper situational analysis, clear strategy, long-term commitment, suitable expertise and appropriate funding, donors cannot achieve long-term sustainability in transboundary water cooperation projects. Lacking in this literature is a practical grounding in a water projects and a specific focus on the role of logical frameworks.

The objective of the paper is to evaluate the use and drawbacks of the logical framework approach in the context of a transboundary component of a large water management project in Central Asia. This paper juxtaposes attempts to improve water cooperation on two transboundary tributaries as a means to revealing concrete factors related to planning that led to delays and implementation issues. Identification of such factors is in turn used to generate recommendations for how implementation could be improved in future projects. The paper begins with a theoretical and conceptual framework section, explaining logframes, especially with regards to the concepts of uncertainty and risks and budget control. Next, it proceeds with the background materials section, which introduces the study area, project and methodology. The subsequent section then provides the results. The discussion relates the findings back to the literature, and the conclusion considers broader theoretical and practical implications of the study.

Theoretical and Conceptual Frameworks

The logframe

The logframe is used to present and organize inputs and objectives within complex development projects in a structured way. It is typically portrayed in a matrix consisting of 4 ×  4 columns that provides a concise one-page statement of the major project elements and their relationships to each other (Ingle, 2009).

The logframe and uncertainty

Uncertainty and risk are the two concepts that are inherent to the logframe. Ward and Chapman (2008) mention internal uncertainties related to the estimates of project parameters, logistics and relationship between parties. Petit (2009) added that unpredictable behaviour of people and variability of inputs can also contribute to a risk, which is a measure of uncertainty. To cope with uncertainties and risks, some researchers emphasized the importance of the essential and baseline information prior to development of logframes; lack of accurate data about the local context could endanger the success of the project making failure inevitable (Akroyd, 1999; Dale, 2003; Hummelbrunner, 2010, ICF, 2012; Uitto, 2004). Aune (2000, p. 687) stated that ‘the more you know about the external factors which can influence a project, the more you can plan in detail’. Armytage (2011) further argued that data and stakeholder perceptions should be collected systematically to inform management about any deviations and unexpected changes.

The Assumptions & Risks column in the logframe matrix arguably already takes into account the external and internal uncertainty factors. Aune (2000, p. 688) stated that the Assumptions & Risks column ‘describe[s] conditions that must exist if the project is to succeed but which are outside the control of the project’, implying that results cannot be delivered if the appropriate environment is not there. Odame (2001) argues that the Assumptions & Risks column includes the contextual and content-related issues that can affect the project. Örtengren (2004) noted that risk analysis enables assessment of conditions under which the project is working and divided risks into two:

• External risks – those outside the scope of the influence of the project, i.e. natural disasters, political development, etc. They can be ‘derailing factors’ if they make the goal fulfilment impossible.

• Internal risks – the projects presumably control these risks, as they are usually practical matters such as delays in deliveries and staff turnover, etc.

In principle, assumptions relate to issues that are critical to the project's success but outside of the project's capability to influence (Örtengren, 2004). These include political institutions, norms, laws and policies. These then create assumptions for the projects, which can be favourable or not. The project's assumptions can be set at different levels, and should take into account whether satisfaction of assumptions is critical to the project's objective fulfilment, e.g. stable political environment. Hence, it should only be a reasonable assumption. If it is not a reasonable one, it should be included as a risk. If it is a killer risk, the project should reconsider whether to continue the implementation or not (Örtengren, 2004).

Budget control

Budget is an important project element that will naturally be formulated after a thorough analysis of the logframe has been conducted. Two types of budget control are discussed below. Merchant's study (as cited in Conboy, 2008, p. 2) refers to ‘tight budget control’ as ‘low tolerance for interim budget deviations, detailed line-item follow-ups, intense discussions of budgeting results, emphasis on meeting short-run budget targets, and level of tolerance for budget revisions during the year’. The focus here is on meeting the budget, increased operational level management and higher precision in accounting. Periasamy (2010, p. 567) mentions the ‘fixed budget’ as is ‘designed to remain unchanged irrespective of the level of activity actually attained’. Van der Stede's study (as cited in Conboy, 2008, p. 3) mentions ‘loose budgetary control’ where ‘management do not routinely inspect deviations at all, or do so only if there is ‘something clearly amiss’. Periasamy (2010, p. 567) along similar lines mentions ‘a flexible budget’ as one ‘designed to change in accordance with the various level of activity actually attained’.

Background materials

Study area

The project focused on two tributaries of the Syr Darya River within Ferghana Valley. The names of two transboundary tributaries are Shakhimardansai (shared between Kyrgyzstan and Uzbekistan) and Khojabakirgansai (shared between Kyrgyzstan and Tajikistan). The Ferghana Valley is part of the Syr Darya basin in Central Asia (see Figure A1 in the supplemental data online). The Syr Darya basin is shared between Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan. There are 20 mountain rivers within the valley, which are small transboundary tributaries (STTs) of the Syr Darya (Wegerich, Kazbekov, Mukhamedova, & Musayev, 2012).

Water management in Syr Darya

During Soviet times, Syr Darya was managed as one unit. With independence in 1991, integrated basin planning collapsed. In the context of a US Agency for International Development (USAID) project, a barter agreement on water–energy swaps was adopted in 1998 between the riparian states of Kazakhstan, Kyrgyzstan and Uzbekistan on the operation of Toktogul reservoir (Lange, 2001; Weinthal, 2001). Following this agreement, the United Nations Special Program for the Economies of Central Asia (SPECA) was launched in 1998. The focus on high-level water cooperation in Central Asia has met with mixed success (Libert, Orolbaev, & Steklov, 2008; Wegerich, 2008). Later the focus of SPECA was transformed onto small rivers, such as the Chu and Talas. The project on the two STTs (Shakhimardansai and Khojabakirgansai) can be seen as part of the shift of donor focus from the larger contested basins to sub-basins.

Project information

Project origins

In the period 2001–12, the Swiss Agency for Development and Cooperation (SDC)1 funded the Integrated Water Resources Management in Ferghana Valley (IWRM-Ferghana) project, with the International Water Management Institute (IWMI) and the Scientific Information Center (SIC) of the Interstate Commission for Water Cooperation as implementing partners. The project was initiated by IWMI. In the concept note submitted to SDC, to reform water management in Central Asia, IWMI identified SIC as a partner. Through this project IWMI has established its presence in Central Asia.

Organizational decision-making

Cultures of the two organizations varied somewhat. While it may not be possible objectively to describe the decision-making dynamic within IWMI,2 the culture of international research organizations typically elicits a reasonable level of participation. Indeed, many IWMI staff possess ‘softer’ or more socially oriented backgrounds which would help engender discussion and participative decision-making. SIC, by comparison, possessed strong leadership and expertise in technical matters of water management. However, as a local organization recently removed from a Soviet culture of decision-making, the internal management style for project implementation differed from IWMI. Usually different people were responsible for managing different components, and such components operated somewhat independently. Further, the degree to which those at the lower end of project hierarchy participated in decision-making may not have reached the levels evidenced in an international research organization.

Project goals – focus on national canals

The main goals of the project were to increase environmental sustainability, secure productive livelihoods, and promote greater social harmony through the improved effectiveness of water resources management (IWRM-Fergana, 2013). The project mainly focused on one canal in each country of the Ferghana Valley: Kyrgyzstan, Tajikistan and Uzbekistan. The key idea was transferring water management responsibilities from state agencies to participatory bodies while promoting the hydrographic principle of water management. From an implementation perspective, this meant the creation of Water User Associations and their scaling up into federations, with more room for other non-agriculture users to participate, along the main canal. One unified canal management organization was established instead of multi-district-based administrative-territorial units. The canal management organization was set to be accountable to a joint state–user governance structure.

Project expansion – going transboundary

In 2007, the project expanded its activities to two STTs: Shakhimardansai and Khojabakirgansai. The goal was to replicate the experience gained in the national canals to the transboundary context. The project anticipated building bottom-up water governance units for the STTs. On each riparian side it was anticipated to form a Sub-basin Water Committee based on Water User Associations and key state organization representatives. The Sub-basin Water Committees of both riparian sides were supposed to work out and agree on joint river-wide institutional arrangements.

This paper considers the whole project, both the national (2001–12) and transboundary periods (2007–12), but on the use of the logframe and implementation of activities the emphasis is placed on the transboundary (STT) component of the IWRM-Ferghana project.

Project planning

The project utilized the logframe as its main project-planning mechanism. All phases, with the exception of phase I, had an independent logframe. The logframe was part of the main project document and contract; hence, it could not be changed or adapted during the phases. The only mechanism to undertake modifications was to change the Yearly Plan of Operations through the Annual Stakeholder Meetings and the decision of the Project Steering Committee, which consisted of representatives from ministries, the SDC and project leaders (without voting power) of the implementing partners. A detailed overview of the project is given in Table A1 in the supplemental data online.

Within Table A1, the first column shows different phases of the project (Phases I–VI) and the second column the duration of each phase. The third column illustrates the project outcomes planned versus achieved; and the fourth the basic elements of the project logframe's impact/vertical logic (goal, objectives, results and activities). Phases in red have specific relevance for this study. The STT component was added in 2007 (during Phase III) and the initially developed logframe back in 2005 was not updated nor were the STT activities integrated. Only the list of planned activities were integrated into the Yearly Plan of Operations of 2007 and 2008, therefore only logframes for Phases IV–VI with an accent on the STT component will be considered in the analysis.

The main planning document was the Project Document. Each phase would have its own document; usually a template would be provided by the donor which would consist of a description of the overall project plan, logframe and budget. The project implementers – the IWMI and SIC (partners) – would be responsible for document preparation.

The actual planning procedure would be first to deal with the allocation of the total budget amongst partners specified by the donor before the start of each phase, regardless of the capacities available or scope of the work. Partners would already know the total budget for the given phase when the project platform (scope of work) was provided by the donor. Afterwards, the objectives, results and list of activities would be matched to the agreed and already allocated budget of the partners. Usually within this project, the logframe was developed at later stages of the project planning, once the partners had already agreed on the budget allocations, main results and activities.

Methodology

Data and information collection

Data utilized included logframes, project documents, budget plans, Yearly Plans of Operations, project agreements (including addendums), annual progress reports, the SDC technical backstopper's mission reports, external review reports, and the SDC Water Sector Strategy for Central Asia. Secondary data sources such as reports of water management departments, field teams' progress reports and personal communication with key stakeholders were also utilized (see Table A2 in the supplemental data online).

Literature review and identification of central factors of the logframe

Literature related to project planning and guidelines of aid organizations with specific reference to logframes were collected and examined. Journal articles and evaluation reports that directly addressed the logframe approach were considered and, in addition, the guidelines of USAID and the Swedish International Development Cooperation Agency were reviewed. The analysis showed that donors are aware of the limitations of logframes and emphasized the prior training and capacity building towards proper use of the logframe. The critical review of the literature considered the theory of change, which is the fundamental basis of logframes; the dealing of logframes with uncertainties and budget constraints. The review helped to identify four issues central to the utility of the logframe: (1) the Assumptions & Risks column; (2) lack of baseline knowledge; (3) budget control; and (4) internal and external uncertainties. These four factors provide the conceptual basis for the methodological approach. Details are provided on each of the four areas immediately below.

Assumptions & Risks column

To assess the Assumptions & Risks column of the logframe matrix, data from project logframes were utilized. Only the STT component and three logframes for Phases IV–VI were considered. For the purpose of the investigation, the vertical logic (objectives/outcomes, results/outputs and activities) and Assumptions & Risks column of the logframes related to the STT component were extracted into a separate table. The columns of logframe matrix such as ‘Objectively verifiable indicators' and ‘Sources of verification’ were omitted because the focus of assessment was to analyse the Assumptions & Risks column, although the above-mentioned two columns could also generate discussions on whether the project focused on showing progress and collecting data for indicators. As argued by scholars, these two columns illogically separate the Assumptions & Risks column from the vertical logic (Gasper, 2000; Hummelbrunner, 2010).

Baseline knowledge

The annual report of the STT component for 2007 (Rysbekov, 2007) was closely studied. It presented the data on hydrology and water withdrawals, existing agreements and information on environmental, water quality and technical condition of the flow monitoring on selected STTs.

Budget control

To evaluate the budget control and its linkage with the logframe, the project budgets were considered for all phases. Personal communication with the project accountant was instrumental in understanding the changes in the budgeting systems introduced by the donor. The budget information for each phase was organized into a table with four columns: (1) phases; (2) total budget; (3) budget allocated by partners; and (4) degree of budget control (‘strict’ or ‘loose’). Instead of monetary information, the percentage allocation of the budget are presented for simplicity of the analysis. In terms of measuring the degree of budget control, each phase was classified into either ‘Loose budget control’ (Conboy, 2008, p. 2)/‘flexible budget’ (Periasamy, 2010, p. 567) or ‘Tight budget control’ (Conboy, 2008, p. 3)/‘fixed budget’ (Periasamy, 2010, p. 567) based on expert justification of the authors in consultation with the project accountant.

Internal and external conditions

To analyse the impacts of internal and external conditions of the project, different sources of data were utilised: Yearly Plans of Operation 2007–12, minutes of project meetings and reports (annual progress, back to office and external review), which were supported by the personal experiences of the authors. The assessment resulted in the emergence of two subtopics related to internal uncertainties: (1) activities, partnership and staffing issues; and (2) management and reporting system change, which appears to be significant for understanding the actual practice when utilizing the logframe.

The identified four central issues were explored in the IWRM-Ferghana project.

Results

Assumptions & Risks

Although internal factors such as relationship, outlook of project partners, or differences of organisations, its structure and staffing were often highlighted as being of concern to the external reviewers, these internal factors (as explained below) were never addressed within the logframe as assumptions or risks. Similarly, budget control or changes from flexible to strict budget control (again as explained below) was missing.

External factors were only marginally reflected in Assumptions & Risks. There are several issues that can be seen from Table 1, which presents the project's impact logic (objectives/outcomes, results/outputs and activities), its summary and the Assumptions & Risks column. First, in Phase IV it is mentioned in the Assumptions & Risks column that government authorities are supportive of project activities, some government employees have even worked full or part-time in the project. Second, various assumptions are non-valid as they do not reflect the current reality. Formulating the assumption positively or neutrally already ignored existing non-cooperation amongst riparian countries on a national level and existing border crossing concerns (Megoran, 2006). They are exemplified by Result 2/6, Activities 2/10 and 3/10 in Phase IV. Third, some are treated as mere statements and similar as activities, e.g. the assumptions in Activities 2/25 and 4/25 in Phase V, and Outcome 1/2 and Outputs 1/4 in Phase VI. Fourth, the assumptions focused on national levels, e.g. inter-state cooperation, though STT, e.g. local-level cooperation was the heart (IWRM-Fergana, 2013; Wegerich, Kazbekov, Kabilov, & Mukhamedova, 2012).

Table 1 Extraction of objective hierarchy and its corresponding Assumptions & Risks

	Narrative summary	External factors (Assumptions & Risks)
Overall goal	Contribution to more secure livelihoods, increased environmental sustainability and greater social harmony through improved effectiveness of water resources management in Ferghana Valley
Phase IV
Project objective 1/3^a	Pilot areas and selected small transboundary tributaries (STTs) function according to true integrated water resources management (IWRM) principles. Approaches are consolidated and ready to be promoted in other areas of the region	Governments and ministries continue to support the project activities
Result 1/6	IWRM water governance principles and management procedures are fully introduced and practised at pilot canals and STTs	Ministries and local authorities continue to support the reform process
Result 2/6	Joint commissions are established and IWRM implementation in the pilot basins of small transboundary rivers introduced and agreement on joint management of water resources is reached (if possible)	Involved governments are positive towards these developments and maintain a cooperative relationship
Activity 1/10	Establish IWRM governance and management fully on both sides of the pilot STTs	National border conditions are conducive for project activities
Activity 2/10	Develop and establish institutional (joint commissions) mechanisms for STT	Bordering governments agree for such commissions
Activity 3/10	Prepare draft agreements for joint water resources management, a list of required structures and equipment, alternative procedures, etc. for STTs	Bordering governments remain positive about such developments
Phase V
Outcome 1/2	IWRM institutions (water user associations, unions of canal water users and canal management organizations, including institutions in the STT area) in the project areas are self-reliant	Favourable economic conditions in all project countries are assumed
Output 1/6	The functioning of governance bodies of IWRM institutions within the project areas is strengthened	Canal water committees and unions of canal water users could not get financial ability for effective interaction
Output 2/6	National visions on ways and means to integrate the new IWRM organizations in the existing water management structures as well as on STT management are elaborated based on a broad consultation with stakeholders	n.a.
Activity 1/25	Finalize the establishment of system water committees in the Kyrgyz and Uzbek parts of pilot STTs (Shakhimardansai and Khojabakirgansai)	n.a.
Activity 2/25	Build/strengthen the capacities of established union of system water users, system water committees and system management units at each riparian side of STTs (needs based)	Assumption in close collaboration with the project of the Deutsche Gesellschaft für Internationale Zusammenarbeit
Activity 3/25	Facilitate the official cooperation and dialogue between riparian sides of two pilot STTs	n.a.
Activity 4/25	Elaborate a donor action plan for Khojabakirgansai and coordinate with the Deutsche Gesellschaft für Internationale Zusammenarbeit, SDC and relevant ministries of the two countries	In cooperation with the Deutsche Gesellschaft für Internationale Zusammenarbeit
Phase VI
Outcome 1/2	Organizational and managerial capacities of IWRM institutions (water user associations, unions of canal water users and canal management organizations, including institutions in the STT area) in the project areas are strengthened	Both riparian sides at two pilot STTs agree to institutionalize their transboundary cooperation
Outputs 1/4	Basic institutional arrangements for IWRM structures are established and local ownership is enhanced	Fundamentals of local ownership are in place
Activity 1/10	Finalize the ongoing transboundary consultation process with joint river-wide arrangements for each pilot STT	n.a.
Activity 2/10	Facilitate the operationalization of the river-wide agreement or Memorandum of Understanding for each pilot STT into a specific short- to mid-term collaborative basin-wide action plan	n.a.
Activity 3/10	Ensure sufficient local ownership of the established transboundary agreements and arrangements at each pilot STT	n.a.

a Note: 1/3 – this fraction symbol indicates that this particular objective/result/activity is one out of three. This fractioning applies for whole table.

In this project, unfortunately, Assumptions & Risks were grouped together in one column, although they are usually analysed differently and thus are put into different boxes. This created a situation where the plans are uncertain. Moreover, if the project success depends heavily on the elimination of the risks, then some activities need to be devised in order to prevent the risk from happening in the first place (mitigation strategies).

Baseline knowledge

The technical focus of the initial data collection impeded the participation process. During Phase III, the SIC was mainly responsible for the STT component, and therefore also for the baseline surveys. Since the SIC was more technically orientated, the baseline studies reflected their expertise. The baseline survey had three components:

• Interstate water allocation (normative and actual) in 1980–2005 in the selected STTs; specifying irrigated areas by small rivers, canals and mixed irrigation areas using geographical information system (GIS); analysis of cropping patterns and irrigation schedule.

• Analysing existing legal aspects of interstate water allocation for identifying water allocation for STTs.

• Studying water infrastructure in border regions with transboundary impact; developing proposals for equipping waterworks of transboundary importance with water meters and water distribution facilities; establishing monitoring and control mechanisms for improving water use efficiency.

Although these studies yielded a significant amount of information about the pilot STTs, these baseline reports still lacked the basic information on stakeholders, institutions and informal cooperation. Important aspects were missing within the baseline reports such as the analysis of the wider regional context and existing legal and institutional framework for transboundary water cooperation (whether this be regional, multilateral or bilateral). The analysis of legal aspects in the baseline focused only on old Soviet agreements for the specific STT. Overall, it lacked practical information, regarding border operation, access within the basin and distances for alternative plans. Hence, the baseline survey was not meant to enable a participatory approach.3 Distance-wise compared with project canals, which were located within one country, STTs were in fact between 300 and 600 km apart, and spanning three countries. Furthermore, the borders between these countries were open irregularly. Administrative–geographical remoteness meant longer trips on poor-quality roads and difficulties with crossing borders, causing logistical challenges and increased costs. These factors were not taken into account in the baseline studies. Taking into account that the SIC dropped STT activities completely when starting Phase IV, the baseline information collected by the SIC was not practical and the IWMI had to gather more information during Phase IV.

Budget control

The project logframe failed to adapt to budget changes due to the absence of a link between them and the budget was an impending element for coping with uncertainties. The budget is one of the components that is separate from the logframe but which is very much determined by various components in the matrix (e.g. activities and inputs). Budget-wise, this project faced difficulties from the start; there were informal disagreements about the ‘equal’ partnership stance about whether it should still be equal, even though one partner had fewer staff and a smaller area of intervention. This became a particular issue after Phase IV when the budget started to decrease. The initial focus had been to allocate the budget first and then adopt activities and staff accordingly.

As can be seen from Table 2, in Phases I–III the project maintained loose control and flexible budgets. These budgets would usually have main budget lines such as staff, travel, supplies and services, publications, and workshops and no specification according to each single activity. It allowed more flexibility in terms of managing the budget, e.g. to direct funds to priority and urgent activities. The auditing was done by the SDC's local coordinator biannually where only total numbers and outputs would be considered. Those times are still remembered by the project staff as ‘the golden era’ of the project.

Table 2 Budget allocation and budget control

	Budget (%)
Phases	Total	IWMI	SIC	Budgeting scheme
I	3%	65%	35%	Loose budget control/flexible budget
II	16%	51%	49%	Loose budget control/flexible budget
III	29%	53%	47%	Loose budget control/flexible budget
IV	36%	49%	51%	Tight budget control/fixed budget
V	10%	32%	68%	Tight budget control/fixed budget
VI	7%	37%	63%	Tight budget control/fixed budget
Total	100%

The situation changed during Phase IV when the donor introduced more complex budgeting in order to increase the SDC's control over the project expenditures and demonstrate the accountability to the three countries.4 Several levels of budgets needed to be developed and allocated between the implementers, by countries and then the same budget lines had to be planned for each result (like 10 in Phase IV). This increased rigidity in the planning and required additional endorsement procedures when things needed a rapid change due to unexpected occurrences. The tight budget control became a hindering factor to adapt for unforeseen developments. As the donor shifted into micro-management, the planning of each activity and budgeting went into more detail in Phases V and VI. Despite the budget changes from flexible to tight control, the logframe for the project remained unchanged, indicating that there was no link between logframe and budget.

Internal conditions

Activities issues

The disconnect between the STT component and canal activities, administrative delays, disputes between partners, and inappropriate staffing caused significant interruptions in implementation. The phases, their duration, activities, staff numbers and the relationship between implementing partners have varied considerably throughout the years. The main focus of the project was canal activities, and during Phase III it expanded to the two STTs. Tarnutzer, Studer, Talipova, Sheraliyev, and Oymatov (2008, p. 35) highlighted the disconnect between canal and STT components, stating that ‘it is very important that the current sub-optimal cooperation and coordination between the two project components considerably improves’. This confirms the lack of comprehensiveness of the baseline and the lack of output and outcome vision during the start-up phase when the STT component was added.

Administrative issues

In Phase III, the approval of the new STT component was delayed by six months largely due to the protracted negotiation process over the project budgets between partners, and late submission of the Project Document. This impeded the approval process within the donor and consecutive signature of the contracts. The difference of accounting systems and organisational budget control between international and national partners triggered different strategies for implementation, from wait and see to get moving. The laissez-faire approach of the donor reduced the importance of the logframe and budget plans. Ultimately, reporting was reduced to somewhat inane paperwork by the end of that phase. However, this approach caused disturbance between the branch and the main office of the international organization, between partners and the donor.

In addition, although from Phase IV onwards the budgets were strictly planned, the donor kept requesting unplanned activities (e.g. to participate in donor and national-level meetings or to organize experience-sharing missions), which were common in earlier phases and easy to accommodate under loose budget control/flexible budget. Given the tight budget control/fixed budget, these requests required additional communication, new budgets or reallocations and administrative clearance with the donor before such trips or participation could be feasible. Obviously, these took additional administrative time, which the donor was not willing to fund.

Partnership issues

The partnership between the SIC and IWMI was also evolving. It transformed from the IWMI being a conceptual lead (in Phase I) to a so-called equal partnership (during Phases II–IV), but it was basically about allocating budgets equally despite disparities in work amount, staff and roles. On paper, partners maintained good working relations, but in practice there was a continuous struggle. Those were conceptual differences in approaches; the IWMI would focus on social mobilization and bottom-up approaches, while the SIC continuously experimented with technical solutions for an ideal situation. The discrepancy between partners was indicated by almost all external reviews as limiting factor (PA, 2004; Tarnutzer et al., 2008; Tarnutzer & Studer, 2010). The debates to come up with a joint harmonized system did not bring the intended marriage. On the contrary, the competition between the partners increased and at later stages the partners fully separated. The overall situation caused delays in the project implementation.

Staffing

There were also differences in the number of project staff between the partners – the SIC had a higher number of staff (e.g., during Phase IV, it had 165) compared with the IWMI (42). The SIC would hire more field staff (usually part-time) on short-term and output-based contracts, while during Phases II–IV the IWMI's field staff would be lesser in numbers but based on long-term and full-time contracts to deliver targeted outcomes. During Phases V–VI, the IWMI changed field staff contracts to being output based. Furthermore, partners maintained their own distinctive administrative system. They were different in dealing with human resources, hiring procedures, financial reporting and management styles. Hence, such differences created duplication, overlapping and higher transaction costs.

Since the STT component was first with the SIC in Phase III, the SIC appointed new key staff, who focused fully on STTs including field activities and were not involved in other activities of the project. At that time, the IWMI's core staff focused mainly on canal activities and the STT component was perceived as additional task only. From Phase IV, the SIC decided to drop the STT component and focused on pilot canals, while the IWMI, in addition to the STT component, still had activities to be carried out in canals. The IWMI still had no staff to focus purely on STT activities. The different outlook of the partners (technical versus participatory) had as consequence that the IWMI had to start within the STT component from the beginning and was ill-prepared for it.

Management and reporting changes

Management changes within the donor and implementing partners together with fluctuations in reporting have negatively influenced on the pace of activities. Overall SDC, as coordinator, was responsible for the achievement of the project's outputs and ensured that the project was linked to SDC's regional water strategy. The project was monitored and supervised through different channels: project management, monthly meetings, SDC consultants as back-stoppers, external reviews and reporting. The IWMI and SIC were purely responsible for implementation.

The SDC's regional management changed three times, whereas the local coordinator changed four times after Phase II and the technical back-stopper has been changed once. Changes appeared part of normal institutional shuffles and evolution, e.g., staff rotation, internal promotion. These changes had various implications toward the project. First, it implicated the change in reporting, from annual (Phase II–III), to biannual (Phase IV), and then to quarterly progress reports (Phases V-VI). These changes indicate that control over the project had been intensified. Secondly, changes involved a shift from having expatriate personnel to local experts. Thirdly, during Phase IV, staffing, plans, operational budgets and the overall project structure shifted from a regional focus to a more national outlook. The changes also funnelled the expansion of project activities to STTs during Phase III and water productivity component into a new project during Phase IV. These changes could have been caused either by internal reforms within the SDC or by changes in the SDC's water strategy. However, the SDC's water strategy for 2002–06 remained unchanged until 2011, hence the SDC's existing projects were not in line with the original strategy (Turner & Mirzaev, 2011). There have also been management changes in the implementing organizations. While IWMI had four changes, the SIC leadership remained unchanged until Phase VI. The dynamics of management changes had consequences for partner relationships, a direct influence on implementation and budget allocations.

External condition

Project implementation was affected by external factors, which may not have been fully analysed beforehand. When the project started its activities to establish participatory institutions for transboundary water cooperation at the level of STTs, it was obvious that the project did not take into account the larger contextual factors such as lack of cooperation amongst the states and sensitiveness of the border situation. This situation caused the project to work on each riparian side of the STTs separately and call any cross-boundary activities as informal, though the wording of the logframe write up did not adopt to new circumstances.

Sub-basin committees could not be created in the Kyrgyz sides of both STTs due to unrest in Kyrgyzstan in April and ethnical clashes in June 2010 (Phase IV), which caused border closure, tightening relations amongst riparians and increasing security measures, and therefore pushed back the planned arrangements. This caused an eight-month delay to meetings. Furthermore, there were local administrative issues and natural disasters that hindered the creation of the sub-basin committees for the Uzbek side of the Shakhimardansai. Here and also in the Tajik part of the Khojabakirgansai, land reforms were also ongoing during project implementation and, therefore, hindered the water user association (WUA) development process. Delays on WUA development, in turn, constrained establishment of sub-basin committees since WUAs' representatives were intended to comprise the constituent elements in sub-basin committees.

It should be mentioned that the organization of workshops in remote areas could face some challenges, which cannot be foreseen beforehand. The project experience shows that during the fall or winter times, absence of electricity and heating devices could impact the planned time, budgets and activities. The unexpected weather conditions (flood events after heavy rain) caused delay and rescheduling of the meetings with stakeholders. Icy roads and snow blocked roads for transportation and therefore triggered incomplete representation of the participants. In some cases, despite the conditions, meetings were conducted in alternative locations, which implied higher costs for the project than anticipated.

The two sections above (internal and external conditions) indicate that the formulation of the logframe had not taken into account Risk Analysis and a Risk Management Plan. The correct input of logframes would incorporate an analysis of possible critical external and internal factors/risks, which in turn would give an opportunity to assess the conditions under which the project is working (Örtengren, 2004). The lack of proper risk analysis has created a situation where project implementation is shaky and unpredictable. The shift from loose budget control/flexible budget to tight budget control/fixed budget around the starting period of the STT (Phase IV) component can be interpreted as a major misfortune for this component. It would have benefitted from more flexible budgets, provided it was a new and transboundary component.

Discussion

This study reviewed the experience of a large water project in Central Asia to understand the role of the logframe. Key findings are that project implementation was constrained by: poor completion of the Assumption & Risks column of the logframe, limited collection of baseline information, and inflexible budgets partially tied to partner dynamics. Based on these results, five overarching lessons can be identified: (1) internal risks should be clearly addressed within a logframe; (2) changes in project outlook should be matched by changes or updates in baseline studies; (3) the logframe needs correct writing of Assumptions & Risks to make it close to reality; (4) strict budgetary control should not directly contradict the logframe as an adjustable planning tool; and (5) separation of the logframe from the contract should be key to enable necessary project adaptation. These will thus be elaborated more below.

Although within logframes it is common to focus on external risks, this long-term project clearly highlighted internal risks related to relationships between partners and between partners and donors, as well as internal changes of reporting and budgeting. Despite the fact that some of these issues were identified during audits and external reviews, the risks of these internal conditions were never included in the logframe. Hence, the major finding of the paper is that internal risks due to ‘project owners’ should be clearly spelled out, and active measures should be taken to minimize internal risks – such as internal stakeholder analysis or, in this specific case, more leadership and guidance by the donor.

The findings highlighted that through the shift of the leading partner and therefore changes within the outlook of the project (from meso-level top-down cooperation to a bottom-up participatory approach), the former appropriate baseline became less meaningful and in this case possibly even redundant and a hindrance. This goes beyond the common literature, which mentioned the importance of baseline studies (Akroyd, 1999; Aune, 2000; Hummelbrunner, 2010; ICF, 2012; Uitto, 2004). Here the presented experience has shown that due to the change of lead partner and therefore change of focus, the earlier conducted baseline study was technical and failed to provide practical insights for participatory governance-building in STTs. Hence, although baselines might be conducted, they could be targeted towards key assumptions of the project implementation. Therefore, changes in outlook might require different sets of information and thus additional baseline studies. In this respect, the shift of the leading party and implementation approach had a major implication for the project, which was not recognised when the shift took place.

The third major finding is that treatment of risks was incomplete, designed to placate and ultimately deterred effective project implementation. Risks were formulated as positive assumptions, and therefore gave a predetermined impression on the feasibility of the project. While writing of the risk analysis is commonly taken for granted (Bakewell & Garbutt, 2005) and indeed in a broad sense risk is a quite common topic in water management (Khanal et al., 2015; Tan, George, & Comino, 2015), the politically sensitive nature of the project on establishing transboundary cooperation, border crossing and support of the government was not addressed appropriately. In the context of this transboundary project, several issues, e.g. political and legal situations of various governments, may not have been fully addressed. Indeed, reference to government support and permission for border crossing as assumptions rather than risks may have attenuated potential donor concerns. Nonetheless, the treatment of such challenges as assumptions rather than as risks likely constrained the project's ability to cope when such challenges materialized.

The fourth major finding is that strict budgetary control can undermine the implementation of the project. McGee (2011) has already highlighted how budgetary control mechanisms tightened to minimise corruption and increase accountability. However, the link between budgetary control mechanisms and the logframe was not spelled out, especially with regards to the notion that logframes are meant to be adjustable planning tools and, therefore, flexible as such. This is in direct contradiction to strict budgetary control. The strict budgetary control created large obstacles to implementation by restricting flexibility of action, forcing a frenzied approach to the project and increased unaccounted transaction costs of management between partners. In this respect, the call for more accountability might lead to the poor implementation of the logframe as a fixed plan and not an adaptive planning tool.

The paper's final finding is that incorporating logframes into the contract eventually constrained the ability to make changes that would have enhanced project performance. This finding is consistent with other project implementers' experience where the logframe document was integral and equated as a contract document, and not only as a flexible planning tool (Hermano et al., 2013). This shows that as much as the logframe is ‘the norm’ in contemporary developmental work, it should not be included as part of contracts. Ultimately, logframes are a planning tool and they should be treated as such.

All in all, the project experience manifested most of the known disadvantages of the logframe, but also highlighted challenges beyond what is stated in the literature on logframes, such as the link with budgetary control, shifts in project outlook and an honest approach to risks. Specifically, this multiyear transboundary project experience highlighted that the logframe needs to be very flexible, written not to be taken for granted but for the sake of being an adaptable tool for planning a project. Therefore, it should be reemphasized that the logframe would be greatly enhanced if it can be rolled out as an iterative process rather than as a rigid product. Given the high variation of development projects, we suggest that logframes should not be treated uniformly, but synchronized according to the nature of the project and its larger environment.

Conclusion

This paper reviewed the use of the logframe on a complex and long-term transboundary water cooperation project that took place in the Syr Darya basin. Analysis of project documents, implementer experiences and the logframe literature revealed issues that were crucial to the effective application of the logframe. From there, several recommendations were generated. First, taking lessons learnt from the project, it could be seen that considerable attention needs to be given to the writing of the Assumptions & Risks columns of the logframe. As the theory claims, internal and external conditions that potentially affect a project could be predicted when properly scrutinised, and from that the project should be able to be directed in the direction advantageous to achieving the intended change. In addition, baseline and feasibility studies are fundamental for creating initial knowledge that will guide the rest of the project planning. Furthermore, the project budget, a component pertinent to the logframe, could create complexities in the planned activities if not decided before hand, especially in terms of how it will be both disbursed and managed.

The above transcends to a second and more general recommendation. Logframes can be perceived as a deceptively simple and innocuous exercise in filling boxes. However, the logframe's use as a planning tool needs proper planning itself. This involves a thorough discussion with all relevant stakeholders, especially about the terminologies used and what they entail. However, the realities of the fact that logframe development does not incorporate allocations of funds for this activity, and that stating the realities on the ground would imply project funding, may disincentivise statements of real risks. This puts into question the fundamentals of the logframe.

Our case has shown that the use of linear logic in the logframe needs to be critically examined – whether it is the best option in guiding change. There are alternative methods that merit more attention, such as the process-oriented Outcome Mapping that has greater flexibility in a turbulent environment. In the end it is the question of whether one merely wants to tick a box or actually engage in a learning process that strengthens project implementation. In reality, the development world is comfortable in using logframes, despite the recognition of their limitations. As such, achieving the adoption of logframe alternatives may require fundamental changes in the landscape of project implementation and management.

Lastly, this study has contributed more knowledge to the field of international development, project management and, specifically, the use of logframes in a complex, transboundary water project in Central Asia. Even after being well designed, a project could also be subject to unforeseen forces and be beyond the control of the project management team. This includes political instability, conflict and natural disaster, among many (ICF, 2012). Thus, a project needs to be flexible, able to shift focus and priorities, and adapt to the constantly changing environment. This involves everyone, from donors to project implementers and partners. Thus, we recommend revisiting the theoretical base of project management of international development activities to develop more informed and thoughtful approaches to guide the use of the logframe.

Supplemental data

Supplemental data for this article can be accessed at http://dx.doi.org/doi:10.1080/07900627.2015.1058766

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

Research is based on data obtained from the Integrated Water Resources Management Ferghana Valley project, funded by the Swiss Development Cooperation (SDC). The authors thank the CGIAR Research Program on Water, Land and Ecosystems for financial support. Master's studies and research internship of the lead author at the International Water Management Institute were funded by the Swedish Institute.

Notes

^1. Although the SDC is treated in this document as the ‘donor’, the actual donors are the Swiss tax payers. The SDC was just the mediator.

^2. All authors are from the IWMI.

^3. The data gathered by the SIC might have been collected with the hope to obtaining additional funding for the rehabilitation and introduction of flow automation in STTs based on their earlier experience in the project canals. This would also explain, since the SDC was not willing to fund technology due to the absence of the demand from the states in STTs, why the SIC left the STT component and handed it to the IWMI.

^4. The smaller countries had complained that project spending was not equal in all three countries.