Balancing development returns and credit risks: project appraisal in a multilateral development bank

Abstract

Debates amongst civil society organizations on the governance of multilateral development banks expressed concerns regarding institutional capacities to ensure quality-at-entry in private sector projects where developmental and social impacts must balance financial viability. This paper delves upon the African Development Bank's experience in this regard by exploring its ex-ante impact assessment and project appraisal tools. Overall, the introduction of an independent development-oriented project appraisal framework has increased the development focus of portfolio decisions along the lines drawn by institutional mandates. Empirical results suggest that development and risk concerns taken into account during project appraisal are independent from each other, and that no assumption should be made about one with respect to another. Findings also suggest that considerations regarding financial additionality matter when it comes to adding value in projects, as do concerns over benefits to households.

Keywords::

• multilateral development banks
• private sector development
• African Development Bank
• project assessment
• development outcomes

JEL classification::

• F5
• O19
• 022
• H43

1. Introduction

Whether as consumers or producers, people across the world participate in markets. Making these markets more inclusive is a way to promote shared growth, lead to new and decent jobs, greater affordability of essential goods and services and reduced exposure to risk (OECD 2006). This principle has led multilateral development banks (MDBs) to include private sector development among their strategic pillars of intervention.

This private sector focus is a relatively new paradigm in the history of development practice. From 2002 to 2013, the International Financial Corporation's (IFC) commitments grew sixfold to stand at USD 18 billion (Eurodad 2014). Over the same period, the African Development Bank's (AfDB) private sector arm grew from less than USD 250 million to USD 1.6 billion. At the European level, similar growth was noted amongst bilateral development finance institutions (DFIs) focusing on private sector, which effectively became the ‘third pillar’ of European development policy after direct aid and the public sector windows of development banks (Dalberg 2010).

Considering this trend, several civil society organizations (CSOs) started investigating the impact of the private arms of MDBs and DFIs. The Bretton Woods Project (2010) reported that MDB approaches to private sector development has not always adequately focused on objectives linked to institutional mandates, and in particular on sustainable development or poverty reduction. Similarly, Eurodad (2010) investigated IFC projects in low-income countries highlighting an overall lack of focus on development effectiveness. This report echoed that of the World Bank Group's independent evaluation department, which found that < 50% of reviewed projects included evidence of poverty and distributional aspects in their objectives (IEG 2011). Along the same lines, further reports (Eurodad 2014) point to the fact that private sector focus MDBs and DFIs should have a greater focus on development impacts. In essence, worries have been put forward over the ‘overall approach, norms and procedures within the Private Sector Operations of MDBs [which] allow these institutions to put reduction of poverty, the protection of human rights, and environmental sustainability in developing countries at the core of what they do’ (Bretton Woods Project 2010).

Ensuring that operations are development focused is also a priority for governments backing MDBs. One underlying reason is that of tax-payer efficiency. This has increasingly become the case with the debt crisis in Europe and elsewhere in the developed world which has put pressure on donors and other public development agencies and stakeholders to ensure that money is not wasted and produces the highest ‘value-for-money’, or ‘development outcomes per USD spend’ ratio (The Economist 2010). This has put the onus on MDBs to show that their private sector development interventions are inclusive, have a high value-for-money ratio and can yield returns which are commensurate with the underlying project risks. As a result, MDBs started testing their proposed private sector operations (PSOs) for quality-at-entry against development outcomes and additionality benchmarks in addition to typical credit risk assessments.

In the realm of PSOs, MDBs find themselves in a situation where they often have to maximize the potential developmental returns of a project as per their mandates, while at the same time catering for credit risks to ensure project sustainability. The underlying assumption is that projects which tend to be most developmental are those which carry a lot of risk. For instance, targeting poor people which need services the most (hence achieving higher development impact) typically entails low recovery rates if the service is unsubsidised because of customers' low affordability. Such a case could weight on financial returns and thus make such private sector projects risky and unsustainable.

At the same time, MBDs must make best use of the scarce capital they hold. Considering that their capital determines the number of projects they can fund,1 the opportunity cost for funding a project which can be financed by the private sector on its own for an equal ‘amount’ of development outcomes is high. Hence, when considering a project, they should ask whether they carry any additionality, i.e. if their participation is necessary, or if they are crowding out the private sector.

Beyond the interest drawn within the CSO community, academic literature does not cover this area. The present paper is thus a first attempt in that direction as it delves upon these issues using the AfDB's project quality-at-entry framework. It seeks to understand whether the introduction of a comprehensive project appraisal framework has led to a more justifiable focus on development focus as per the mandate of the institution. It also aims to identify the trade-offs between developmental and financial returns.

Acknowledging that the AfDB has a broad set of tools to help develop the private sector, this paper focuses on one specific instrument, i.e. its portfolio of operations not covered by sovereign guarantees (PSOs), and on the specific framework put in place to measure its value-added and development outcomes for each operation, namely the ‘Additionality and Development Outcomes Assessment’ (ADOA) framework. This paper uses a unique data-set comprising 121 PSOs which were assessed by the framework between October 2008 and August 2013.

The remainder of the paper is organized as follows. Section 2 looks at the core concepts of the project appraisal framework that aims at gauging AfDB's contribution to private sector development through private sector development investments. Understanding the workings of the framework is a crucial input determining the methodology used for analysis. Section 3 presents an empirical analysis on framework components. More specifically, it looks at (1) what are the trade-offs between framework components (using data on additionality measurement, expected development outcomes and credit risk); (2) what are the most important elements for project board approval and (3) what are the underlying drivers of the elements identified in (2). Section 4 highlights some caveats and possible areas of improvement to strengthen the drive towards managing for development effectiveness and results.

2. The project appraisal framework

The ADOA framework is built on best practice put together by the MDB Evaluation Cooperation Group,2 yet still incorporates elements which are specific to the AfDB's mandate. This is in line with the idea that project and impact assessment tools are plural and personalistic (Cashmore et al. 2009, p. 92). It also captures key features of best practice in social impact assessment as described in Vanclay (2003), in particular with regards to its purpose3 and aims at ‘helping (…) understand and anticipate the possible social [and economic] consequences for human populations and communities of planned and unplanned social [and economic] change resulting from (…) programmes and projects’ (Burdge 2003, p. 84).

The framework incorporates two key variables: development outcomes and additionality. They seek to address both focus and relevance of intervention. The underlying questions they imply for each intervention are the following:

• Are the PSOs consistent with the mandate to foster social and economic development and reduce poverty in countries of operation?

• Is the participation of the bank necessary, or would the project – with the same outcomes – be funded by commercial operators alone?

Answering these two questions calls for an assessment of whether PSOs produce development outcomes (first question) and whether the bank plays an additional role, adding value to the project (second question).

2.1 Development outcomes

Development outcomes are viewed in line with the Millennium Development Goals: a project has a positive (negative) development outcome if it leads to an increase (reduction) in households' living standards, either directly or indirectly, relative to the no-project scenario (AfDB 2008). This definition sets the basis for the counterfactual used to measure the extent to which a project delivers on development.

ADOA breaks down development outcomes in seven categories: household benefits, infrastructure development, government benefits, macroeconomic resilience, gender and social effects, environmental effects and private sector development (see Box 1). As in the context of the stakeholder analysis of a qualitative cost–benefit analysis, categories referring to household benefits, government benefits and private sector development are standard areas looked at. Outcomes from infrastructure development regard both households and firms; gender and social effects and environmental effects concern mainly local and national populations (or cross-border in the case of public goods); finally, macroeconomic resilience is most relevant for the governments at large. These four categories are singled out, instead of being included in the discussion of the effects for the three stakeholders, to reflect specific concerns and operational priorities of the AfDB. For example, infrastructure development is one of the AfDB's operational pillars as defined in its 2008–2012 Medium-Term Strategy, and the institution has an interest in highlighting its area focused and the results achieved.

Box 1. AfDB's ADOA categorization for development outcomes.

  • Household benefits capture the effects on households' budget constraints. These include job creation, increase in current wages, variations in prices for commodities and services, and availability of new goods.

• Infrastructure development measures variation in access and prices for infrastructure services such as power, water supply, transportation and telecommunication.

• Government benefits records the change in tax and other revenues, and the quality of the deal from concessional agreements including those for extraction of natural resources.

• Macroeconomic resilience captures the contribution to foreign exchange reserves, sector diversification and regional integration.

• Gender and social effects measure the effects on gender balance (e.g. through female employment or credit to female borrowers) and the project's social outcomes, ranging from the contribution to human capital, public health, poverty reduction. Negative social effects such as population displacement or increased incidence of sexually transmitted diseases, and the related mitigating measures, are also addressed.

• Environmental effects records the variation in environmental quality relative to the counterfactual no-project scenario, and the impact on climate change as measured by the variation in greenhouse gas emissions.

• Private sector development captures the contribution to the development of local enterprises as measured by upward business linkages and transfer of management capacity, the alleviation of credit constraints through the development of the financial markets, and the project's demonstration effects defined as likelihood of replication without support from MDBs. Improvements to corporate governance and institutional capacity building are also captured.

To ensure full harmonization, ADOA also includes a business success category, whose rating is determined by the difference between financial rate of return and cost of capital as a proxy of commercial viability (Grettve 2007).

Development outcome ratings are a function of both the potential results and the likelihood that these will be delivered, conditional on commercial success. The last qualification is important, and it should be stressed that there is no overlapping between the evaluation of commercial viability performed by the credit department and the assessment of development outcomes. ADOA assumes commercial success as a precondition, and weighs the potential results by the remaining uncertainty which is specifically related to the delivery of the development outcomes. For example, commercial success can be guaranteed by the fact that a take-or-pay purchase agreement ensures payments against the production of electricity, but power may fail to be delivered to the final consumers because the transmission line has not been built. Uncertainty on the construction of the transmission line will negatively affect the development outcomes rating.

2.2 Additionality assessment

The ADOA framework goes beyond traditional quality-at-entry tools by considering the ‘additionality’ for a given intervention; i.e. what is it that MDBs bring into a private sector deal that commercial/private players cannot do? Additionality is considered from the perspective of all MDBs present in an intervention and makes no attempt to single out the specific role of AfDB. This acknowledges the complementary role of DFIs, which take turns in leading in different transactions and cooperate to avoid duplication of efforts and to achieve optimal risk sharing. The assessment of additionality aims to avoid that MDBs crowd out commercial investors. Their contribution is measured along three dimensions, as follows:

• Political risk mitigation, reflecting MDBs' contribution to reducing the risks for sponsor and other commercial investors due to governments' adverse actions against the project, including nationalizations and breach of contracts.

• Financial additionality, measuring MDBs' contribution to currency matching, maturity matching and resource mobilization, plus any other improvement to commercial viability excluding those captured under political risk mitigation.

• Improved development outcomes, recording MDBs' contribution to increasing the expected development outcomes, both by fostering developmental components (e.g. by developing linkages with local small and medium enterprises) and by sponsoring mitigating measures which exceed those that would be adopted if MDBs were not in the deal. These improvements are often achieved through the provision of technical assistance grants and improvements in project design.

Concepts of additionality have been developed with a relative uniformity across institutions. In the European Bank for Reconstruction and Development (EBRD), additionality assessment is based on a combination of the three core dimensions which overlap with those developed by the AfDB. Financial additionality is seen as the provision of financing at reasonable terms (i.e. appropriately priced with reference to market prices if available or otherwise not offered from private sources). The essence being that MDB financing ought not to crowd out private financiers. Improved development outcomes are defined as structuring conditionalities, in particular to deliver transition impact as per the EBRD mandate. The Inter-American Development Bank (IADB) looks at financial additionality in the same terms as the AfDB, and non-financial additionality based on the contribution to improving development outcomes. One divergent element in additionality is political risk mitigation. Although present to some extent in the IFC's understanding of additionality, it is not clearly articulated in other MDBs.

Ultimately, additionality will be a function of the environment in which the project operates. Political risk mitigation and financial additionality are likely to be higher in the case of fragile states where there are generally weak institutions and non-existent capital markets versus for instance South Africa, where institutions are stronger and capital markets more developed. Other things being equal, the level of additionality will be a function of the gap between national regulations and the norms set by the MDBs.

The involvement of MDBs in the oil and gas sector offers interesting insights in that respect. When it comes to environmental and social impacts, not all countries have adequate regulation in place. Even if that is the case, they sometimes do not have the means to enforce it along a project's life cycle. As a consequence, sets of non- or quasi-legal norms and guidelines (in the sense that they are not part of codified legislation) on environmental issues have been developed by MDBs and DFIs alike to fill in gaps (Wagner & Armstrong 2010). It is from filling these gaps that additionality derives. For instance, a study on environmental norms conducted by Alba et al. (2010) shows that while Gabon appears to have a rather complete legal arsenal, Mauritania falls behind. Hence, by comparing the gaps between MDB and national norms, additionality of a project in Mauritania would be greater than in a project in Gabon. Evidence suggests that MDB investment in 2010 into the Hasdrubal Oil and Gas Project in Tunisia and the 2009 loan to Tullow Oil in Ghana allowed for the strengthening of environmental and social norms versus national ones, thereby justifying the project's additionality (Wetherill 2010).

Collapsing political risk mitigation and financial additionality into a single category which measures the contribution to commercial viability, the ADOA concept of MDB additionality can be represented graphically (Figure 1). The five-point star represents a project which is both developmental and commercially viable without MDBs' participation. In this case, the B and C arrows represent MDBs' contribution to increasing development outcomes and commercial viability, respectively. Additionality is a function of these two improvements. On the contrary, the triangle represents a project which is not commercially viable without MDBs' intervention. The arrow depicts a common case of additionality, which occurs when development institutions increase the project commercial viability, e.g. through the provision of long maturity which are needed to ensure debt coverage.

Figure 1 MDBs' additionality in AfDB's ADOA framework.

The internal institutional arrangements for ex-ante assessment frameworks are crucial for the success of the undertaking since they will determine the incentive structure for assessors. Within the AfDB, additionality and development outcomes are looked at independently by a team of development economists.4 This institutional arrangement aims to ensure that approval of the operation is not among the criteria for performance evaluation of the responsible staff. Further to that, while a single economist is handed over a given assessment, final ratings and their justification undergo a series of peer and team reviews to ensure that they reflect a consistent analysis of the project without positive priming or personality bias.

With this in mind, at the early stage of the project cycle ADOA is one of the main pillars for decision-making. If a project's rating falls below a certain threshold, whether on additionality or development outcomes, the project is likely to be dropped. The organizational independence and the implicit influence held in ADOA (through ratings below satisfactory) provide an extra safety net, ensuring that the bank is indeed funding projects according to its mandate and strives to mainstream expected value-added into its decision-making process.

2.3 Caveats and framework limitations

Past the definitional issues, it is important to note that MDBs face common issues in measuring additionality ex-ante. In order to reflect reality, measurement must go beyond assumptions about the value they bring about, to concrete evidence and results. For instance, in measuring financial additionality using market prices to compare available financing would be a best case solution (Arvanitis & Mutambatsere 2012). Market prices are, however, not always available or easy to determine in developing countries with restrained markets. In that context, MDBs are to further develop their frameworks in order to capture additionality in practice.

A key limitation in claiming potential development outcomes in ex-ante appraisal frameworks is attribution. As currently set-up, ex-ante appraisal frameworks across MDBs involve various degrees of certainty. Immediate outcomes such as job creation can safely be attributed to an intervention and thus rewarded through the relevant rating since they can only happen as the investment takes place. For less direct effects, such as secondary job creation it is more difficult, if not impossible, to confidently attribute results to the intervention.

The underlying issue of attribution is the existence of a counterfactual. In the case of ex-ante frameworks, it is difficult to use control groups given the nature of the projects. As such, only a ‘before versus after’ comparison becomes possible. In the case of ADOA, the counterfactual is the ‘no project’ scenario. The crucial aspect of ex-ante frameworks is, however, not necessarily to measure the change in participant's well-being had the intervention not taken place. It is also about setting targets for what an acceptable social benefit should be as the intervention takes place. Monitoring such indicators would then check whether targets have been achieved. In other words, ex-ante frameworks are not necessarily just about measuring the extent of change, but also about ensuring and guaranteeing that projects comply with quality-at-entry criteria that will bring about positive social benefits and mitigate the negative ones.

3. Ex-ante frameworks in practice: evidence from AfDB's experience

In this section, we first analyse the relationship between three ratings that are considered by the AfDB's management and Board of Directors in the process of approval of PSOs. These are the development outcome ratings, the additionality rating and the credit risk rating. Credit risk is assessed independently by the credit department; it focuses on the project sponsor and on commercial viability. Second, we analyse the relationship between each type of rating and the decision to approve or reject a project. Finally, we attempt to understand the main drivers of each type of rating.

3.1 Project sample and methodology

Our analysis is based on a sample of 121 PSOs considered by the AfDB between October 2008 and August 2013 (descriptive statistics are found in Appendix 1). Of these, 107 were presented and approved by the board and 14 were dropped beforehand.

While development outcome and additionality ratings are available for all 121 operations, we have the credit rating of only 109 operations, as some projects were dropped before receiving such rating. Projects are usually dropped over the course of appraisal for a variety of reasons that include insufficiency in one of the three ratings studied or simply a withdrawal of the project by the sponsor. In the former case, it can be that a project received an ADOA rating so low that project managers decided it was not worth pursuing it, thus not submitting the project for credit rating. Ultimately, the result of this process is that all projects brought to the board already meet a minimum level in terms of their development outcomes, additionality and credit ratings. Therefore our estimates of relationships involving the credit rating must be interpreted with caution and must be taken as new research impetus as more data are availed.

We estimate , , etc. for the following models:

(1)

(2)

(3)

(4)

(5)

(6)

where development outcomes are rated on a six-point scale from ‘1 – highly unsatisfactory’ to ‘6 – excellent’, with rating from 4 to 6 representing assessments that exceed the satisfactory threshold; additionality is rated on a four-point scale from ‘1 – none’ to ‘4 – strongly positive’, with ratings 3 and 4 indicating a more than satisfactory level of additionality, and ratings 1 and 2 indicating that additionality is less than satisfactory; credit risk is assessed on a scale from 1 to 10, with 1 representing the lowest level of risk exposure and high-risk ratings equal to or exceeding 5. Household, infrastructure, government, macroresilience, environmental, gender and social, private sector are subcategories of development outcomes, rated on a four-point scale from ‘1 – poor’ to ‘4 – excellent’. Financial additionality, political risk mitigation, improved outcomes are subcategories of additionality, rated on a four-point scale from ‘1 – none’ to ‘4 – strongly positive’. All rating variables are quantitative representations of qualitative assessments (e.g. a rating of 4 instead of 2 does not mean that the project is twice as good).

Equations 12(3) simply estimate the significance of the correlation between each couple of ratings. The sign of the coefficient β₁ is expected to be: positive in Equation (1) as AfDB's advisory role during project preparation positively contributes to both additionality and development outcomes; positive in Equation (2), as operations in low-income and fragile countries, although risky from the credit standpoint, are expected to bear higher development outcomes; positive in Equation (3), as additionality is higher in risky projects that would typically not be funded by private financial institutions.

Equation (4) aims to assess the relative importance of the three ratings in determining the decision to approve or reject a project. The dependent dummy variable ‘Accepted’ takes value 1 for projects that are approved by the board and 0 for projects that are either dropped during the appraisal phases or rejected by the board. It is expected that the sign of β₁ and β₂ will be positive, as management will be more likely to approve operations with high development outcomes and additionality. On the contrary, the sign of β₃ is expected to be negative, as credit risky operations will be less likely to be approved. It is worth reminding that the additionality rating variable includes a component which measures the contribution of AfDB in terms of increased development outcomes. This contribution is also reflected in an increased development outcome rating. Consequently, the coefficient β₂ will be likely to be underestimated.

Equations (5) and (6) attempt to measure the relative importance of each subcategory in determining the overall development outcome and additionality ratings. In Equation (5), we omit the business success subcategory of development outcomes as this was only introduced in the ADOA framework after one year of implementation, and is therefore missing for all operations assessed during the first year. The sign of all coefficients in both equations is expected to be positive, as all subcategories contribute to the overall development outcomes and additionality ratings.

The authors Noreen (1988), McFadden (1984), and Davidson and MacKinnon (1984) have suggested that ordinary least square (OLS – i.e. a linear regression model) estimators significance tests statistics perform better than probit (when the dependent variable can only take two values) when the sample size is less than or around 100 observations. This is the reason why we selected a linear specification for Equations 45(6), which are estimated using OLS.

We also calculate the correlation between the overall additionality rating and the ratings of the subcategories of development outcomes. This aims to identify the types of projects in which the AfDB achieves the highest levels of additionality. Similarly, we calculate the correlation between the credit risk rating and the ratings of the subcategories of development outcomes. The purpose is to understand which types of projects are least likely to be selected because of credit risk considerations.

3.2 Trade-offs and relationships amongst additionality, development outcomes and commercial viability

The ex-ante evaluation of additionality and development outcomes has complemented the assessment of commercial risk traditionally conducted in MDBs to minimize capital losses. The idea of a trade-off between development results and credit prudence is sometimes used to justify operations in low-income countries and, even more, in fragile states. Some also argue that accepting higher levels of commercial risk is needed to reach those that are most in need, e.g. through the provision of credit to micro and small enterprises.

The requirement that AfDB operations are characterized by high levels of additionality is likely to have similar implications for the composition of the portfolio, leading to higher shares of interventions in least developed areas and sectors. These expectations (or fears, for some) are based on the assumption of a positive correlation between credit risk, development outcomes and additionality. In this section, we focus on the analysis of these relationships with the aim to shed light on the veracity of the above-mentioned assumptions and inform future portfolio decisions.

3.2.1 Development outcomes versus additionality

As discussed above, MDBs are not meant to crowd out the private sector through their operations. First, because that would entail a degree of ‘negative’ development outcomes as it would work against the private sector development objective taken into account in the ADOA framework, but also because it would imply a high opportunity cost in terms of foregone developmental impact from projects which would have otherwise been made possible only with some type of MDB contribution.

Figure 2 shows the correlation between additionality and development outcome ratings, with the size of the circles representing the number of projects with a given rating combination. Results from the estimation of Equation (1), based on 121 observations, highlight the existence of a positive and significant relationship between the two ratings, with development outcomes increasing by 0.4 points for each extra point of additionality.

Figure 2 Relationship between development outcomes and additionality. Notes: Sample size: 121 operations. Model (1): slope, 0.4; p-value, 0.003. Development outcomes are rated on a six-point scale from ‘1 – highly unsatisfactory’ to ‘6 – excellent’. Additionality is rated on a four-point scale from ‘1 – none’ to ‘4 – strongly positive’.

This result can be partly explained by the fact that one of the three sub-categories that make up additionality (namely ‘improved development outcomes’) establishes a direct link between the two rating dimensions (see Section 2.2 for details). To remove this bias, the same process was reiterated with the improved development outcome sub-category ignored. The relationship remains positive and significant (slope: 0.33, p-value: 0.002). This means that the bank's contribution to higher development focus comes also through making otherwise unviable projects (for either financial or political risk reasons) feasible, especially during the recent global financial crisis.

An example is the Rift Valley Railway funded by MDBs where the concessioning of the railway reached financial close, thanks to support which helped to avoid pitfalls experienced in similar projects (Mutambatsere et al. 2013). However, in the same project, an opportunity for further design-based additionality was missed in that MDBs should have further facilitated compliance with conditions precedent to disbursement to meet performance targets. In this regard, Mutambatsere et al. (2013) note that MDBs should go further by concurrently playing the roles of advisor, honest brokers, guarantors and financiers.

One consideration is in order on the magnitude of the estimated relationship. An estimated coefficient β₁ of 0.40 implies that to increase the development outcome rating by one point, a 2.5-point increase in the additionality rating would be needed. There are relatively few ratings in the lowest additionality categories, consistent with the evidence that assessors rate towards a general moderately positive score and the necessary reduction (or improvement) in quality to warrant a lower (or higher) rating increases towards the extremes of the scales (i.e. the distance between the categories is not constant). Consequently, a 2.5-point change in the additionality rating is unlikely, which implies that the magnitude of the relationship between additionality and development outcomes is in practice moderate.

The correlation between additionality and the subcategories of development outcomes was also analysed. Table 1 gives that the AfDB tends to be more additional in projects that benefit households, the environment, private sector development and to some extent gender and social effects. This is an interesting finding to the extent that two of these, namely household benefits and gender and social effects, are the only categories which are directly related to the population or project beneficiaries, while the other categories are one step removed. This gives greater confidence ex-ante that projects include poverty and distributional aspects in their objectives (IEG 2011; Eurodad 2014).

Table 1 Correlation between additionality and subcategories of development outcomes.

	Additionality
Additionality	–
Household benefits	0.2001
Infrastructure	0.0361
Government	0.0426
Macroresilience	0.0352
Environmental effects	0.2090
Gender and social effects	0.1511
Private sector development	0.3077

3.2.2 Development outcomes versus credit risk

The trade-off between developmental outcomes and credit risk has been a concern of many CSOs (Eurodad 2014). The analysis queries whether the AfDB tends to approve risky operations when they promise to deliver a premium in terms of outcomes. From an operational point of view, this is socially desirable. Yet, results from the estimation of Equation (2), based on 109 operations that were assigned a credit risk rating, show a positive but non-significant relationship (Figure 3). This finding supports the idea that the two decision factors are somewhat independent, and that it is not given that investing in riskier sectors will induce higher development outcomes.

Figure 3 Relationship between development outcomes and credit risk. Notes: Sample size: 109 operations. Model (2): slope, 0.048; p-value, 0.49. Development outcomes are rated on a six-point scale from ‘1 – highly unsatisfactory’ to ‘6 – excellent’. Credit risk is assessed on a scale from 1 to 10, with 1 representing the lowest level of risk exposure.

A further correlation analysis was undertaken to understand the relationship between credit ratings and the sub-categories underpinning the development outcomes rating. Table 2 suggests that credit risk is lower in projects in which development outcomes are driven by the infrastructure sub-category and by macroeconomic resilience (i.e. export-oriented). This can be explained by the fact that although risky by nature, infrastructure projects tend to have more guarantees and risk-mitigation measures attached to them (sometimes even sovereign guarantees on purchase agreements in the case of water or electricity projects) which can bring risks down. Similarly, corporate projects with an export component may benefit from the fact that sales are geographically diversified, thus providing a risk mitigation measure.

Table 2 Correlation between credit risk and subcategories of development outcomes.

	Credit risk
Credit risk	–
Household	0.1778
Infrastructure	− 0.1681
Government revenues	− 0.0409
Macroresilience	− 0.3160
Environmental effects	0.0451
Gender and social effects	0.0854
Private sector development	0.2196

3.2.2 Additionality versus credit risk

The analysis of the correlation between additionality and credit risk looks into the idea that the AfDB has greater scope for additionality when operating in riskier regions and sectors which are unlikely to attract the interest of private commercial investors. In these settings, crowding out is unlikely. However, results from the estimation of Equation (3), based on 109 operations that were assigned a credit risk rating, show a positive but non-significant relationship (Figure 4). The result holds also when the analysis is repeated for the correlation between credit risk and the financial additionality sub-category rating.

Figure 4 Relationship between additionality and credit risk. Notes: Sample size: 109 operations. Model (3): slope, 0.48; p-value, 0.29. Additionality is rated on a four-point scale from ‘1 – none’ to ‘4 – strongly positive’. Credit risk is assessed on a scale from 1 to 10, with 1 representing the lowest level of risk exposure.

Although financial additionality is seen as the main driver of the additionality rating altogether, and is thought to be closely linked to the credit environment, it should be borne in mind that (1) there are other drivers of additionality which are unrelated to the credit environment such as improved development outcomes and (2) credit risk is not only a function of the environment in which the firm operates, but also of the internal structure of the firm (strength of management, corporate practices, etc.). A substantial part of credit risk analysis is focused on the client, contrary to financial additionality, which puts emphasis on the overall environment in which the client is seeking financing. The readiness of the client to receive the funding and pay back is not necessarily linked to the exogenous environment which would justify MDB intervention. Private banks may, for instance, be unwilling to lend long term, which gives scope for MDB additionality and at the same time the business plan of the client may present risks that loans are not paid back. Both elements are unrelated on paper and thus assessed separately.

3.3 Which matters most for approval: development outcomes, additionality or credit risk?

Understanding the trade-offs between developmental and financial consideration also requires an understanding of what matters most to decision-makers (i.e. the board of directors). To verify the relative importance of development outcome, additionality and credit risk rating on the decision to approve a project, we perform a multivariate analysis. Results from the estimation of Equation (4), based on 109 observations with availability of the three ratings, show that the three dimensions have similar importance. Results presented in Table 3 show that one more point in either the development outcome or additionality ratings increases the probability of approval by approximately 8% points. One more point in the credit risk rating decreases the probability of approval by 4% points (Table 3). As credit risk is rated on a wider scale (from 1 to 10 relative to 1–4 for additionality and 1–6 for development outcomes), the relative effects have comparable magnitudes. Results are robust to the adoption of a nonlinear model (probit).

Table 3 Drivers of project approval.

Constant	0.056 (0.176)
Development outcomes	0.080^*^* (0.038)
Additionality	0.086^* (0.047)
Credit risk	− 0.042^*^* (0.016)
R^2	0.207
No. of observations	109

Notes: Standard errors are reported in parenthesis. Sample size: 109 operations; Model (4): development outcomes are rated on a six-point scale from ‘1 – highly unsatisfactory’ to ‘6 – excellent’. Additionality is rated on a four-point scale from ‘1 – none’ to ‘4 – strongly positive’. Credit risk is assessed on a scale from 1 to 10, with 1 representing the lowest level of risk exposure.

^*Significance at the 90% level.

^**Significance at the 95% level.

^***Significance at the 99% level.

The magnitude of the coefficients may be nuanced by the fact that, as mentioned above, all projects brought to the board already meet a minimum level of development outcomes, additionality and commercial viability. When projects are first considered for financing and before ratings are given, operations officers filter out those, which at first glance will not be likely to score high along any of the three dimensions.

3.4 What are the underlying drivers of category ratings?

3.4.1 Drivers of development outcomes

In order to understand the drivers of development outcomes, we estimate Equation (5). Results show that all categories significantly contribute to the overall rating, but the largest effects are due to household benefits, government, gender and social, and private sector development as shown in Table 4.

Table 4 Drivers of the development outcome rating.

Constant	− 0.218 (0.738)
Household benefits	0.368^*^*^* (0.108)
Infrastructure	0.190^*^*^* (0.061)
Government	0.327^*^*^* (0.104)
Macroresilience	0.108 (0.084)
Environmental effects	0.174^*^* (0.080)
Gender and social	0.323^*^*^* (0.112)
Private sector development	0.358^*^*^* (0.126)
R^2	0.517
No. of observations	88

Notes: Standard errors are reported in parenthesis. Sample size: 109 operations; Model (5): development outcomes are rated on a six-point scale from ‘1 – highly unsatisfactory’ to ‘6 – excellent’. Subcategories of development outcomes are rated on a four-point scale from ‘1 – poor’ to ‘4 – excellent’.

^*Significance at the 90% level.

^**Significance at the 95% level.

^***Significance at the 99% level.

The results can to some extent be explained by the preponderance in our sample of projects in financial institutions (55% of the sample), which tend to favour government revenue and private sector development. At the same time, categories such as household benefits and infrastructure contribute substantially to the overall rating despite being inversely correlated (because infrastructure projects tend to create little direct employment).

It is also interesting to note that three out of the four most prominent drivers (household benefits, government and private sector development) are categories which are present in typical stakeholder investigations of a cost–benefit analysis rather than categories which reflect specific concerns and operational priorities of the AfDB. They cut across all sectors of operation, and are less likely to be biased by the nature of the project (e.g. an infrastructure project is likely to score high on infrastructure as it is the purpose of the project, vs. in the case of a project in financial institutions).

3.4.2 Drivers of additionality

In order to understand the drivers of additionality, we estimate Equation (6).

Results presented in Table 5 clearly highlight that financial additionality is the main driver of the overall additionality score. This can be explained by the fact that the AfDB has yet to develop specific political risk mitigation tools to accompany projects. Also, results can be informed by the fact that there tends to be a link between the state of the country (whether fragile, low-income or middle-income) and political risk mitigation. Considering that the sample has few projects in fragile or post-conflict states, political risk mitigation is unlikely to be high. With regards to improved development outcomes, the results suggest that the ADOA framework could be used as an advisory function as opposed to an internal rating agency as suggested in Mutambatsere et al. (2013).

Table 5 Drivers of the additionality rating.

Constant	0.638 (0.160)
Political risk mitigation	− 0.005 (0.055)
Financial additionality	0.716^*^*^* (0.044)
Improved development outcomes	0.152^*^*^* (0.034)
R^2	0.824
No. of observations	120

Notes: Standard errors are reported in parenthesis. Sample size: 109 operations; Model (6): additionality is rated on a four-point scale from ‘1 – none’ to ‘4 – strongly positive’. Subcategories of additionality are rated on a 4-point scale from ‘1 – none’ to ‘4 – strongly positive’.

^*Significance at the 90% level.

^**Significance at the 95% level.

^***Significance at the 99% level.

4. Discussion and conclusions

This paper reviewed the AfDB's ex-ante quality-at-entry assessment framework. In the emerging ‘inclusive growth’ agenda pushed by MDBs, strong ex-ante assessments that will help the AfDB and other MDBs keep a strong development focus of portfolio decisions and strengthen the drive to manage for development effectiveness are essential. It sought to understand whether the introduction of a comprehensive project appraisal framework has led to a more justifiable focus on development focus as per the mandate of the institution. It also aimed to identify the perceived trade-offs between developmental and financial returns.

Overall, the introduction of an independent ‘ADOA’ to support the approval of all new PSOs has increased the development focus of portfolio decisions along the lines drawn by institutional mandates and according to best practice in the field (Vanclay 2003, p. 6).

The only evident association between the framework's variables is that between development outcomes and additionality. This suggests that expected contribution to higher development outcomes comes to some extent through making otherwise unviable projects feasible. Conversely, the lack of significant relationship between development outcomes and credit risk suggests that it is not given that investing in riskier sectors will induce higher development outcomes. However, these results should be read with caution to as they are based on relatively small and unbalanced sample.

The paper also finds that both additionality and development outcomes raise the probability of project approval. With regards to the former, financial additionality considerations are the underpinning drivers to the overall additionality rating. This is partly due to the fact that the AfDB's political risk mitigations tools are not very developed. In this regard, the use of MDB-specific guarantee tools should be encouraged. Considering the latter, while all categories contribute to the development outcomes overall rating, household benefits, government, gender and social effects and private sector development come out stronger. These also happen to be cross-cutting considerations which are relevant no matter the project type.

The study also showed evidence of the importance of additionality in the project selection process. This puts the onus on MDBs to strengthen their efforts in measuring additionality. Analysis also highlighted the importance of improved development outcomes as a key feature of value-addition, although it appears to be somewhat underused. This calls for a reinforcement of MDB's advisory role. With regards to development outcomes, results highlight the importance of benefits accruing directly to households as a driver of positive externalities in ex-ante decision-making, but also as a feature under which projects tend to be more additional.

Findings also highlight methodological difficulties related to such qualitative analysis. While it is important to check on framework consistency, or for any design flaws in such analyses, the findings of this paper are a caution against quick inferences that can be made by simply looking at aggregated project ratings. This is important as MDBs roll out ex-ante frameworks and report back to their shareholders.

Finally, some caveats to conclusions should be kept in mind. First, the assessment is currently limited to operations which are not covered by sovereign guarantee. A systematic approach which leads to the selection of complementary PSOs and regulatory and policy reforms is still missing and should include the wider use of policy-based operations (public sector windows of MDBs), or technical assistance (as specifically advocated in private sector focused MDBs; EBRD 2014).

Second, development outcomes have been measured at project level, and there is yet no attempt to assess impact in terms of living standards. This is due to the strategic choice to contain the amount of resources dedicated to the collection of baseline data and to the measurement of results during the execution of the project.

Third, ex-ante assessments are a dynamic exercise which must be informed by lessons learned through tracking outcomes during project implementation. For this reason, MDBs are increasingly engaged in monitoring of PSOs which includes development outcomes. Some like the EBRD have chosen to invest in monitoring systems to ensure operational follow-up. The system in place provides regular institutional reporting concerning the quality of projects, covering both quality-at-entry and quality-during-implementation. EBRD projects are required to meet minimum standards of transition impact potential before they can be approved by the Board at the entry stage, and to maintain a minimum standard of transition performance during the implementation stage (EBRD 2010). However, the data presented in this paper refer to a period in which AfDB had yet to set up an independent monitoring of development outcomes, including extracting lessons for future ratings.

Acknowledgements

The authors are grateful to Mthuli Ncube, Léonce Ndikumana, Steve Kayzzi-Mugerwa and Issa Faye for their guidance, to the participants in the internal seminar at the African Development Bank in Tunis held on August 25th 2011, the participants at the Global Development Finance conference held in Cape-Town on 5–7 November 2013, Gary Bond and four anonymous referees for useful comments, as well as to Ichiro Toda for his kind collaboration. Thanks also to José Morte Molina for assisting with data, to Aymen Dhib, Kaouther Abderrahim, and to Yaovi Gassesse Siliadin for his first-class statistical support. Marco Stampini worked on this paper while at the African Development Bank, i.e. before joining the Inter-American Development Bank. The opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the African Development Bank, the Inter-American Development Bank, their Board of Directors, or the countries they represent.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

^1. It is more specifically the risk to capital ratio which determines the limits of MDB lending. In order to maintain their AAA credit ratings (which allow them to tap the markets with low rates to then on-lend), MDBs impose themselves a very prudent use of capital. This increases the impetus for project financial soundness.

^2. The Evaluation Cooperation Group was established by the heads of evaluation in MDBs in 1996, with the goal of harmonizing evaluation methodologies and performance indicators. Members include the African Development Bank, the Asian Development Bank, the EBRD, the European Investment Bank (since 1998), the IADB, the International Monetary Fund (since 2001) and the World Bank Group. Observer members are: the Council of Europe Development Bank Ex-Post Evaluation, the International Fund for Agricultural Development Office of Evaluation, the Islamic Development Bank Operations Evaluation Office, OECD–DAC Evaluation Network and the United Nations Evaluation Group.

^3. ‘To bring about a more ecologically, socio-culturally and economically sustainable and equitable environment’ (Vanclay 2003, p. 6).

^4. Similarly to the description of institutional resistance to the introduction of social development perspectives in Caribbean Development in Harrison and McDonald (2003), the introduction of the ADOA system was perceived by some stakeholders as increased procedures delaying the pace of project approval. However, with the integration of economists into the project appraisal team (albeit with different reporting lines), the process became more acceptable leading to a gradual shift in mentalities and greater acceptance.

Appendix 1. Sample descriptive statistics

Table A1 Descriptive statistics.

	Cost of project ($m)	ADB contribution ($m)	Share of AfDB contribution	Additionality score	Political risk mitigation sub-score	Financial additionality sub-score	Improved development outcomes sub-score	Development outcome score	Credit risk
Mean	604.19	68.33	0.27	1.83	3.61	1.96	2.67	2.57	4.70
Standard error	140.39	8.21	0.03	0.06	0.06	0.07	0.08	0.07	0.13
Median	223.40	36.96	0.19	2.00	4.00	2.00	3.00	3.00	5.00
Mode	200.00	50.00	1.00	2.00	4.00	2.00	2.00	3.00	5.00
Standard deviation	1531.45	89.98	0.27	0.65	0.69	0.77	0.83	0.82	1.34
Sample variance	2,345,327	8096.27	0.07	0.43	0.48	0.59	0.69	0.68	1.81
Range	13,368	498.50	0.99	3.00	3.00	3.00	3.00	5.00	6.00
Minimum	10.00	1.50	0.01	1.00	1.00	1.00	1.00	1.00	2.00
Maximum	13,378.13	500.00	1.00	4.00	4.00	4.00	4.00	6.00	8.00
Count	119.00	120.00	119.00	121.00	120.00	120.00	120.00	121.00	109.00

Note: $m is million of dollars.

Table A2 Sector and country distribution.

	Corporate sector (%)	Financial institutions (%)	Infrastructure (%)	Total (%)
Least developed countries	14.9	14.0	14.0	43.0
Middle-income countries	5.8	5.0	3.3	14.0
Regional projects	2.5	36.4	4.1	43.0
Total	23.1	55.4	21.5	100.0

Table A3 Project approvals and refusals at board level.

	Corporate sector (%)	Financial institutions (%)	Infrastructure (%)	Total (%)
Projects accepted	15.7	53.7	19.0	88.4
Projects dropped	7.4	1.7	2.5	11.6
Total	23.1	55.4	21.5	100.0