Financial viability of school retrofit projects for clients and ESCOs

Abstract

The energy performance contracting market is potentially substantial but very little work has been undertaken to understand the characteristics of successful projects. This study uses a probabilistic analysis of four hypothetical projects in the UK schools sector under the 2014 policy regime, combined with qualitative interviews with practitioners, to explore the conditions for a viable project. It finds that the proposed approach has the potential to allow more detailed exploration of project structures and scope for creating greater understanding of likely returns and the factors affecting them. Evidence is found that the use of deterministic risk screening techniques such as simple payback results in viable opportunities being overlooked. The risk profiles for clients and contractors (energy service companies – ESCOs) are not symmetrical and they will each find different projects more attractive. The results suggest that greater consideration needs to be given to the precise risk allocation between client and contractor to ensure that likely returns are properly understood. This study demonstrates a method for exploring project characteristics that can be used to understand their impacts on the financial returns for clients and contractors.

Keywords:

• energy contracts
• energy performance
• energy service companies (ESCOs)
• financial viability
• non-domestic buildings
• outsourcing
• retrofit
• risk management

Introduction

Despite increasing attention since the 1970s, and a clear statement by the UK government that energy efficiency is fundamental to achieving the UK's carbon reduction commitments (Department of Energy and Climate Change (DECC), 2012), a variety of studies have identified an ‘energy paradox' whereby organizations (and individuals) forgo energy-efficiency projects that would generate more in savings than they would cost to implement (Hausman, 1979; United States Congress Office of Technology Assessment, 1992). While part of this can be explained as a rational choice based on the option value of delayed investment (Ansar & Sparks, 2009; Jaffe & Stavins, 1994), an important part of the explanation for this apparent paradox is conflict between the upfront investment in the energy-efficiency measure and the long-term, potentially risky, savings that are expected to accrue (DeCanio, 1998).

By transferring risk from the client to the provider of the energy efficiency measure, energy performance contracting (EnPC) can be seen as a form of securitization of energy-efficiency investments (Jackson, 2010) which removes much of the long-term uncertainty and, as a result, has been identified as an important mechanism for delivering energy-efficiency projects in both public and private sectors (DECC, 2012; European Commission, 2014).

EnPC typically involves a client engaging a contractor, known as the energy service company (ESCO), to provide an energy service based on either the operation of new energy supply equipment (e.g. a combined heat and power plant) or the implementation of an energy conservation measure (ECM) (e.g. lighting upgrade) and taking some long-term risk on the performance of the service provided (Satchwell, Goldman, Larsen, Gilligan, & Singer, 2010). The UK EnPC market is estimated to be worth £5 billion per annum by 2020 (O'Dumody, 2012), with the introduction of the Energy Savings Opportunity Scheme in July 2014 contributing to raising awareness among potential clients (HM Government, 2014).

The contracts used can be categorized as involving shared savings, where the ESCO typically pays the energy bills or guaranteed savings, where bills are paid by the client but the ESCO provides a guarantee that a particular level of savings will be achieved (European Association of Energy Services Companies, 2011).

Risk transfer to the ESCO from the client is typically greater in a shared savings contract, with the ESCO also assuming financing risk (Okay, Okay, Konukman, & Akman, 2008) but importantly the ESCO also takes a share of savings above the target level. A typical guaranteed savings structure is shown in Figure 1, although it should be noted that third-party finance from a bank as shown here is not a fundamental feature, e.g. clients might use their own funds.

Figure 1 Guaranteed savings structure

Source: Okay et al. (2008).

The choice of contract form seems to vary between countries with Ellis (2009) and Goldman et al. (2005) reporting that the shared savings model dominates in China and the US, respectively. However, in France guaranteed savings are more common (Singh, Limaye, Henderson, & Shi, 2009).

The literature relating to EnPC is comprehensive in its geographic scope, with papers published exploring the state of the market in the European Union (Bertoldi, Rezessy, & Vine, 2006; Forni, 2012; Marino, Bertoldi, & Rezessy, 2010), the Mediterranean (Patlitzianas, Doukas, & Psarras, 2006), Turkey (Okay & Akman, 2010), Slovenia (Kavcic, 2010), Denmark (Jensen, Nielsen, & Hansen, 2013), Japan (Shito, 2003), China (Xiaoli, Luping, & Wen, 2011), the US (Goldman et al., 2005; Goldman, Osborn, Hopper, & Singer, 2002; Satchwell et al., 2010), Bulgaria, Hungary and Macedonia (Rezessy, Dimitrov, Urge-Vorsatz, & Baruch, 2006). These studies are generally written for a local audience and consider the potential development of the particular national market. There are also a number of global studies covering 94 countries in total (Urge-Vorsatz, Koppel, Kiss, Goopalan Nair, & Celikyilmaz, 2007; Vine, 2005) and case studies drawn from projects undertaken by the World Bank (Sarkar & Singh, 2010). These studies typically explore the current state of the market through interviews with individuals identified as experts by the authors.

A number of key themes emerge:

• The public sector has a critical role to play in the delivery of energy efficiency projects, not just in setting the appropriate enabling environment but also as a key client, leading by example (Jensen et al., 2013; Rezessy et al., 2006). The model contract for energy performance projects recently issued by the UK government is an example of this (DECC, 2015).

• Payback periods are likely to be shorter in private sector projects than in public due to the shorter tenure of property. This significantly limits the range of ECMs that can be considered (Davies & Chan, 2001; Goldman et al., 2005; Heo, Augenbroe, & Choudhary, 2011).

• Energy saving contracts are often focused on only a small range of ECMs. Bleyl-Androschin and Schinnerl (2010) report on comprehensive energy retrofit projects in Austria, but these are noted as exceptional with payback periods exceeding 10 years.

• The key barriers are financial and administrative (Ellis, 2009), with solutions such as standardized contracts, super-ESCOs and facilitators being put forward based on positive experiences in some countries (Bleyl et al., 2013; Limaye & Limaye, 2011; Vine, Nakagami, & Murakoshi, 1999).

A much smaller body of work has been undertaken to understand the elements that are likely to make a project successful within an environment that is well suited to the development of ESCO projects.

Sorrell (2005, 2007) attempted to address this by establishing a model for the viability of outsourcing energy services based on transaction economics, where the key considerations are the project scope and depth, relative costs of transactions and energy, and the specificity of the project in terms of both assets and knowledge. Some support for Sorrell's hypothesis that transaction costs may be prohibitive for the smallest projects is found in Goldman et al. (2005) who report the view of their ESCO interviewees that there is a minimum threshold of project value to ensure profitability, and in Guertler et al. (2013) who note concerns about high administrative costs relative to savings for smaller projects. Kutlu and Polat (Global Business and Technology Association, 2011, p. 484) used a real options approach integrated with Sorrell's transaction cost economics model for EnPC to develop a framework for analysing clients’ decisions to outsource. There has been recent interest in the application of game theory to EnPC negotiations between client and ESCO (e.g. Yang et al., 2008).

While some writers, such as Xu et al. (2011), have considered key success factors for projects in particular settings, these are deeply embedded in their legislative and procurement context, making it difficult to draw clear parallels for projects in other countries (Singh et al., 2009). More recently, Nolden and Sorrell (2015) and Hannon and Bolton (2015) have explored the engagement of UK local authorities with energy service contracting, of which EnPC is a subset. This work explores different models through a case study approach.

Jackson (2010) reports on the near-universal use of simple payback as a risk-screening tool for organizations making investment decisions, with a short payback indicating a low-risk investment. Heo et al. (2011, p. 2579) make the important observation that one unintended consequence of the energy savings guarantee will be that:

ESCOs are less likely to recommend high-impact, high-cost technologies, unless the probability of energy savings can be quantified appropriately and associated risks expressed such that comparison between competing technologies is explicit.

Together with Zhao (2007), they point to the inadequacy of standard, deterministic energy modelling to achieve this.

The size of the potential market combined with a lack of literature discussing the characteristics of successful projects indicates a significant gap in the existing literature in explaining which types of project are most likely to be successful in a given context.

The aim of this study was to develop a method for exploring how the scope and form of EnPC projects might affect their financial viability from the perspectives of both clients and ESCOs. This study was based on the following hypotheses:

• The interests of clients and ESCOs are separate, often opposite (i.e. one party gains at the expense of the other) and non-symmetric (different factors will influence outcomes for different parties) (Zhao, 2007).

• Outcomes for both clients and ESCOs will be sensitive to changes in project scope (Sorrell, 2005, 2007).

• Deterministic approaches to risk management result in viable energy efficiency investments in a more diverse range of projects being rejected (Heo, Choudhary, & Augenbroe, 2012; Jackson, 2010).

Theory and calculation

Research methods

This study is focused on an understanding of the characteristics of individual projects rather than overarching market structures and a variety of approaches to exploring this issue were considered. Since the success of a project is typically defined in terms of its financial returns, this was selected as key aspect for exploration.

A probabilistic approach was selected as the most appropriate method to explore the range of possible returns due to the large number of different sources of uncertainty inherent in these relatively complex projects. The inherently non-symmetrical nature of returns in guaranteed savings EnPC projects that arises from the fact that ESCOs bear the costs of lower savings than anticipated but do not benefit from higher than expected savings, makes a probabilistic approach even more important to gain a full picture of uncertainties.

Financial returns could either be explored through an approach based on the collection of historical data or through modelling of hypothetical projects. It was considered unlikely that it would be possible to obtain data on existing projects within the timescales of the current study. While this approach could have significant benefits in terms of reliability and validity of data, it would also not allow the exploration of types of project that have not been undertaken to date. A case study method, focusing on the in-depth analysis of a previous project, was also rejected for this reason.

Economic modelling of hypothetical projects was selected as the principal research method since this would allow the interaction of different factors to be explored and the model could be extended to cover new types of project not currently undertaken. It was also decided to supplement this approach with in-depth interviews with market participants. This was necessary in order to collect data on transaction costs, which are not currently in the public domain. This method also offered the advantage of being able to explore participants’ experiences and ensure a thorough understanding of the framework agreement in practice.

Schools were selected as the subject of interest for this study since they account for 15% of public sector carbon emissions in the UK (Tian & Choudhary, 2012). In addition, the tenure of school buildings is generally long in comparison with other building types, removing a key barrier to investment (Goldman et al., 2005; Salix Finance, 2013).

The RE:FIT framework is an EnPC framework created for use by the public sector in London and schools in general (RE:FIT, 2013). As a result this study focused on the RE:FIT model of EnPC, which defined the form of contract and in particular the form of the energy savings guarantee. An outline of a typical RE:FIT contract structure is shown in Figure 2.

Figure 2 Typical RE:FIT contractual structure

Definition of subject projects

In order to collect views from a range of industry participants it was necessary to use generic projects that would avoid the need to share commercially sensitive information. This approach also allowed the scope of retrofit to be extended beyond that generally undertaken through EnPC. Four projects were developed to explore a range of project sizes and scopes. Project A was designed to reflect a typical RE:FIT project; project B is a smaller scale project; while projects C and D involve more comprehensive energy efficiency retrofit than has typically be undertaken to date.

Projects A and B were based on the Department of Education's exemplar design for a primary school with places for 420 students (Department for Education, 2014). Projects C and D were based on an operational secondary school which was the subject of an earlier study (Caruana Smith, 2009). The proposed ECMs were selected to reflect both ECMs typically delivered through the RE:FIT programme and the building fabric interventions included in projects C and D, which are generally excluded as the payback period is considered too long. Details of the projects are shown in Table 1.

Table 1 Project details

Details	Project A	Project B	Project C	Project D
Installation cost (£)	340,787	52,793	1,011,921	4,047,686
Number of buildings	10	1	1	4
Type of building	420 place primary school plus nursery (Department for Education, 2014)	As for Project A	800 place secondary school (Department for Education, 2014)	As for Project C
Energy conservation measures (ECMs)	Lighting upgrade (Chantrasrisalai & Fisher, 2007; Philips, 2010). Heating controls (TRVs)	Lighting upgrade as for Project A. Heating controls as for Project A. Boiler renewal (BRECSU, 1996)	Lighting upgrade as for Project A. Heating controls as for Project A. Boiler renewal as for Project B. Roof insulation. Wall insulation. Floor insulation. Replacement glazing	As for Project C
Simple payback (years)	6.7	8.7	21.6	As for Project C
Gas consumption	BRECSU (1996)	As for Project A	Caruana Smith (2009)	As for Project C
Electricity consumption	Philips (2010)	As for Project A	Philips (2010)	As for Project C
Energy prices	Department of Energy and Climate Change (DECC) (2013)	As for Project A	As for Project A	As for Project A
Inflation	2%	As for Project A	As for Project A	As for Project A
Discount rate	3.5%	As for Project A	As for Project A	As for Project A
Guaranteed saving	90% of mean expected saving	As for Project A	As for Project A	As for Project A

Selection of input parameters

A literature review was used to identify the factors believed to influence outcomes of EnPC projects. The most complete treatment of risks was held to be by Mills et al. (2006), drawing on the US Department of the Environment's Risk and Responsibility Matrix, and so this was used to identify potential input factors. This was combined with Sorrell's (2005) model of transaction costs to produce the nine input variables shown in Table 2. Each factor has an associated range and, for the purposes of this study, the distribution across the range was assumed to be uniform for all factors.

Table 2 Variable inputs

Variable	Description and assumptions	Range
Client transaction costs	Client staff and associated costs directly incurred in undertaking the project	±67% based on a range reported in interviews. This was extended to ±77% for projects C and D to reflect the interviewees' lack of experience in procuring these types of projects
Energy service company (ESCO) at-risk costs	ESCO internal and external costs incurred in competing for the project	Lowest and highest figures reported in interviews
ESCO investment-grade proposal costs	ESCO internal and external costs incurred in a period from selection as the preferred bidder to contract signature	Lowest and highest figures reported in interviews
ESCO installation costs	ESCO incurred costs for installation of the energy conservation measures (ECMs)	Lowest and highest figures reported by a quantity surveyor
ESCO margin	Applied to the installation costs charged to the client. Based on the results of interviews, the margin is not applied to the ESCO at-risk or investment grade proposal stage costs	Lowest and highest figures reported in interviews
Gas saving	Expected gas saving (kWh), as detailed in the second section	For projects A and B this is dominated by a ±5% variation in operating hours as reported in interviews. A total of 35% tolerance in saving for projects C and D (BRECSU, 1996)
Electricity saving	Expected electricity saving, as detailed in the second section	Based on a 5% tolerance in operating hours
Guaranteed gas saving	Set at 90% of the median value for expected gas saving to reflect a risk margin for the ESCO	Maximum set to equal the maximum gas saving with a symmetrical minimum
Guaranteed electricity saving	Set at 90% of the median value for the expected electricity saving to reflect a risk margin for the ESCO	Maximum set to equal the maximum gas saving with a symmetrical minimum

An economic model was created in Matlab to calculate separate net present value (NPV) of returns for both client and ESCO which consisted of 12 scenarios created from four different sets of project inputs and three different sets of assumptions about energy price growth. The model logic is illustrated in Figure 3; data collection methods for these inputs are explained in more detail below.

Figure 3 Structure of the economic model

Market participant data collection

To date six London boroughs have undertaken RE:FIT projects involving their schools estate. Twelve ESCOs have currently entered into framework agreements. This forms the total potential sample size. Two clients and two ESCOs were selected for interview in the current study in order to test the method.

Sorrell (2005) postulated that the level of transaction costs, the costs associated with entering into a contract (e.g. bid costs, legal costs and due diligence costs), would have a significant impact willingness of market participants to enter into EnPC arrangements. However, he reported a very low response rate to his survey that aimed to verify his hypotheses about the importance of transaction costs to energy outsourcing. As a result, it was decided to focus on face-to-face interviews for this research as these would offer the opportunity to build trust and rapport which it was hoped would facilitate the elicitation of commercially sensitive information (McDowell, 1998; Schoenberger, 1991) about the likely level of transaction costs for each of the subject projects. The small sample size precluded the undertaking of a more formal expert elicitation process.

A two-step process for transaction cost data collection was established based on the member-checking approach summarized by Cresswell and Miller (2000). Initial interviews with clients were used to define the project timetable and staffing levels and initial cost estimates were established based on this. As part of the second-round interview process, these estimates were tested with clients and ESCOs. A range was assigned to each variable based on the views of the interviewees.

This process was used to establish the values to be modelled for client transaction costs, ESCO at-risk costs, ESCO investment grade proposal costs and ESCO margin.

Both the ESCOs interviewed for this study can be described as facilities management businesses capable of delivering the majority of the proposed ECMs themselves. This is very different to a utility provider who would be likely to need to work in partnership with other organizations to deliver the proposed ECMs. The transaction cost information obtained during this study cannot therefore be assumed automatically to relate to other types of organization involved in the framework.

Calculation of energy savings

A simplified spread sheet approach to calculating energy savings was selected rather than a dynamic building energy simulation as this was considered to be appropriate to the exploratory nature of the work undertaken. This approach had the merit that it reflected the high-level estimates of savings that are undertaken by ESCOs engaging in the RE:FIT process. This simplified approach would give sufficient depth of understanding to allow the sensitivity analysis to be undertaken which would demonstrate if further detailed work on energy savings would be appropriate.

ECMs were selected based on the most commonly reported measures employed in the RE:FIT programme. For projects C and D, this was extended to include a full building fabric upgrade.

Lighting upgrade

Lighting energy savings are based on the calculations contained in Philips (2010, pp. 21, 27, 31, 35).

Heating controls

Energy savings due to the installation of thermostatic radiator valves (TRVs) are unclear. The Carbon Trust (2012) reports possible savings of 8% on domestic energy bills due to a reduction in temperature of 1°C; however, it is not clear if this can be translated to a non-domestic setting, or how users will operate TRVs in practice. As a result a conservative energy saving of 2% has been assumed.

Boiler renewal

The energy saving due to boiler renewal is calculated as the saving due to the increase in seasonal efficiency of the boiler.

Fabric upgrade

A heating energy saving of 81% has been used based on work previously undertaken by Caruana Smith (2009).

Interaction effects

Heat replacement effect

The reduction in energy emitted as heat from the lighting installation may give rise to an increased demand for heat, the heating replacement effect (HRE). The International Performance Measurement and Verification Protocol (Efficiency Valuation Organization, 2012), the accepted international standard, recommends estimating the effect but provides no details on how this should be done. Guidance for a domestic setting is provided in BNXS05 (Market Transformation Programme, 2007). Henderson (2007) reports the results of experiments designed to test the validity of this guidance. These experiments were inconclusive and demonstrated the difficulty of isolating the increase in heat due to increases in lighting efficiency.

The BNXS05 calculation assumes that 90% of heat discharged by light fittings is rejected into the living space and that 70% of lighting-hours are coincident with heating-hours giving a replacement factor of 63% (this figure is then adjusted to reflect the efficiency of the boiler supplying the replacement heat). It was considered likely that these figures would not be applicable to a school where the six-week summer holiday would be likely to result in a higher number of hours when heating and lighting would be on simultaneously than for other non-domestic building types. In addition, the use of suspended ceilings and in-ceiling luminaires in non-domestic buildings suggest that the assumption of 90% of heat being rejected to occupied space may be an overestimate.

It is assumed that the heating season is from 1 October to 30 April and that 10 hours of heating per day are required, six days per week excluding a two-week period over Christmas. Lighting is assumed to be on for 2500 hours a year, which equates to six days per week during the school term (Chartered Institute of Building Services Engineers, 2015). This means the heating is on for 72% of the hours that lighting is being used.

Chantrasrisalai and Fisher (2007) undertook experiments on a number of light fittings to calculate heat discharge for the purposes of cooling load calculations. Their work suggests that for a T8 recessed luminaire with parabolic louvres, a conditioned space factor of 69% and a convective factor of 27% are appropriate. T8 light fittings were found to use less power than their rating, so a ‘special allowance factor' of 89% is included to reflect this fact. Li (2000) demonstrates significant thermal stratification in a naturally ventilated room, consequently it is also assumed that 50% of convective heat is trapped at the ceiling level due to poor mixing.

Combining this with the number of hours of simultaneous heating and lighting calculated above suggests that 36% of the annual lighting energy load would have been providing useful heat which will need to be provided by the heating system once the lighting efficiency is improved.

Consequently, in addition to the lighting energy saving, an additional heat requirement equivalent to 36% of the lighting energy saving is calculated at the relevant seasonal boiler efficiency.

Heating energy interactions

Savings were calculated sequentially with changes affecting heat demand calculated prior to changes to the efficiency of the heat delivery mechanism to ensure that the interaction between different savings was properly accounted for.

Other cost inputs

Installation costs

A professional quantity surveyor provided installation cost data based on the quantities prepared for the subject projects. Pricing was derived from tender returns for recent school projects. The location of the projects was assumed to be in London; the base date for prices was Q2 2014.

Preliminaries were included at 8% for Project A and 6% for the other projects based on the build-up in the Building Cost Information Service (BCIS) alterations refurbishment price book (BCIS, 2014).

Consequential redecorations were included for projects C and D at a rate of £10/m². A high and low range was provided for each item.

Energy prices

Energy prices are based on the retail prices for services in the Department for Energy and Climate Change (DECC) 2013 energy price projections. These are inflated to 2014 to match the study base date (DECC, 2013). The low price scenario is notably based on a set of circumstances in which the production of unconventional shale gas in the UK and Europe results in a fall in gas prices. As these projections represent three unique trajectories, these were modelled as three separate scenarios.

Since monthly energy prices were required, it was assumed that a constant price applied throughout the year, rising annually on 1 April.

The profiles of the electricity and gas price scenarios are shown in Figures 4 and 5.

Figure 4 Electricity price projections to 2030

Source: Department of Energy and Climate Change (DECC) (2013).

Figure 5 Gas price projections to 2030

Source: Department of Energy and Climate Change (DECC) (2013).

The payback period for projects C and D exceeds the range of these forecasts. It was assumed that prices continued at the 2030 level, rising by inflation each year. Although this is a crude assumption, the effect of changes in price post-2030 will be very small due to the discount rate of 3.5% applied in the NPV calculation.

NPV calculation

The UK Treasury Social Discount rate of 3.5% was used for client NPVs as this is the mandated rate for consideration of investments by the UK government.

ESCO returns were not expected to be particularly sensitive to the choice of discount rate since the only long-term cash flows are those resulting from an energy savings guarantee shortfall. Consequently, a lower discount rate would provide a more conservative estimate of likely returns. As a result, although using mean earnings before interest and tax as a proxy for the return forgone on alternative opportunities (Lind, 2011, based on Grant Thornton, 2013) suggests a discount rate of 5%; the more conservative social discount rate of 3.5% was also used to calculate ESCO NPVs.

Energy saving guarantees

Energy savings guarantees were calculated as 90% of the expected energy saving. This does not reflect the views of the ESCOs interviewed who both expressed a view that their organization did not have a fixed buffer that was applied to each project. This is in contrast to the practices described in the US by Goldman et al. (2005) and Satchwell et al. (2010). This conflict was addressed by ensuring that the range of uncertainty attached to this input reflected this range of views.

Modelling assumptions

The following modelling assumptions were made:

• The RE:FIT call-off agreement allows the ESCO to decide whether to rectify or to pay the difference in the event of a failure to achieve the guaranteed level of savings. This model assumes the ESCO will pay the difference as this represents the worst-case scenario for the client.

• Operations and maintenance costs are typically excluded from RE:FIT contracts with clients preferring not to incur the costs of renegotiating broad facilities management contracts with another set of suppliers rather than trying to access the operations and maintenance cost savings resulting from upgrade of specific systems. Consequently, these are excluded from the current analysis.

• Energy savings are constant over the life of the guarantee, that is, degradation of savings is assumed not to occur. It is likely that the impact of any degradation would be small due to the effect of the discount rate.

• The procurement programme for each project was based on the suggested programme set out in the RE:FIT prospectus. The works programmes were based on the typical programmes described by clients.

• Inflation was assumed to be constant at 2%; a variation in the rate of inflation would have the same effect as varying the energy prices so this was not modelled separately.

Results

Results of model runs

A total of 10 000 sets of variables were generated for each project using Latin hypercube sampling (McKay, Beckman, & Conover, 1979) in Simlab (European Commission – IPSC, 2008). The NPV of the client and ESCO cash flows were calculated for each run. Each project was modelled under the three different energy price scenarios, as shown in Table 3 and Figure 6.

Table 3 NPV results of 10 000 model runs.

	ESCO mean	ESCO standard deviation	Client mean	Client standard deviation
Project A (High)	£4 600	£31 000	£952 000	£99 000
Project A (Low)	£15 000	£18 000	£640 000	£64 000
Project A (Reference)	£10 000	£25 000	£812 000	£81 000
Project B (High)	£1 700	£3 700	£28 000	£24 000
Project B (Low)	£2 900	£2 200	£15 000	£23 000
Project B (Reference)	£2 300	£3 000	£7 900	£24 000
Project C (High)	£37 000	£86 000	£84 000	£145 000
Project C (Low)	£60 000	£47 000	£380 000	£114 000
Project C (Reference)	£48 000	£67 000	£140 000	£128 000
Project D (High)	£147 000	£343 000	£537 000	£572 000
Project D (Low)	£242 000	£188 000	£1 317 000	£446 000
Project D (Reference)	£194 000	£266 000	£358 000	£504 000

Figure 6 Distribution of financial returns: (a) Project A, (b) Project B; (c) Project C and (d) Project D

Some general features of results can be observed:

• The magnitude of ESCO returns is much smaller than those of client returns since the margin on installation costs is the only source of profit for ESCOs.

• The impact of energy prices on ESCO returns is relatively small. This is a result of the modelled risk share which means the ESCO is only exposed to energy price risk when the guarantee is triggered and then only for the shortfall. This is in contrast to the client for whom the only source of income is the energy savings.

• Higher energy prices result in lower returns for the ESCO as it means a higher price is paid on any shortfall. Conversely, for the client higher energy prices result in a greater saving.

• The low price scenario has a significant impact on returns for clients when gas savings form a significant proportion of the overall energy saving.

Sensitivity analysis

This study was limited by the number of responses it was possible to obtain in the available time, as described above. Consequently, caution has be to be exercised when interpreting the results, as Stirling (2001, p. 78) notes, presenting the results of modelling as absolute value risks conveying ‘the impression of great accuracy, and distracts attention from the crucial question of the sensitivity of final results to changes in starting assumptions'. A key outcome for this study is to understand the input factors that have the greatest impact on uncertainty in the model outputs. However, the economic model is inherently non-linear due to the interaction between the actual energy saving and the guaranteed level meaning that a one-a-time (OAT) linear analysis would not provide meaningful results (Saltelli, Tarantola, Campolongo, & Ratto, 2004). Instead, a global sensitivity analysis was selected as this would allow all the input factors to be varied simultaneously and to compute the contribution of each factor to the uncertainty in the overall results.

A variance-based approach was selected since this offered the capacity to capture the full range of variation of each input variable and allowed interaction effects to be considered. The principal drawback of a variance-based approach is generally considered to be computational cost (Saltelli et al., 2008). However, sufficient computational power was available to ensure this was not problematic. Global sensitivity analysis routines from the python SALib library were used to generate input variables (Herman, 2014b). These were passed to the economic model that had been created in Matlab, transformed in accordance with the range and distribution for each parameter, as detailed above in ‘Selection of input parameters', and outputs generated for a total of 110 000 sets of inputs. Sensitivity analysis was then undertaken on these outputs in python (Herman, 2014a) based on Saltelli and Annoni's (2010) recipe for a Sobol’ analysis. The results are presented in Figure 7.

Figure 7 First-order contributions to output variance of each input factor – reference prices: (a) Project A, (b) Project B; (c) Project C and (d) Project D

The sensitivity indices do not sum to 1 because of the contribution of interaction effects between the various input factors to the overall variance in outputs. The interaction effects are greater for ESCO returns than for client returns. The uncertainty in ESCO returns is dominated by the uncertainty in the level of gas saving and the level of guaranteed gas saving for all projects. This effect is sensitive to the energy price scenario: in the high-price scenarios this is more pronounced and at the low-price scenario it is less pronounced. This is because the ESCO's returns are fixed after installation unless the guarantee is called upon. Since the ESCO does not share in savings if they are higher than predicted, this has only a negative effect. Since electricity savings are not affected by interaction between the various ECMs installed in each project, they make less contribution to overall uncertainty. The implications for the form of the guarantee are discussed below.

The contribution of each input factor to the client returns varies depending on the characteristics of each project.

In Project A, the variation in gas saving is extremely high relative to the expected saving due to the measurement and verification (M&V) approach used as the effects of variation in occupancy are also being captured. While this remains an issue in the other projects, the expected levels of saving are much greater so the additional uncertainty added is less significant.

Uncertainty in client returns for Project B is dominated by the impact of client transaction costs which ranged from 18% to 82% of total costs for the project. In interviews with clients it was clear that allocating transaction costs to individual projects was very difficult and this issue needs to be resolved in order to draw more significant conclusions from results for this project.

Projects C and D are the deepest retrofit projects with commensurately high installation costs and these have a very significant impact on client returns.

Discussion

Impact of probabilistic analysis compared with deterministic screening

Jaffe and Stavins (1994), Jackson (2010), and Heo et al. (2012) raised concerns that the use of simple payback period as a risk screening tool would result in the rejection of valuable investment opportunities. However, this work related to individual ECMs rather than the suite of measures typical under EnPC. Of the four projects studied, only project A had a short enough payback period to qualify for a well-established public sector scheme providing interest-free loans (Salix Finance, 2013). The implications of this were reported by Goldman et al. (2002) who found that lighting upgrades represented 40% of the projects undertaken in the US due to the imposition of this type of criteria. This pattern is replicated in the RE:FIT case study projects, with lighting upgrades featuring in 90% of the projects (McKinnon, 2013).

The use of a probabilistic approach in this study confirmed the results of the deterministic analysis, Project A, with the lowest payback period is the project most likely to produce a positive return for the client. However, the probabilistic approach has allowed a much more sophisticated understanding of the interaction of different factors to be understood.

Differences in outcomes for clients and ESCOs

ESCO mean returns are of a smaller magnitude than client mean returns whether positive or negative and exhibit less variation. This is due to two key reasons:

• Differences in risk profile: ESCOs do not share in the upside of savings above the guaranteed level so there is an upper limit to the returns the ESCO can achieve. Exposure to downside if the guarantee is not achieved is not limited.

• Relative importance of transaction costs: transaction costs are a more significant consideration for clients than ESCOs since they are exposed to variance in both their own and the ESCO transaction costs. This means that projects with higher ratios of transaction costs to savings are less preferable for clients, e.g. Project A is preferred to Project B and Project D is preferred to Project C.

Importance of project scale

Sorrell (2005, 2007), proposed an ideal zone for outsourcing in terms of an organization's total energy costs and the proportion of these to be outsourced. In particular, he suggested a minimum project threshold below which transaction costs would be too high as a proportion of total project costs for the projects to succeed. Figure 8 shows the four subject projects mapped onto the space Sorrell defined. The shaded area is the area that Sorrell identified as the space within which the balance of project scope and cost was likely to lead to a successful project.

Figure 8 Subject projects mapped against Sorrell's (2005) model for successful energy outsourcing

The results show that, even for Project B where transaction costs are expected to be as large as the installation costs, the project is still likely to produce a positive return for the client (unless the ‘low' energy price scenario is used). In practice, although the balance of risk and reward is affected for the smallest project, it is still a viable opportunity. This suggests that although returns are small, these projects should still be considered as viable outsourcing opportunities since the risk-adjusted return is positive.

It is possible that assertions that smaller projects are not viable (Goldman et al., 2005) may in fact be due to the nature of the framework rather than to the existence of a minimum threshold. It is notable that even the smallest of the framework contractors for whom turnover information is available has an annual turnover of £514 million (FM World, 2012). This is in contrast to the relatively small scale of many of the RE:FIT projects undertaken to date (McKinnon, 2013). Concerns that the costs associated with winning a place on a framework may place an insurmountable barrier in the way of smaller suppliers have been noted in a wide range of markets (Arnek, 2004). Smaller suppliers would be expected to have lower overheads and hence lower transaction costs.

Predicting energy consumption

While Heo et al. (2011) and Reddy, Maor and Panjapornpon (2007) have pointed out the unreliability of energy models calibrated using utility bills due to the large numbers of degrees of freedom, this has to be balanced with the time and cost associated with M&V of savings. The approach taken on two of the projects discussed in interviews was to apply both options A and C of the International Performance Measurement and Verification Protocol (IPMVP) (Efficiency Valuation Organization, 2012). Specifically, measurement of key parameters (option A) was used to verify savings associated with lighting upgrade, and option C in which energy savings are determined by ‘continuous measurement of the energy use of the whole facility' was used to verify the gas savings. In the absence of extensive sub-metering this is likely to be the most pragmatic approach.

While this simplifies the monitoring process, there are a large number of adjustments inherent in the calculation (Efficiency Valuation Organization, 2012):

Savings = (baseline energy – reporting-period energy) ± routine adjustments ± non-routine adjustments

This means that there is significant scope for dispute in the event of a savings shortfall due to the difficulty of determining whether or not a change from the baseline operating conditions was anticipated within the contract and is thus an allowable adjustment or whether this risk must remain with the ESCO. The RE:FIT framework agreement (Mayor of London, 2012) mandates the use of the IPMVP but does not prescribe the option that should be used and, consequently, there is no standardized drafting on adjustment factors affecting the operational performance of the facilities. For simpler projects with limited interaction effects, the whole building approach to M&V may expose the ESCO to excessive risk and create the potential for costly disputes.

Impact of guarantee

Although the energy savings delivered through an EnPC project are described as guaranteed, in practice the detail of what is actually covered by the guarantee may be highly variable. For example, in the case study projects the lighting energy savings are delivered due to a combination of higher efficiency lighting and improved controls such as motion and daylight sensors. If the M&V approach selected is option A as described in the preceding section, then the savings are deemed to be achieved if the lighting power draw on installation is within an agreed band. This means that the savings due to installation of improved controls are not actually guaranteed, nor is the continued performance of the lighting (although it would be normal for a manufacturer's warranty to be in place).

Conversely, the scope of the guarantee may be broader than anticipated by the ESCO since the (HRE) replacement heat would offset the gas savings delivered by other ECMs installed. In addition, the ESCO is potentially exposed to a much greater range of fluctuations in gas consumption unless these are explicitly excluded. While ESCOs do include a number of caveats to the savings guarantees provided, these are often subject to an element of tolerance. One client reported a tolerance of ±5% in occupancy of the building. When translated into a 5% change in the hours of use of the building, this results in an uncertainty which is 2.5 times the size of the anticipated saving.

Risks such as this need to be explicitly explored and considered if they are to be understood and addressed; this is not possible without a systematic probabilistic approach to risks in the input factors.

Generalizability and future work

This study has demonstrated that a range of factors affect the outcomes for the case study projects. Although the study was based on a very small sample, it is notable that with the exception of client transaction costs which dominate the uncertainties for Project B, transaction costs do not make a significant impact on the uncertainty in the model outputs. This suggests that while other ESCOs might have very different cost structures, this is unlikely to have a significant impact on model results.

Conversely, the guaranteed gas saving and actual gas saving have a significant impact in all projects except Project B. This suggests that more detailed energy modelling may result in more certain results.

All variables in this study were assumed to have a uniform distribution since most variables represented values which would have been selected by an organization and thus all values in the range could be considered to be equally likely. This assumption may not be appropriate in all cases, particularly the case of the actual gas and electric savings. It is expected that alternative distributions would be likely to result in less uncertainty in the model outputs but this should be tested in future work.

Although this study has focused on schools, the model and data collection procedures could be extended to any public sector project to test if the results remained applicable under different circumstances.

The current study has only modelled one approach to risk transfer between the client and ESCO, as discussed in the preceding section. A priority for future work will be to explore the impact of varying the allocation of risk between the client and ESCO particularly through consideration of the choice of M&V approach.

The probabilistic approach taken in this study means that these results should be applicable for other schools projects undertaken through the RE:FIT programme. The ECMs considered in projects A and B are not dependent on the building type and so are more easily generalizable. The uncertainty around heating energy savings in projects C and D is significant, but more testing is required to understand if the results are maintained in different building types.

Implications for industry, stakeholders and policy-makers

ESCOs

The sensitivity of ESCO returns to actual and guaranteed gas savings point to the importance of a more detailed understanding of the uncertainties associated with gas consumption. A more detailed building modelling would allow the ESCO to quantify better the risks to which it is exposed and thus facilitate more informed decision-making. This is particularly important due to the difficulty of measuring some of the variables that are likely to affect energy consumption, for example changes in levels of occupation. While any change is likely to be a client risk, the difficulty of establishing a baseline and then monitoring against it makes it likely some risk will remain with the ESCO. In cases where the anticipated gas saving due to the ECMs installed is small, this could have a significant impact on returns for the ESCO.

Clients

The potential for mismatch between the terms of the guarantee and the detail of the M&V approach selected is a key issue for clients since the M&V plan will determine how the savings are confirmed in practice. If the scope of the M&V plan is less comprehensive than the terms of the guarantee suggest, it is likely that clients’ returns will be lower than expected. Understanding the implications of a whole-facility approach versus that of an ECM-specific measurement is critical to ensuring that the expected scope of the guarantee is delivered in practice.

The results of the sensitivity analysis suggest that selection criteria in a competitive process should be focused on the level of guaranteed savings proposed by prospective ESCOs and not on the margin applied by the ESCO. However, installation costs have a significant bearing on client returns.

The results of Project B suggest that although even very small projects have the potential to deliver a positive return for both clients and ESCOs, the framework approach may not be the most appropriate delivery method for these projects. Client transaction costs have an overwhelming influence on the variability of client returns and streamlining these is of fundamental importance. The reduced level of cost associated with a traditional procurement approach might well offset the increased risk borne by the client in the absence of a performance guarantee for this type of project.

Policy-makers

The model form of contract for EnPC projects published in 2015 (DECC, 2015) is heavily based on the RE:FIT model and the associated guidance notes reflect some of the issues raised by this work, in particular, the fundamental importance of the M&V plan in determining the scope of the guarantee.

An important issue that merits further consideration by policy-makers is the potential mismatch between the scope of individual projects and the costs for prospective ESCOs of bidding for a place on the framework. If resulting projects are small and low risk, the qualification criteria may preclude smaller, local organizations that might be able to offer better value for money for clients.

Conclusions

The UK government's carbon emission reduction targets, combined with the introduction of the Energy Savings Opportunities Scheme, suggest a very large potential market for EnPC in the UK. However, the factors affecting client and ESCO returns are different and a more detailed assessment of potential risks from each perspective is needed to avoid disputes around the M&V of savings where a whole-building approach is being used.

The results of this study point to greater complexity in determining outcomes for both clients and ESCOs than is often considered to be the case, particularly as a result of the need to consider exactly what is covered by the performance guarantee.

Some potential opportunities may be being overlooked due to reliance on simple, deterministic risk screening methods. Of particular note is the potential for deep retrofit projects, offering drastic reductions in carbon and energy costs that are currently rejected due to long payback periods but which could have a high likelihood of positive return if the lifecycle replacement costs for the building fabric are also taken into account.

Small, single-site projects that have higher transaction costs relative to installation costs still have a high likelihood of successful outcomes and may be being unnecessarily rejected, although alternative procurement mechanisms might offer better value for money.

In sum, this study has demonstrated a greater complexity in the factors affecting returns for clients and ESCOs than is currently considered in the literature. This is an issue that needs to be explored in more detail if the potential of EnPC to help deliver energy efficiency targets is to be achieved. This study originated in an effort to address the limited literature available about individual EnPC projects and it has demonstrated a method for exploring the impact that project characteristics have on financial returns for clients and ESCOs. Future work will focus on applying the method to a wider variety of projects with a range of procurement methods.

Acknowledgements

The authors thank the interviewees for their time and insights. Particular thanks are due to Martin Sloan of Cyrill Sweett for pricing information and Greta Caruana Smith for providing access to a previous study.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the EPSRC Centre for Doctoral Training in Energy Demand (LoLo) [grant numbers EP/L01517X/1 and EP/H009612/1].