Assessing the public interest in environmental assessment: lessons from cost-benefit analysis of an energy megaproject

ABSTRACT

The potential contribution of cost-benefit analysis to environmental assessment is assessed through a case study of a proposed Canadian oil project and a comparison of results with those of the method of economic impact analysis. While the latter concludes that the project would generate substantial economic benefits, the cost-benefit analysis concludes that the project would be a net loss to society and that new oil mining is uneconomic. The case study demonstrates that economic impact analysis can help inform decision-makers of projects’ economic impacts, but the cost-benefit analysis should be used to help inform decision-makers with respect to the contribution of projects to the public interest. It is time to move beyond relying solely on economic impact analysis to measure project benefits in environmental assessment decision-making.

KEYWORDS:

• Cost-benefit analysis
• oil
• project evaluation
• environmental assessment
• economic impact analysis
• public interest

Introduction

In 2011, Canadian mining company Teck Resources submitted its proposal for a new bitumen mine (also known as oil sands or tar sands) called Frontier for approval in the Canadian province of Alberta.¹ With a capital cost of $21.5 billion (2017 Canadian) and a forecast output of 260,000 barrels per day (bpd), the project would be one of the largest in the history of Canadian oil development. The project sought approval from both the province of Alberta (pursuant to the Environmental Protection and Enhancement Act, Responsible Energy Development Act, Alberta Oil Sands Conservation Act) and the Canadian federal government (pursuant to the Canadian Environmental Assessment Act, 2012) to determine if it would be in the ‘public interest’ and whether any ‘significant adverse effects’ of projects are justified.² The Alberta and federal governments appointed a joint review panel (JRP) to oversee the environmental assessment (EA), and after a lengthy review, the JRP concluded that the project was in the public interest and its significant adverse effects were justified because the economic benefits outweighed adverse environmental effects (JRP Frontier 2019). However, before the federal government could make a final approval decision, the proponent withdrew its application citing its concern that Canada lacked a clear framework for reconciling climate change and resource development issues (Teck 2020). The project is currently shelved.

Teck’s proposal would have added to a long list of bitumen oil extraction projects in Canada that began in 1967 when oil executives and politicians proclaimed the opening of the Great Canadian Oil Sands mine (now Suncor) a ‘red letter day, not only for Canada but for all North America’ (Finch 2007, 106; Joseph 2013). Since then, Canada has become one of the largest producers of oil in the world with 4.7 million barrels of production per day in 2019 (bpd) (CER Undated).

A major issue with bitumen expansion has been how to ensure that the public interest is promoted. In the case of Frontier, Teck followed applicable EA law by submitting an application that identified the project’s positive and negative impacts as well as mitigation measures to address the negatives (Teck 2015). A key component of the application was an estimate of economic impacts using economic impact analysis (EconIA) based on input-output modelling, a technique that has become the standard method in Canadian EA for examining projects’ economic impacts.

The proponent’s EconIA concluded that the project would generate $19.1 billion in gross domestic product (GDP) during construction and $2.2 billion GDP annually during operations, over 278,000 person-years (PY) of total employment, and $72.2 billion in tax and royalty payments (all in 2017 Canadian). According to the proponent, the project would ‘yield substantial net benefits to residents …, Alberta and to Canada’ (p1-19) that would outweigh the adverse environmental effects. The JRP used these results to justify its conclusion in its report to the government that the project was in the public interest.

The concept of ‘public interest’ is a key element to most EA processes, but it is challenging to use in practice. There are many definitions across academia and jurisdictions, and therefore EA laws need to provide clarity in their definition of the concept (Hierlmeier 2008). Canada’s national energy regulator defines the public interest in terms of the balance of a project’s public benefits and negative impacts (NEB 2010); other factors include resource utilization efficiency, environmental protection, economic effects, and impacts on landowners. These factors reflect society’s values (Hegmann and Yarranton 2011; Ehrlich and Ross 2015), but historically Canadian EA has heavily weighted economic factors in public interest determinations (Hierlmeier 2008).

Importantly, EconIA has a number of limitations with respect to assessing the public interest (Joseph et al. 2020). EconIA assumes there are no constraints on inputs such as labour, thus ignoring the opportunity costs of project inputs, and is concerned only with a limited subset of economic effects. While there are economic benefits from resource development, the value of a project to society is contingent upon determining if incremental benefits are greater than incremental costs generated by the project. EconIA does not incorporate costs into the analysis and overstates benefits by estimating gross impacts that do not account for the fact that most of the labour and capital used in a project would be employed elsewhere in the economy if a project was not built. Consequently, the net benefit of a project can be expected to be significantly lower than the gross impacts estimated by an EconIA.

An alternative method for assessing projects’ economic impacts is cost-benefit analysis (CBA). CBA is a well-developed methodology for examining net benefits and has become the standard method of project evaluation used by a host of national and international agencies and is the prime method in economics for examining a project’s public interest contribution (e.g. World Bank 2010; Romijn and Renes 2013; Dobes et al. 2016; Boardman et al. 2018). But although CBA has been considered as an option by a few Canadian government authorities (e.g. ERCB 1991; Health Canada 2004; MVEIRB 2007) and has been used on occasion by intervenors in Canadian EAs (McLeod-Kilmurray and Smith 2010), CBA is rarely used in EA (World Bank 2010; Dobes et al. 2016). In Canada, standard EA texts make little to no reference to CBA (Noble 2014; Hanna 2016), CBA is not officially required in EAs, and CBA is not used by project proponents.

The purpose of this paper is to examine the potential contribution of CBA in the EA process with respect to informing the public interest through a case study application of the method to the Frontier project. We begin by presenting an overview of the CBA, and then we explore the results including a comparison with the proponent’s EconIA. We then explore key methodological issues involved in undertaking the CBA, including stakeholder and regulator responses, and the strengths, weakness, and challenges facing the use of CBA in the EA context. This study will be informative to those interested in methodology for assessing project costs and benefits and the public interest in EA, especially given the continuing debates surrounding the roles of CBA and EconIA in public policymaking around the world (e.g. de Jong and Geerlings 2003; Gillespie and Bennett 2015).

Cost-benefit analysis of Frontier bitumen mine

Overview

The basic steps in modern, good practice CBA are described in texts such as Boardman et al. (2018). CBA entails comparing the impacts of alternative scenarios in monetary terms as much as is feasible and appropriate, discounting monetized impacts, computing net present values (NPV) and internal rates of return (IRR) of alternative scenarios, and performing sensitivity analyses to address uncertainty. Project inputs are measured in terms of their opportunity costs, and project benefits and costs (i.e. outcomes) are measured in terms of peoples’ willingness to pay (WTP) for positive impacts or what people require as compensation for negative impacts. A positive NPV result suggests that a project would be a net gain for society, and a negative NPV a net loss. Certain types of impacts may not lend themselves to monetization due to technical and/or philosophical issues, and these impacts can be left unmonetized and considered alongside.

Our Frontier CBA was conducted using a spreadsheet-based model based on data from the Teck EA application (Teck 2015) and other relevant sources (see Appendix A).³ Two future scenarios were compared – one with and one without the project – and the CBA was conducted from the perspectives of Canada and the globe.

Development schedule

Frontier was planned to achieve a nominal capacity of 260,000 bpd by 2037 and reclamation was expected from 2066 to 2081. Teck stated that it would achieve 93% capacity utilization (Shewchuck 2018), but to avoid optimism bias it is a good practice to use a method called reference class forecasting by relying upon the actual performance of comparable projects (Joseph et al. 2015). We averaged the capacity factors used by three Canadian energy authorities to arrive at 88%.

Revenue from oil production

The main benefit of Frontier is revenue from oil production, estimated by multiplying annual production by price. Predicting future oil prices is notoriously difficult, and so good practice entails using a range of benchmark oil price forecasts to reflect this uncertainty. Three price scenarios were used in the CBA: the Canadian National Energy Board’s (NEB) 2018 Reference Case for the base case and the NEB’s 2018 Technology Scenario reflecting lower prices (NEB 2018b); and the International Energy Agency’s 2018 New Policies scenario reflecting higher prices (IEA 2017). Adjustments were then made to these price forecasts to account for the quality of the oil produced by Frontier and transportation costs from Frontier to market.

Private costs of oil production

Teck estimated the project’s capital costs to be $21.5 billion, but Teck’s per barrel estimate was low compared to estimates of energy authorities (AER 2018; NEB 2018a) and costs of recently constructed mines.⁴ Proponents have a propensity for underestimating costs (Flyvbjerg et al. 2003; Gunton 2003; Olaniran et al. 2015), a phenomenon observed in Canadian bitumen development (AEDA 2004; Jergeas and Ruwanpura 2010). To address this uncertainty we used reference class forecasting based on actual costs incurred in similar projects, with $90,000 per barrel of capacity for the base case, Teck’s estimate as a low case, and 25% over the base case as a high-cost case.

Teck estimated annual average operational costs (OPEX), excluding energy, to be $2.2 billion, which is consistent with other bitumen mines. We used Teck’s estimate for the base case as well as ± 25% for sensitivity analysis. To account for natural gas use in the mine we adopted a Canadian energy authority’s estimate of new bitumen mine gas consumption and a government price forecast (AER 2018; Millington 2017).

We adopted Teck's (2018) expected costs of $2.9 billion for final reclamation, but given the uncertainty in the costs of future reclamation, we also tested higher costs from the literature (Foote 2012).

Employment

Employment is usually championed by proponents as a key benefit, but net employment benefits only occur if workers would otherwise be unemployed or paid less. Teck anticipated requiring 4,500 workers during construction and 2,150 workers per year during operations. However, Teck acknowledged that due to tight labour market conditions many of these employees would otherwise be employed if the project did not proceed and therefore not all of this employment generated by the project would be incremental. Indeed, recent labour market information suggests that few incremental jobs would be created (BuildForce Canada 2018). However, it is possible that economic conditions could change and some employees – such as local Indigenous labour with high unemployment – may not have alternative employment in the absence of the mine.⁵ Consequently, we included a sensitivity analysis that assumed 10% of the employment going to otherwise unemployed workers.

Air pollution

Each phase of the project will lead to air pollution affecting human health. Government reviews indicate that any exposure to project pollutants are harmful, not just exposure above air quality standards. No study of the specific damage costs of air pollution has been done for the region, and so we employed the benefit transfer method using damage cost values from other comparable studies (Muller and Mendelsohn 2007; Jaramillo and Muller 2016) to estimate air damage costs.

Greenhouse gas emissions

Teck estimated that Frontier would emit 4.1 megatonnes of greenhouse gases (GHG) annually. Teck concluded that its emissions were minor relative to Canadian emissions, a common argument (Joseph 2019). We estimated GHG damage costs by multiplying annual emission volumes by the Canadian government social cost of carbon (SCC) estimates (ECCC 2016). Due to substantial uncertainty about the SCC (Tol 2013), we also used Canadian government high-risk SCC factors, though these may also be conservative given current climate science (van der Bergh and Botzen 2015; WMO 2019).

There are several other complications when estimating GHG damage costs. First, emissions from oil extraction are relatively small compared to the total emissions associated with extraction through refining and end-consumption (CEIP 2015). As new extraction projects could lead to incremental downstream emissions, the question arises as whether to include downstream emissions. The lifecycle GHGs from a project may be incremental or might simply offset emissions from other projects that would have been developed to meet world demand if the project did not proceed. A second issue is who has standing in GHG impact assessment. One approach is to restrict standing to only the jurisdiction of the EA (in this case, Canada), consistent with the scope of other impacts being assessed (Heyes et al. 2013; Bennett 2014), while another approach is to include damages incurred by all countries on the grounds that the adverse impacts of GHG emissions are worldwide in scope. To test these alternative scenarios, we ran our model with and without downstream emissions and with and without damage costs borne by citizens outside of Canada. Notably, restricting standing to just Canadians means that 99.5% of GHG damage costs are ignored, based upon Canada’s proportion of the global population.

Water

Water consumption and quality are key issues in the region, but according to Teck the project will comply with relevant requirements and have only negligible effects. From a CBA perspective, such impacts are relevant, but due to a lack of studies examining the monetary value of the bitumen industry’s water impacts these impacts were not monetized.

Ecosystem services

The project will alter 29,217 hectares of landscape over its lifespan, or even longer if reclamation is not successful, resulting in damages to ecosystem services (ES). We estimated ES losses using boreal forest ES values from Anielski (2012).

Additional impacts

Several additional impacts were identified but not monetized in the CBA including: (1) user cost from liquidation of natural capital (El Serafy 1989), (2) foreign investment benefits and profit leakage, (3) subsidies, and (4) social costs such as incremental costs to government. Some of these impacts were estimated in a past bitumen study (Joseph 2013) but found to be relatively low, and so we did not attempt to estimate them for Frontier.

Discount rate

Despite decades of debate, economists and philosophers have not reached agreement on appropriate discount rates. The standard approach to discounting is to apply a uniform rate to all of project’s impacts. The Canadian government’s guidance for regulatory CBA (TBCS 2007) recommends an 8% rate for all impacts over the analytical period reflecting opportunity costs of capital, though in situations involving impacts over 50 years or more a ‘social discount rate’ of 3% is appropriate (TBCS 2018). Rates of or near 3% have often been used in CBAs focused on environmental issues (e.g. ECCC 2016; Goulder and Williams 2012; Luttrell 2011; Wright 2017). Other countries’ guidance offers similar recommendations (e.g. US EPA 2010(updated 2014); UK HM Treasury 2013), though a common message across the discounting literature is that context matters. The US EPA, for example, notes that its recommendations are a starting point and that analysts may consider ‘the various factors that are most relevant to the specific policy scenario under consideration’ (2010(updated 2014), pp.6–18 to 6–19).

Alternative approaches for discounting are hyperbolic and dual discounting to address peoples’ tendency to discount the near future at a higher rate than distant futures, the scarcity and limited substitutability of natural capital, and intergenerational ethical concerns (Almansa and Martínez-Paz 2011; Hanley and Spash 1993; Gowdy 2004). There are many advocates for such approaches in both academia and government (e.g. Boardman et al. 2010; Postma et al. 2013; Kolosz and Grant-Muller 2015; Freeman and Groom 2016). The US government, for example, employed dual discounting in recent regulatory impact analyses of carbon pollution policy (Wright 2017), employing rates of 2.5% to 5% for GHG impacts and rates of 3% and 7% for other impacts.

Another important factor is the source of a project’s investment funds. Most of the discounting and CBA literature comes from the perspective of public investment and public policymaking, but private projects compete for private capital. Many industry studies typically use rates between 10% and 15% (AER 2018; Millington 2017; Millington et al. 2014), and Teck used 8% in some of its application.

In light of Frontier being a privately funded project with longer-term sustainability implications associated with its environmental impacts, we adopted a dual discounting method for our base case with 10% for private market impacts and 3% for environmental impacts. In sensitivity analysis, we ran our model with a uniform 8% rate.

Distribution of impacts

While traditional CBA focuses on the aggregation of costs and benefits to determine the net benefit to society, assessment of the distribution of impacts across stakeholder groups is good practice (Pearce et al. 2006; TBCS 2007; Boardman et al. 2018). Disproportionate distribution of impacts may be contrary to societal values and the public interest.

Our analysis assessed distribution across the private sector, the Alberta and Canadian governments, Indigenous people, and externalities accruing to Albertans, Canadians, and the rest of the world. We were able to quantify the impacts for each group except Indigenous communities; confidentiality of benefit agreements signed between the project proponent and Indigenous communities, and the rights-based nature of many of the impacts affecting Indigenous people prevented accurate quantification.

Results

We estimate the NPV of Frontier to be a net cost of $4.1 billion under base case parameters but excluding several negative impacts that could not be monetized (Table 1). From a private investor perspective, we estimate that the project generates an internal rate of return of only 8.2% in the base case, which is low but consistent with the conclusions of energy authorities who conclude that new bitumen mines are uneconomic (IEA 2017; NEB 2017, 2018b) and Teck’s own decision to shelve the project.

Table 1. Frontier benefits and costs (base case).

Impact	NPV (2017CDN)
Revenue	+$21.7 billion
CAPEX	-$10.7 billion
OPEX	-$9 billion
Reclamation	-$16 million
Employment benefits	No incremental benefit
Air pollution	-$1.3 billion plus additional unmonetized costs
GHG damages	-$4.1 billion
Impacts on water resources	Cost (unmonetized)
Impacts on ecosystem services	-$733 million plus unmonetized cultural impacts
Additional impacts	Unmonetized impacts including: user costs; foreign investment benefits and profit leakage costs; costs of subsidies; and additional social and cultural costs
NPV	-$4.1 billion

The sensitivity analyses test alternative assumptions and find the project’s NPV to range from a low of -$44 billion to a high of $5 billion (Figure 1), excluding several unmonetized impacts. The majority of scenarios conclude the project would be a net cost to society. Private financial viability as measured by IRR ranges between a low of 4.4% and a high of 11.9%, but only one scenario (high oil prices) results in an IRR over 10%. Using a uniform 8% rate our model predicts a net benefit of $3.9 billion NPV, excluding several unmonetized impacts, with a range from -$8 billion to a high of $18 billion (Figure 2).

Figure 1. NPV in base case and sensitivity analyses with 10%/3% discounting rates.

Figure 2. NPV in base case and sensitivity analyses with uniform 8% discount rate.

Key conclusions from the CBA are as follows.

1. Frontier’s economics are highly sensitive to oil price and project cost. Regardless of discount rate, if either of the NEB’s Technology oil price scenario, a 60% price differential, or greater CAPEX or OPEX or reclamation costs come to fruition, the project’s NPV worsens, in some cases substantially.

2. The project’s value to society is highly sensitive to changes in the social cost of GHG emissions. The Canadian government’s high-risk SCC values generate the highest damage costs, but the damage costs would be even higher based on higher SCC estimates in the literature. Given repeated findings that climate change is progressing more rapidly than previously thought, we can assume that the SCC will continue to rise over time, signalling over-estimation of the social value of the project under current estimates. Carbon policy may also tighten, further challenging financial viability.

3. The project’s contribution to the public interest is highly sensitive to how people value the future. Under the base case dual discounting method we forecast a $4.1 billion net loss, and under a uniform rate of 8% the project is a net gain of $3.9 billion.

4. From a private investor’s point of view, the project’s financial outlook is poor. Under the base case, our model predicts a private IRR of 8.2%, signifying a return below typical investor expectations, and under all other scenarios except one (high oil prices), IRR is lower than 10%.

5. There is little if any employment benefit. Under full employment assumptions there would be no incremental benefit to labour from the project, and even assuming that 10% of the employees would otherwise be unemployed makes little difference in the NPV estimates. This finding is in stark contrast to Teck’s EconIA findings.

The only parties expected to gain from this project are the Alberta provincial government and the Canadian federal government from royalty and tax revenue (Table 2). Major losses would be incurred by private investors by way of weak returns on investment and by citizens around the world due to carbon damages. We expect substantial adverse impacts on Indigenous communities, though these may be mitigated by benefit agreements with the project proponent; we were unable to quantify the net impact on Indigenous communities due to the confidentiality of the agreements.

Table 2. Distribution of net benefits and costs (base case).

Stakeholder	Impact	NPV (2017CDN)
Private investors	Project revenue, project costs, royalties and taxes owing	-$2.9 billion
Government of Alberta	Royalties, Alberta corporate income tax, property tax	+$3.1 billion
	User cost, subsidies, and incremental government costs	Unknown
Government of Canada	Corporate income tax	+$538 million
	Foreign investment benefits and leakage, subsidies, incremental government costs	Unknown
Externalities: Alberta	GHG damages, air pollution, and ES impacts	-$2.1 billion
	Water impacts and other social costs	Negative
Externalities: Rest of Canada	GHG damages	-$12 million
Externalities: Rest of World	GHG damages	-$2.8 billion
Indigenous Communities	Adverse impacts on use of land and natural resources, economic benefits	Unknown

Discussion

One of the primary roles of project EA is to determine if a project is in the public interest and whether adverse effects are justified.

The Frontier EconIA concludes that the mine would generate sizable economic benefits, suggesting that the project is in the public interest, while the CBA concludes that the mine would generate a net loss to society, suggesting that the project is not in the public interest. These differences in the conclusions summarized in Table 3 are due to fundamental differences in the methodologies.

Table 3. Frontier economic impacts from the alternative perspectives of CBA and EconIA.^1.

CBA

EconIA^2

Base case results of -$4.1 billion NPV due to: $21.7 billion NPV in revenues, $20 billion NPV in capital, operations, and reclamation costs, No employment benefits, $6.1 billion NPV in environmental impacts, Various additional unmonetized costs Various additional unmonetized costs Various additional unmonetized costs Base case royalties and taxes of $3.5 billion NPV. Sensitivity analysis results ranging from -$44 billion NPV to +$5.3 billion NPV

‘Net benefits’ of $19.1 billion in GDP from construction and $2.2 billion annually during operations, 278,190 person-years of employment across Canada, $72.2 billion in royalty and tax revenues. No sensitivity analysis.

1. 2017 Canadian dollars. 2. See Teck (2015, Volume 1).

EconIA measures gross impacts associated with employment, economic output, and government revenue without taking into account opportunity costs of the project inputs, or other costs such as incremental costs to government and environmental damage. The EconIA finding that the project would generate $19.1 billion in GDP over construction, for example, does not help determine the project’s contribution to the public interest because GDP signifies gross output without deducting any economic, environmental, or social costs. The EconIA results for employment and government revenue similarly ignore constraints in the economy and costs on government.

The CBA provides a more accurate accounting of the project’s contribution to the public interest because it estimates net impacts. The CBA’s estimate of a $4.1 billion NPV loss deducts economic, environmental, and social costs from project benefits, accounts for labour’s alternative opportunities, considers incremental costs on government, and accounts for government revenue that would have been generated by capital in other sectors if the mine is not built.

CBA’s focus on net impacts is consistent with the general principle in impact assessment to determine incremental change as well as numerous definitions of the public interest. Hierlmeier’s (2008) global review of the public interest concept identified seven distinct interpretations, and CBA’s NPV test is consistent with six of them: as ‘the common interest’, ‘the majority interest’, ‘the balance of interest’, the superior moral or scientific standard, ‘the economic interest’, and ‘shared values’. This broad applicability of CBA’s NPV test combined with the assessment of the distribution of costs and benefits means that it is highly relevant to helping to determine whether a project is in the public interest.

Another advantage of CBA relative to EconIA is that the CBA can be used to assess the private financial viability of projects. The Frontier CBA concludes that there would be a low private IRR. The proponent’s EconIA does not assess financial viability, yet testing financial viability is important in EA to ensure that taxpayers are not liable for clean-up or dealing with other project failure risks (Joseph 2013), also consistent with the public interest.

The results of this financial viability test provided by the CBA highlight the challenges facing oil developers in Canada and elsewhere where resource development is capital intensive. Canadian bitumen has previously been found to be among the highest cost source of oil in the world, and mining is the most expensive way to extract bitumen (Rystad Energy 2016; Jaccard et al. 2018). From the private perspective alone, the CBA confirms that across the range of scenarios, other than the high oil price scenario, it is uneconomic to build a massive bitumen megaproject, let alone a societal perspective. The CBA further suggests that the end of the era of big bitumen mining projects would seem to be over, and the recent decision of Teck to shelve the Frontier project is consistent with this conclusion.

While the case study illustrates the important contribution that CBA can make to EA by providing decision-makers with important information that is not provided by EconIA, it also illustrates some of the methodological challenges that need to be addressed in using CBA.

One challenge of CBA – that is common to all techniques in EA – is properly addressing uncertainty. Forecasting oil prices and capital costs over a project’s life is difficult, and alternative price and cost scenarios can result in wide divergence of results. Likewise, estimating the value of environmental damages is challenging given the absence of market prices and inevitable range of results when using non-market valuation techniques (e.g. Horowitz and McConnell 2002). Estimating the damages of GHG emissions is especially challenging because SCC guidelines can significantly underestimate costs by not incorporating the willingness to pay to avoid potentially catastrophic events (Weitzman 2013; van der Bergh and Botzen 2015). As such, addressing uncertainty by including sensitivity analysis to test different assumptions and scenarios can lead to confusion as to whether a project is a net cost or net benefit to society. However, it is important to stress that this challenge is not unique to CBA; project investors face the same challenge when completing financial viability analyses, and proper use of EconIA should also entail exploration of uncertain input parameters.

Another challenge concerns the definition of boundaries of project impacts and determining who should have standing. Traditional CBA defines impacts based on the definition of the project being assessed and normally restricts the assessment of project benefits and costs to a national or subnational population (Boardman et al. 2018). However, the emergence of worldwide issues such as climate change, and the global impact linkages with projects, makes project definition and determination of standing complex. Projects such as Frontier are part of a supply chain in which the output is used as an input in other downstream activities that have significant costs and benefits. As the case study shows, the exclusion or inclusion of downstream emissions has a substantial impact on results. Contrary to the general practice in CBA to limit analysis to the national or sub-national scope, the Frontier case study makes clear that when global externalities such as GHG emissions are at stake, it is important to conduct the analysis from national and global perspectives (Boardman et al. 2018).

Climate change may be the most pressing issue of our time and conducting EA from the limited perspective of a single national or sub-national jurisdiction means ignoring a major portion of a project’s impacts. In some jurisdictions, such a limited analysis may also be contrary to requirements to consider impacts outside of its borders. Based on global population distribution, a CBA of the Frontier project concerned only with Canadian damages means ignoring 99.5% of the project’s climate change impacts. Despite the recommendation by Boardman et al. to consider the global impacts of projects, the question remains of how to do so properly. All projects, whether due to the end-destinations and impacts associated with their outputs (as with Frontier) or their incorporated materials and the lifecycle impacts of these inputs, have global linkages. Future research should examine how to address the global context of major project development.

A related challenge is assessing the distribution of costs and benefits by stakeholder group to identify who gains and who loses. As the case study illustrates, the information necessary for disaggregating costs and benefits by group is not always available and qualitative listing of impacts in the absence of data makes it difficult to reach conclusions on the net impacts by stakeholder group.

The third challenge in CBA is the choice of discount rate. There is no consensus on what the appropriate discount rate is, and as the case study shows, the choice of discounting rate has substantial influence.

The case study CBA dealt with these methodological challenges through a variety of techniques. Reference cost forecasting was used to sort through a range of possible production capacities and project costs, GHG damage costs were examined both in terms of alternative project scope definitions and alternative definitions of standing, both dual and uniform discounting approaches were used, and the distribution of costs and benefits was assessed. These techniques were the basis for sensitivity analyses in which a range of results was generated.

The case study illustrated, though, that while the inclusion of sensitivity analysis to test alternative assumptions in the face of uncertainty is good practice, providing a range of results can draw criticism. In the case of the Frontier EA, Teck argued that because CBA results varied substantially the method was therefore ‘not robust … and should not inform the decision’ of the Panel (Shewchuck 2018, 9). The JRP likewise noted the sensitivity of CBA to assumptions and input parameters (JRP Frontier 2019) and expressed concern that the use of a range of discount rates and the use of a dual discount rate ‘ran the risk of subjectively skewing the results’ (JRP Frontier 2019, 891).

These critiques of addressing uncertainty by testing different assumptions to provide a range of likely results raise a larger issue in EA. It is certainly possible in CBA to generate only one estimate of net benefit based on one set of assumptions – this is what the project proponent did in their EconIA – but to do so would present an appearance of certainty in findings that is misleading. Quantitative models are by definition responsive to inputs, and a key reason for sensitivity analysis is to test the robustness and guide further inquiry in a context of great uncertainty about how the world will actually unfold. Analysts can skew results through their analytical choices, but good practice requires transparent exploration of uncertainty by showing alternative scenarios based on a range of credible assumptions (Joseph et al. 2015).

EconIA is shaped indelibly by inputs, such as project costs and inter-industry linkages. Certainly, two common problems with typical EconIA are the lack of transparency of inputs and associated assumptions, and the lack of sensitivity analysis to test the impact of alternative input values. Given the weighty role of economic benefit information in EA decisions – especially those EA frameworks that place economic benefits at the centre of the public interest equation – decision-makers and stakeholders should at least expect that inputs to EconIA are fully disclosed and that any EconIA, like CBA, is subject to sensitivity analysis. Criticizing methods because they provide a range of possible results reveals a lack of appreciation for the nature of uncertainty and the role that it plays in impact assessment. Uncertainty is inherent in predicting the future, and one should expect it to be explored in any credible analysis.

The fourth challenge in economic analysis revolves around modern economics’ conception of value. In both CBA and EconIA, monetary figures represent the value that people attach to things, and CBA by virtue of its broad scope of interest is most vulnerable to limitations of this conception. In modern economics, value is a function of preferences and ability to pay, and CBA seeks to satisfy these preferences; however they are shaped and whether they are informed or not. Basing valuation on the one dollar one vote calculus of ability to pay is contrary to the democratic principle of one person one vote. Not surprisingly, this focus on preference satisfaction is a key locus of critique (Sagoff 1988; Niemeyer and Spash 2001; Green 2009). The Frontier CBA brings these issues to the fore, as consumer demand for refined petroleum products is pitted against societal interests of global climate health and associated ecological and societal stability. One might therefore question whether policy decisions should be shaped by individuals’ WTP for oil as revealed in the marketplace and their expressions today of the worth of future (potentially catastrophic) climate change. Many projects undergoing EA may not hinge so severely on such issues, but Frontier is a clear example of the need to view CBA as a tool to inform and not a rule (Campen 1986). EconIA is not free from this critique given its use of monetary units, but its more limited scope of impact coverage means that it tends to dodge this spotlight. Economic impact information – gathered through both methods – must be understood within the limitations that modern economics operates.

Going forward, there are several key findings from this study that are relevant for EA and the assessment of the public interest. First, while EconIA is useful for informing regional planning, such as with respect to the scale of in-migrating labour and the needs for infrastructure and services, it is not suited to determining whether a project is in the public interest because it does not estimate net impacts. In contrast, CBA is capable of informing decision-makers about whether a project is in the public interest and whether the adverse impacts are justified by the benefits because CBA provides a comprehensive framework that allows for a comparison of project costs and benefits to determine the net contribution of the project to society. As the case study illustrates, though, there are a number of challenges involved in undertaking CBA in the EA context.

To address these challenges, clear guidelines are needed for both CBA and EconIA. Guidance is needed on the limitations and appropriate interpretation of results, how to discount future costs and benefits in the EA context, appropriate examination and presentation of uncertainty, how to address equity and properly assess the distribution of costs and benefits, and methods of price and cost forecasting. Further research and guidance are also needed on how best to assess the climate change implications of projects, especially those with large GHG footprints (both direct and induced). Current SCC estimates may already be out of date, and the cumulative effects of GHG emissions and worldwide impacts may require wholesale modification of assessment methodology. All of this guidance will assist practitioners and decision-makers alike and should help ensure that the best available methods are used for informing EA processes about projects’ contributions to the public interest. However, it is important to reiterate that while CBA provides important information to help decision-makers determine if a project is in the public interest, it is not and should not be the final arbiter of this determination. As the application of CBA to the Frontier mine illustrates, there are many judgements that need to be made in applying CBA that can impact results, and consequently, CBA needs to be combined with other good practices, such as public engagement and transparent rationales for decisions (Hierlmeier 2008; Joseph et al. 2015), to ensure that decisions reflect the public interest.

Conclusion

Economic analysis is an important input to EA decision-making, but like any assessment, methodology must be sound. The proponent of a major new oil project in Canada argued that its proposed project will be in the public interest, but the EconIA completed by the proponent was inherently incapable of supporting this conclusion. We conducted a CBA of the project and found that under a range of reasonable scenarios the project would be a net loss to society and a poor private investment. We found that only under a limited set of unlikely assumptions would the project be a net gain to society and a sound private investment. Through the lens of CBA, the project would not contribute positively to the public interest.

Our analysis affirms conclusions of energy authorities in Canada and internationally with respect to the financial wisdom of investing in new bitumen mines. Costs of development and oil price outlooks are not favourable, and in an era where the globe is shifting away from high-carbon energy and towards electrification of transport, these findings are even more prescient. While lower cost in situ bitumen extraction in Canada may continue for some time amidst continued global demand for oil, the stalwart of the Canadian bitumen oil industry – the bitumen mine – would seem to have had its day, as indicated by the shelving of Frontier by its proponent.

Our analysis demonstrates the important role that CBA can have in EA. In many jurisdictions, including Canada, EconIA is the main method used to inform decision-makers and stakeholders of a project’s economic benefits, but this method is not capable of estimating projects’ net economic impacts, and consequently, it is not capable of informing with respect to the public interest, which is the proper focus of impact assessment. While CBA has methodological challenges that require caution and further research, the case study shows that CBA provides a comprehensive framework for weighing costs and benefits and determining the overall net benefit to society thus helping decision-makers to determine whether the project is in the public interest. For this reason, it is time to move beyond the era of relying on EconIA to assess project benefits and to use CBA so that decision-makers are provided with the information they need to determine whether the adverse impacts of a project are justified by the benefits and whether the project is in the public interest.

Acknowledgments

We would like to acknowledge funding of the original study by the Athabasca-Chipewyan First Nation and the Oil Sands Environmental Coalition. Thanks also to Robin Gregory and anonymous peer reviewers for their comments on drafts.

Notes

1. As both of the terms ‘oil sands’ and ‘tar sands’ are heavily politicized we use the technical and more neutral term ‘bitumen’.

2. Canada has adopted a new federal EA law – the Impact Assessment Act – but the Frontier project was initiated while the previous law was in force.

3. The CBA presented here updates evidence submitted to the Frontier EA (Joseph 2018) with respect to oil prices, natural gas consumption, reclamation costs, air pollution, and GHG damages.

4. All monetary values in 2017 Canadian.

5. At the time of writing, there is a recession in Alberta induced by the downturn in oil markets. Consequently, at this moment there would be an employment benefit until the economy recovered. However, these weaker markets will also reduce the revenue from the mine so the NPV of the mine would be lower than our base case even with higher employment benefits.

6. Dollar figures in 2017 Canadian.

Appendix A.

Assessing the Public Interest in Environmental Assessment: Cost-benefit Analysis of an Energy Megaproject

This appendix summarizes key steps, sources, and assumptions of the Frontier CBA. The CBA is an update of a CBA submitted to the Frontier EA (Joseph 2018).

Cost-benefit Analysis Steps

The CBA model was built in Microsoft Excel 2013 and followed these standard steps:

1. Specify alternative scenarios for comparison.

2. Determine standing.

3. Identify impacts of project.

4. Predict and monetize impacts of project.

5. Discount impacts to obtain present values.

6. Compute net present value of all impacts.

7. Perform sensitivity analysis.

8. Assess distribution of impacts.

9. Interpret results.

Step 1: Alternative Scenarios

1. Frontier development scenario as specified by proponent.

2. No Frontier scenario.

Step 2: Standing

Standing given to all Canadian entities, except for GHG emissions in the base case. In sensitivity analysis, only climate change damages accruing to Canadians was examined.

Step 3: Identify Impacts

The following impacts were identified for examination in the CBA:

• revenue from sale of crude oil;

• private costs of oil production;

• employment;

• air pollution;

• greenhouse gas emissions;

• water;

• ecosystem services; and

• additional impacts including user costs, foreign investment and leakage, subsidies, and other social impacts.

Step 4: Predict and Monetize Impacts

Development Schedule

According to Teck (Teck Resources Limited) (2015):

• construction will entail two phases: phase 1 will begin in 2019 and last through 2025, and phase 2 will begin in 2030 and last through 2036. Operations are expected to commence in 2026 and last through 2066 (for a 41 year life);

• nominal capacity of 260,000 bpd (initial 85,000 bpd in 2026, 170,000 bpd of bitumen by 2027 as phase 1 operations ramp up, and then 260,000 bpd of bitumen by 2037 when phase 2 operations commence);

• reclamation currently expected from 2066 to 2081.

According to Shewchuck (2018), Frontier will achieve 93% capacity utilization, but industry capacity utilization average of 88% was used derived from NEB (2018b), AER (2018), and CERI (Millington 2017).

Revenue from Oil Production

Revenue is oil production volume (see Development Schedule) times oil price by year. Frontier crude oil trades at a discount from benchmark crudes due to quality differences and transportation costs. Benchmark oil prices (base case and high and low prices in sensitivity analysis; Figure A1) taken from NEB (2018a) and IEA (2017). Frontier crude oil trades at a discount from benchmark crudes due to quality differences and transportation costs. The base case method for accounting for these quality and transportation differences to estimate minehead prices for the Frontier used the method used by CERI (Millington 2017) and an alternative method in ARRP (2007) for a sensitivity analysis.

Figure A1. Price scenarios used in the CBA.

Sources: IEA (2017), NEB (2018a), Teck (2017).

Private Costs of Oil Production

There are three main types of private costs of the project: capital costs (CAPEX), operational costs (OPEX), and reclamation costs:

• CAPEX base case assumed to be $90,000 per barrel of capacity based on NEB (2018b), AER (2018), Tait (2013), Healing (2017), Anonymous (2018), and Morgan (2018)⁶;

• OPEX (excluding gas consumption) based on Teck (2015, Volume 1);

• OPEX energy based on gas consumption rate in Millington (2017) and prices in AER (2018).

• reclamation costs (base case) based on Teck (2018) and sensitivity analysis based on Foote (2010).

Employment

Teck estimates of labour requirements in Teck ((2015), but labour market information at time of analysis found in BuildForce Canada (2018a; 2018b), Alberta (Undated-a), and PetroLM/Enform (2017; 2018). Natural rate of unemployment assumed to be roughly 6%. Assumptions for social opportunity cost of labour in Townley (1998) and Shaffer (2010).

Air Pollution

Air pollutants and estimated emissions are provided by Teck (2015). Teck provides emission estimates for a 277,000 bpd project, and so estimates were scaled down proportionately to the 260,000 bpd nominal capacity. Using benefit transfer, damage cost factors for the base case are in Jaramillo and Muller (2016); alternative values for sensitivity analysis are found in Muller and Mendelsohn (2007).

Greenhouse Gas Emissions

Estimates of project GHG emissions and emission intensity are provided in Teck(2015). We assumed annual reclamation emissions would match annual construction emissions. SCC values from ECCC (2016); ‘updated central’ SCC values for the base case, and ‘updated 95^th percentile’ values for the high-risk sensitivity analysis. Additional sensitivity analysis examined (1) downstream emissions using GHG emission factors from IHS CERA (2010) and based on a volumetric adjustment from STC (2011) to convert Frontier crude oil volume to the volume refined and volume turned into combustible fuels, and (2) damages only to Canadians, which compose 0.5% of the global population. For the distributional analysis, Alberta carbon tax rules are provided in Alberta (2017; Undated-b; Undated c).

Ecosystem Services

Estimates of land cover altered by the project in Teck (2015). Values of boreal forest ecosystems are provided in Anielski (2012).

Step 5: Discounting

The base case employed a dual discounting procedure where market impacts (i.e. revenue and private costs) were discounted at 10% consistent with AER (2018) and Millington (2017), and non-market (environmental) impacts were discounted at 3% consistent with Goulder and Williams (2012), Luttrell (2011), Wright (2017), ECCC (2016), and TBCS (2018). Sensitivity analysis used a uniform 8% rate consistent with Teck (2015) and TBCS (2007).

Step 6: Net Present Value

The net present value of the project is the sum of the present values of all project benefits and costs:

$NPV=\sum _{t=0}^n\frac{B_t-C_t}{{\left(1+r\right)}^t}$

where B is the sum of benefits, C is the sum of costs, r is the discount rate, and t is the duration of the project.

Step 7: Sensitivity Analysis

Within the Frontier development scenario (step 1), the following alternative scenarios were tested:

• higher and lower oil prices;

• alternative bitumen valuation;

• higher and lower CAPEX, OPEX, and reclamation costs;

• 10% of labour otherwise unemployed;

• lower air pollution damage costs;

• higher SCC, Canada-only climate change damages, and downstream GHG emissions; and

• uniform 8% discount rate.

See step 4 for further information on these alternative scenarios.

Step 8: Distribution

The impacts were disaggregated to determine how key stakeholder groups were affected:

• private investors: net of revenue, project costs, royalties, and taxes owing;

• provincial Government of Alberta: net revenue is the sum of royalties, provincial corporate income tax, and property tax (other tax revenues (personal income taxes) were assumed to be offset by incremental government costs to provide services for employees, and user cost was not estimated);

• Canadian government: net revenue is the sum of federal corporate income tax (other taxes (personal income tax, sales tax) are assumed to be offset by incremental government costs to provide services for employees, and foreign investment benefits and profit leakage costs were not estimated);

• citizens of Alberta: net environmental impacts occurring in Alberta (including share of GHG damage costs);

• citizens from the rest of Canada: GHG damage costs accruing to Canada (excluding Alberta) and

• citizens from other countries: remaining GHG damage costs accruing to rest of the world.

References

AER (Alberta Energy Regulator) (2018). Alberta’s Energy Reserves and Supply/Demand Outlook. Calgary, AB, Alberta Energy Regulator.

Alberta (2017). Carbon Competitiveness Incentive Fact Sheet. Retrieved March 2, 2018, https://www.alberta.ca/assets/documents/cci-fact-sheet.pdf.

Alberta (Undated-a). Unemployment Rate. Retrieved February 26, 2018, http://economicdashboard.alberta.ca/Unemployment.

Alberta (Undated-b). Carbon Competitiveness Incentive. Retrieved March 2, 2018, https://www.alberta.ca/carbon-competitiveness-incentives.aspx.

Alberta (Undated-c). Carbon levy and rebates. Retrieved March 2, 2018, https://www.alberta.ca/climate-carbon-pricing.aspx.

Anielski, M. (2012). Evaluation of Natural Capital and Ecological Goods and Services at Risk Associated with the Proposed Enbridge Northern Gateway Pipeline. 19pp.

Anonymous (2018) Fort Hills Mine up and running as partner Total mulls selling another stake. Oil Sands Magazine, http://www.oilsandsmagazine.com/news/2018/1/29/fort-hills-oilsands-mine-begins-operations.

ARRP (Alberta Royalty Review Panel) (2007). Our Fair Share: Report of the Alberta Royalty Review Panel. 105pp.

BuildForce Canada (2018a). National Summary Highlights 2018-2027. Ottawa. 12pp.

BuildForce Canada (2018b). Construction and Maintenance Looking Forward Highlights 2018-2027, Alberta. Ottawa. 13pp.

ECCC (Environment and Climate Change Canada) (2016). Technical Update to Environment and Climate Change Canada’s Social Cost of Greenhouse Gas Estimates. 27pp.

Foote, L. (2010). The State of Oil Sands Wetland Reclamation. Reclamation and Restoration of Boreal Peatland and Forest Ecosystems: Towards a Sustainable Future. Edmonton, AB.

Goulder, L. H. and R. C. Williams III (2012). The Choice of Discount Rate for Climate Change Policy Evaluation. Washington, DC, Resources for the Future. 22pp.

Healing, D. (2017) CNRL mulls over bitumen-only expansion at Horizon oilsands mine. Global News, https://globalnews.ca/news/3855510/cnrl-mulls-over-bitumen-only-expansion-at-horizon-oilsands-mine/.

IEA (2017). World Energy Outlook 2017. International Energy Agency.

IHS CERA (2010). Oil Sands, Greenhouse Gases, and US Oil Supply: Getting the Numbers Right. Special Report. 41pp.

Jaramillo, P. and N. Z. Muller (2016). Air pollution emissions and damages from energy production in the U.S.: 2002–2011. Energy Policy 90: 202-211.

Joseph, C. (2018). Teck Frontier Mine: Review of Economic Benefits and Cost-Benefit Analysis. Submission to the Joint Review Panel of the Frontier Oil Sands Mining Project Swift Creek Consulting. 66pp.

Luttrell, M. J. (2011). The Case for Differential Discounting: How a Small Rate Change Could Help Agencies Save More Lives and Make More Sense. William & Mary Policy Review 3(80): 80-128.

Millington, D. (2017). Canadian Oil Sands Supply Costs and Development Projects (2016-2036). Calgary, AB, Canadian Energy Research Institute. 52pp.

Morgan, G. (2018). Fort Hills, last of the major oilsands projects, starts amid pipeline and rail constraints. Financial Post. January 29, 2018.

Muller, N. Z. and R. Mendelsohn (2007). Measuring the damages of air pollution in the United States. Journal of Environmental Economics and Management 54(1): 1-14.

NEB (National Energy Board) (2018a). Canada’s Energy Future 2018: An Energy Market Assessment. National Energy Board. 108pp.

NEB (National Energy Board) (2018b). Canada’s Energy Future 2017 Supplement: Oil Sands. 20pp.

PetroLM/Enform (2017). Labour Market Outlook 2017 to 2021: Canada’s Oil and Gas Industry. Calgary, AB. 62pp.

PetroLM/Enform (2018). Canada’s Oil and Gas Employment and Labour Market Data Q4 2017. Calgary, AB. 2pp.

Shaffer, M. (2010). Multiple Account Benefit-Cost Analysis: A Practical Guide for the Systematic Evaluation of Project and Policy Alternatives. Toronto, University of Toronto Press. 152pp.

Shewchuck, P. (2018). RE: Review of OSEC Cost-Benefit Analysis. Edmonton, AB, Canada, Nichols Applied Management Inc. 9pp.

STC (Statistics Canada) (2011). The Supply and Disposition of Refined Petroleum Products in Canada. Catalogue no. 45-004-X. 88pp.

Tait, C. (2013). Imperial bumps up cost of Kearl oil sands project. Globe and Mail. February 1, 2013.

TBCS (Treasury Board of Canada Secretariat) (2007). Canadian Cost-Benefit Analysis Guide: Regulatory Proposals. 51pp.

TBCS (Treasury Board of Canada Secretariat) (2018) Policy on Cost-Benefit Analysis. https://www.canada.ca/en/treasury-board-secretariat/services/federal-regulatory-management/guidelines-tools/policy-cost-benefit-analysis.html.

Teck (2015). Frontier Oil Sands Mine Project – Project Update EPEA Application No. 001-247548, Water Act File No. 303079, CEAA Reference No. 65505 and ERCB Application No. 1709793.

Teck (2017). Teck Resources Limited Responses to Joint Review Panel Information Request Package 5 – Socio-economics. 183pp.

Teck (2018). Review of Economic Viability Related Concerns from August 2018 Participant Submissions. Edmonton, AB, Canada, Teck Resources Limited. 5pp.

Townley, P. G. C. (1998). Principles of Cost-Benefit Analysis in a Canadian Context. Scarborough, ON, Prentice Hall. 333pp.

Wright, D. (2017). Carbonated Fodder: The Social Cost of Carbon in Canadian and U.S. Regulatory Decision-Making. Georgetown International Environmental Law Review 29(3): 513-554.