Report of an Expert Panel to Review the Socio-Economic Models and Related Components Supporting the Development of Canada-Wide Standards (CWS) for Particulate Matter (PM) and Ozone To the Royal Society of Canada

Abstract

Canada-Wide Standards (CWS) for particulate matter (PM) and ozone may be the most ambitious environmental standards ever proposed in Canada. They have attracted considerable attention and debate. This report addresses the validity of the socioeconomic modeling aspects of the Canada-Wide Standards development process.

Socioeconomic considerations are addressed in one of eight principles underlying the development and attainment of CWS, according to a CWS subagreement signed by the Canadian Council of Ministers of the Environment (CCME). Principle 3.1.7 states that

measures to attain agreed-upon Canada-Wide Environmental Standards will be determined in a sustainable development context, recognizing environmental and socio-economic considerations.

This work has been previously published by the Royal Society of Canada, Ottawa (Ontario), in June 2001, appearing as “Review of Socio-Economic Models and Related Components Supporting the Development of Canadian Standards for Particulate Matter and Ozone,” ISBN 0-920064-73-6.

A CCME Framework for Socio-economic Analyses in Setting Environmental Standards (CCME, 1998) describes procedures and information requirements for socioeconomic assessments of potential or proposed environmental standards. This framework states that while it may not be possible or necessary to carry out all of the analytical steps because of time, data, or resource constraints, a partial assessment can produce information that is useful for policy deliberations. The framework also notes that socioeconomic findings are not intended to be prescriptive concerning decisions about environmental standards because other input factors such as toxicity, epidemiology, ecological consequences and geographical distribution of effects, and other equity considerations are also necessary and important to an informed choice with respect to standard setting. Socioeconomic considerations are also specified under Government of Canada regulatory policy requiring federal regulatory authorities to demonstrate that the benefits of regulatory requirements are greater than their costs. Regulatory authorities must “ensure that the benefits outweigh the costs to Canadians, their governments and businesses. In particular, when managing risks on behalf of Canadians, regulatory authorities must ensure that the limited resources available to government are used where they do the most good” (Government of Canada, 1999). This implies not only that benefits should be greater than costs, but also that benefits minus costs, or net benefits are to be maximized, which means an attempt should be made to make standards efficient.

The objective of the expert panel process was to provide an independent, expert review and critique of the socioeconomic analyses (SEA)—in this case a cost-benefit analysis (CBA)—conducted in developing the Canada-Wide Standards on PM and ozone. Through a review of the models and associated data and assumptions used in the analyses, the panel was asked to produce a report to address the following questions:

1. What are the strengths, merits, limitations, gaps, and the degree of uncertainties of the proposed approaches, models, and their inputs and outputs?

2. By what means could the models and analytical approaches be improved, so as to minimize uncertainties and maximize the relevance, reliability, and utility of outputs?

3. What other approaches and/or tools could be used to conduct these analyses?

The benefits and costs associated with Canada-Wide Standards for PM and ozone are highly uncertain and controversial. Uncertainties are associated with each step in the analysis of benefits and costs—including the link between emission reductions and ambient air quality, the extent to which human health and the environment are affected by changes in ambient levels of pollutants, the economic values (as measures of preferences) associated with improvements in environmental and human health, accuracy of the emissions inventory and projections of what this inventory and other factors will be in a future baseline, and the scope and magnitude of economic costs associated with emission reductions, both to industry and to society.

With uncertainties so pervasive, analysts are required to make many choices and assumptions in estimating costs and benefits. For example, while it is clear that the epidemiological association between PM and excess mortality is consistent and robust, there are many remaining gaps in current understanding of the relative toxicity of PM components and gaseous copollutants and the magnitude of potential life-shortening effects. These uncertainties introduce possible biases into the estimation of PM-related health benefits.

The panel draws the following major conclusions from its review of the CBA undertaken for the development of CWS for PM and ozone and from the academic and policy literature relevant to this topic. In drawing these conclusions, the panel views the use of a structured approach to the examination of costs and benefits as a positive development in Canadian regulatory policy analysis.

1. The CWS socioeconomic analysis (SEA) was in fact limited to a cost-benefit analysis (CBA). Because the CWS implementation of the SEA process was judged by the panel to be limited to a CBA, it was reviewed as such. CBA is just one of many available decision support tools. The requirements of CBA in the CWS process depend on whether the purpose is to select an ambient air quality standard or to guide and evaluate the implementation process. The extent to which the results of the CBA can inform the CWS decision process is limited for various reasons including the following:

   • Provinces are required to establish implementation plans to ensure that CWS will be met. Therefore, when doing a prospective CBA, the measures to be implemented (i.e., revised emission control regulations) are necessarily undetermined, as are the compliance levels for the revised control regulations that will be achieved in various provinces.

   • The distributional impacts of both costs and benefits have not been assessed.

   The panel acknowledges that the CWS Development Committee for PM and Ozone describes the cost estimation as “preliminary and, in some instances, a cursory analysis used to provide a macro level order of magnitude perspective on the costs associated with the various optional levels for PM and ozone CWSs” and notes that caution should be exercised in their interpretation (Canada-Wide Standards Development Committee for PM and Ozone, 1999). Deficiencies in Canadian data and modeling capabilities and limited time and resources restricted the scope of the analysis that could be undertaken for the CWS process.

2. Credible CBA should be conducted to support the development of Canada-wide standards. The panel recommends that CBA be used to inform decision makers about the projected costs and anticipated benefits of CWS. CBA needs to be designed to distinguish between the costs and benefits of meeting alternative PM and ozone standards within the limits of current science. There are potential overlaps in the estimation of costs and benefits for PM and ozone because emission control strategies will impact both PM and ozone levels and it is not clear which components of the air pollution mix are responsible for the various health effects. These uncertainties in the CBA need to be clearly communicated. At its best, CBA provides the decision maker with a systematic identification, estimation, and measure of uncertainty of monetary values for the relevant costs and benefits of interest to decision makers and stakeholders. To be fully informative, the CBA results provided to stakeholders and decision makers need to adequately analyze and explain the major sources of uncertainty in the inputs of the CBA model projections and their likely effect on model outputs.

3. In view of the importance of the proposed regulatory decision, the CBA performed for the CWS for PM and ozone is deficient in relation to the state-of-the-art for CBA. If the CWS CBA was intended to provide an adequate basis for balancing costs and benefits and for influencing where the CWS should be set, this CBA was not up to the task. If the objective was strictly to confirm that costs were not exorbitant for CWS that were deemed to be both technically feasible and associated with some substantial benefits, then this CBA provided contributions toward those judgments.

   When judged against the elements of process and structure of CBA required for credibility as indicated in conclusions 4 and 5 that follows, the panel finds that the CWS CBA does not satisfy these requirements and does not meet a reasonable level of quality for a CBA to support a decision of this import. While the CWS CBA has some value as a scoping analysis and provides a limited degree of guidance for decision-makers, it requires substantial improvement to meet the criteria for credibility.

4. The process for using CBA in CWS needs improvement. A Discussion Paper on Particulate Matter (PM) and Ozone Canada-Wide Standard Scenarios for Consultation prepared by the Canada-Wide Standards Development Committee for PM and Ozone (May 1999) states that “in selecting PM and ozone CWS level scenarios for stakeholder considerations, an attempt was made to balance the anticipated benefits of improved air quality with the technological feasibility and costs of achieving those improvements.” But it is apparent that the standard CBA procedure of comparing incremental benefits for tighter standards with incremental costs was not done for the CWS CBA process. Timelines for the analysis were exceedingly short, and the CBA effort appeared to be underfunded, resulting in short-cuts that substantially reduced the credibility of the analysis. Reporting and communicating the CBA results was also ineffective, particularly in terms of conveying a clear understanding of what was done and why it was done as it was.

5. The panel has identified several CBA elements of primary concern that require attention in order to ensure the credibility of CWS CBA. The elements of primary concern for assessing credibility of CBA are the following:

   • Accuracy of emissions inventory data.

   • Accuracy of cost estimates.

   • Use of state-of-the-art air quality models.

   • Sufficiency of air quality monitoring.

   • Use of reasonable baseline assumptions for regulatory regime.

   • Inclusion of well-documented environmental conditions.

   • Inclusion of demographics in the CBA.

   • Adequate consideration of economic growth.

   • Selection of dose-response functions based on current weight of evidence.

   • Selection of valuation functions based on current weight of evidence.

   • Explicit expressions of uncertainty (measurement, model, and statistical).

   • Compatibility of scenarios with the form of standard (8-h and 24-h averaging times).

   • Inclusion of distributional analysis of costs and benefits (identification of sensitive subgroups, affected sectors).

   • Internal consistency of analyses (linking costs with benefits consistently).

   • Discussion of nonquantifiable endpoints.

   • Explanation for the choices of benefits and costs included.

   Performing CBA that meets these requirements will involve substantial investment on a continuing basis. Critics of the adequacy of the CWS CBA should support the generation of a knowledge base adequate to perform credible CBA. Generation of that knowledge base will require substantial investment of money, infrastructure and expertise. The panel has outlined its view on the limitations of the CBA undertaken for the development of CWS for PM and ozone. Given those concerns, the panel provides the following two conclusions on the measures of benefits and costs as calculated within the CWS CBA.

6. As in all CBA the estimates of benefits and costs are uncertain. Emerging analyses (in particular, of the tax interaction effect and the value of a statistical life) suggest that the costs associated with reducing emissions may be underestimated and the human health benefits overestimated. However, there are additional uncertainties that temper the impact of these emerging studies on the CBA. Most notably, the cost analysis performed in the CWS process is based on engineering estimates (resulting in overestimated costs) and the benefit measures do not include ecosystem effects (potentially large, but highly uncertain). The direction of the bias in net benefits depends on the weight placed on these factors.

7. The overall direction of the errors in benefits estimation is undetermined. The premature deaths reduced and broader range of health effects avoided by reducing PM ambient levels to the CWS PM standard are likely underestimated in the CWS analysis. However, the dollar value estimates for mortality reductions (based on value of statistical life, VSL) are very likely overestimated. The overall effect of these potential biases on the benefits realized from emission reductions is not clear from the current evidence.

Based on these conclusions, the panel makes the following recommendations.

1. Capacity building. Given the identified deficiencies in the CWS CBA, the panel recommends the following:

   • Canada should build a capability for conducting CBA for CWS by improving emissions inventories, air quality modeling capability, air quality monitoring networks, socioeconomic modeling, human health data gathering, and developing economic analyses of health-environment interactions. This capacity building will require long-term financial support to build the infrastructure, as well as government and industry commitment to making these improvements.

   • Data and models should undergo continuing development and refinement with reporting and documentation at periodic intervals that are integrated within the timeframe for decision-making. This includes particularly the air quality valuation model (AQVM) used to estimate the health benefits of air quality improvements. As detailed later in this report, the AQVM may already be out of date in its choice of dose-response functions for estimating mortality risk reductions and for valuation of this health endpoint. The decision to update AQVM to include more recent work depends on the criteria of study inclusion, such as degree of peer review, number of confirmatory studies, etc. The panel recommends that such criteria used to develop the AQVM be reviewed and updated, as necessary.

   • Improve Canadian capacity for air quality modeling. Collaborations with other North American agencies and research groups should be encouraged and supported with long-term funding.

   • Inclusion of risk–risk trade-offs (estimation of damages associated with risks from pollutants that increase as a result of the pollutants of interest being reduced, e.g., UV-B radiation increasing as a result of ozone concentrations being decreased).

   • All CBA model specifications and input values (e.g., risk coefficients, health event valuations) used for the purposes of regulatory decision making should be fully transparent and readily accessible to all interested stakeholders and researchers.

   • An explicit procedural and consultative framework should be developed for CBA to inform the decision process. Informed decisions require dialogue and consultation between decision makers, stakeholders, and CBA analysts in an open, transparent process. The decision process should require consideration of results of CBA along with other inputs to the decision.

   • Funding should be allocated, and roles and responsibilities within the CWS process should be defined—including an external expert advisory body to review approaches, progress, etc.

   • Formal guidelines for considering evidence and making and communicating decisions should be developed.

2. Communication. Improved two-way communication concerning the assumptions, limitations, and uncertainties associated with the methods and results of CBA is needed between analysts and policymakers and between policymakers and the public. Clear communication of the conceptual underpinnings and limitations of valuation techniques and the interpretation of the results of cost and benefit studies is needed to correct prevailing misconceptions about the conduct and interpretation of these studies.

3. Cost-benefit analysis and other types of socioeconomic analyses. Socioeconomic analysis (SEA) includes a wide variety of social and economic analysis methods, of which cost-benefit analysis (CBA) is one example and is typically the foundation for other socioeconomic analyses. However, the panel recommends that CBA be conducted separately from broader socioeconomic analyses, including plant closures, unemployment, regional economic impacts, competitiveness, or inflation for broad-based rules. Such analyses often ignore labor and capital mobility and are not commensurate with values used in CBA and therefore, when provided alongside CBA estimates, the results of broader SEA models may give rise to double counting of benefits or costs. If effects are expected to be borne disproportionately by only a few sectors, these types of analyses are useful, but they should be presented as contributions to the assessment of the distribution of impacts. A more promising, but more resource-intensive, approach is the expansion of the CBA to a general equilibrium analysis to capture the costs of the tax interaction effect. (see Section 7.4) As for competitiveness analysis, shifts to imports may have positive environmental effects that would need to be taken into account, and in this sense, a broader SEA or general equilibrium framework would be useful.

4. Cost-benefit framework for analysis of environmental quality regulations. The panel endorses the use of a cost-benefit framework for the analysis of environmental regulation while recognizing the empirical limitations of CBA. The panel recommends:

   • Continued development of methods for accurate assessment of costs and benefits, including methods for the analysis of general equilibrium (including tax interaction) effects and international trade impacts of regulatory change.

   • Continued development of and communication regarding alternative decision-making frameworks, including multi-attribute methods, to be used as methods to “triangulate” with traditional CBA.

   • Investments in human capital in the area of CBA of environmental regulation so that policy makers and the Canadian public can be confident that cost and benefit measures accurately reflect Canadian values and public preferences as well as Canadian institutional arrangements.

Notes

This work has been previously published by the Royal Society of Canada, Ottawa (Ontario), in June 2001, appearing as “Review of Socio-Economic Models and Related Components Supporting the Development of Canadian Standards for Particulate Matter and Ozone,” ISBN 0-920064-73-6.

∗See U.S. EPA, September 2000. Guidelines for Preparing Economic Analyses. 240-R-00-003.

∗The theoretically appropriate measure of opportunity cost is the change in producer surplus. Producer surplus is defined as the area above the firm's supply or marginal cost curve but below price. As such it is a measure of the gains the producer has for selling at prices higher than marginal cost. Regulatory policies typically shift the marginal cost curve by increasing costs of production. However, since the firm can respond to regulatory change in many ways, not only through direct implementation of emissions reduction technology, the cost of implementing emissions reduction technology is thought of as an upper bound on the cost estimate.

^†While the benefit measures are referred to as “willingness to pay” or “willingness to accept,” most benefit measurement techniques do not actually directly ask individuals to reveal this amount. Individuals may reveal their willingness to pay or willingness to accept through market transactions or other forms of behaviour. There is considerable confusion in the popular literature about the concept of “willingness to pay” (a theoretically appropriate welfare measure) and the method of benefit estimation, contingent valuation, that actually asks individuals what they would be willing to pay.

∗A special but important issue arises here when regulatory impacts have an intergenerational time scale (e.g., costs borne today but benefits received by the next generation). Even a consumption-based discount rate could reduce future impacts to trivial levels over a long time frame. In terms of discount rate policy, it could be argued that intergenerational effects deserve “special” treatment reflecting societal trade-offs across generational income distributions. One simple example is the argument that a discount rate reflecting the long-term rate of economic growth can reflect an “equal” treatment of the generations, reflecting the presumed greater economic ability of future generations to shoulder burdens relative to ourselves.

∗A CWS for PM_2.5 of 30 μg/m³, 24 h averaging time, by year 2010. A CWS for ozone of 65 ppb, 8 h averaging time, by year 2010.

∗Rowe, R. D., et al. 1995. The New York Electricity Externality Study. Dobbs Ferry, New York: Oceana.

^†Cropper and Freemen selected six VSL studies of 21 as “best” for use in policy analysis. Four are wage risk studies and two are contingent valuation studies. The wage-risk estimates range from $3 million to $9 million (1996 CDN dollars), and the contingent valuation estimates range from $4 million to $5 million (1996 CDN dollars). The arithmetic mean of all six selected VSL estimates is about $5 million (1996 CDN dollars).

∗Tables 1A–4H of Section 2 of Compendium of Benefits Information.

^†Tables 5A–9H of Section 2 of Compendium of Benefits Information.

^‡Tables 10A–12H of Section 2 of Compendium of Benefits Information.

∗The assumptions are based on the analyses performed for the Atmospheric Science Expert Panel for the Sulfur in Gasoline and Diesel Fuels Program, the Environment Canada review of proposed U.S. Environmental Protection Agency (EPA) NO_x Rules for 22 eastern U.S. states, and scientific work for the NO_x/VOC program and the Canada-Wide Acid Rain Strategy for Post 2000.

∗We use the term total particulate matter or TPM to refer to all forms of particle matter entering the atmosphere. This material comes in a variety of sizes, shapes, and compositions and it is often characterized by an effective radius or diameter (effective since the particles need not be spherical.). Thus PM_2.5 refers to that part of TPM that has an effective aerodynamic diameter less that 2.5 μm and is often described as the “fine” part of the distribution, while larger particles are referred to the “coarse fraction.” PM₁₀ refers to that part of the TPM distribution that has diameters less than 10 μm and thus includes the PM_2.5 part of the distribution. We use the term particulate matter or PM when we do not need to discriminate between the various parts of the distribution.

∗Estimates of NH₃ emissions have recently been obtained for regional studies in southwestern Ontario, but are not generally available for Canada wide studies.

∗The third highest maximum is chosen in an attempt to ameliorate the effects of outliers, that is, anomalous extreme events.

∗Although there is a growing suspicion that these types of air quality model can represent the climatology of ozone more robustly than specific incidents (Bouchet et al., 1999; Kasibhatla & Chameides, 2000).

∗The planetary boundary layer (PBL), also called the atmospheric boundary layer and over the ocean the marine boundary layer, is the atmospheric region next to the surface in which the air is rapidly mixed vertically during the daytime to a (generic) height of about 1 km. At night, over land, the thickness of the layer shrinks to about 100 m due to the lack of input of solar energy to drive the mixing. During the night most emissions of pollutants into the PBL are trapped unless they are from stacks or have sufficient buoyant energy to “break through” the PBL.

∗The benefits estimated by Lutter and Woltz (1997) are probably overestimated since they assumed that the application of CAAA would lead to a decrease of 10 ppbv in seasonally averaged ozone levels. This is probably an overestimate since it is often overlooked that background ozone is actually the global accumulation or generation of regional chemistry in the industrial countries and biomass burning in the developing countries along with ozone input from the stratosphere. Thus local or regional air quality problems have a global context, and this should not be overlooked.

∗The AQVM Methodology Final Report (pp. 5–27) notes that average levels of sulfate particles in southern Ontario are about 18% of average PM₁₀ levels in the same area according to Dann (1994).

∗Hazilla and Kopp (1990) conclude from their study of the costs of the Clean Air and Water Acts that monetary measures of change in well-being grow to exceed compliance costs over a 10-yr period.

^†A partial equilibrium analysis would focus on a narrow set of economic agents (producer and consumers) and would assume agents outside this set would be unaffected by the policy.

^‡A general equilibrium analysis makes no assumptions about affected parties and treats all agents in the economy as if they could be affected.

∗Supporting arguments cited: A free trade environment will lead to similar business/environmental compliance strategies, and multinationals in Canada and the United States will use similar control methods.

∗The delay in the realization of risk reductions could occur either because the installation of pollution control equipment today will not benefit young people until they become susceptible to the effects of pollution (the air pollution case described above), or because the program reduces exposure today to a substance that increases risk of death only after a latency period (e.g., asbestos).

^†Similar adjustments can be made to account for the effect of latency periods. According to the life-cycle model, a 40-year-old's WTP to reduce his probability of dying at age 60 should equal what he or she would pay to reduce his current probability of dying at age 60, discounted back to age 40.

∗We ignore here the large body of literature using an hedonic property value approach. This approach provides a revealed WTP for air pollution reductions but is dependent on housing market perceptions about pollution and links to all types of effects, health being only one. It has the advantage (some would say disadvantage) of not using any concentration-response information.

∗Note that these studies were published before the recent literature questioning the traditional estimates of the VSL.

∗Extracted from Kopp et al., (1997).

^†Laws that bar discrimination are other obvious examples of instances where the preferences of some have been over-ridden by the political decisions of society as a whole.

∗The welfare economist Harsanyi (1955) states the economic view most directly: “The principle that, in deciding what is good and what is bad for a given individual, the ultimate criterion can only be his own wants and his own preferences.” The “Principle of Autonomy” that Harsanyi articulates does not depend on the reasons one has for particular preferences. What matters for Harsanyi is that individuals apply the weights and the weights are permitted to be specific to each individual.

^†Models of “egoism” generally restrict preference to those things that benefit the individual directly. Thus a “preference” for self-sacrifice in the attainment of some worthy goal, for example, would be excluded.

^‡See the report of the Senate Judiciary Committee on S.343, 25 May 1995, p. 59.

∗If the argument is that preferences are not linked to well-being and therefore economic value is not linked to well-being, one is restating the preference satisfaction critique.

^†A corollary to this statement is that there are some things that should not have a price tag placed on them.

^‡Analyses of the economic value of recreational experiences have used this approach, quantifying the monetary value of those things given up to recreate, to calculate a lower bound on the value of recreation experiences.

∗In reality, benefactors and sufferers may be one and the same.

∗WGAQOG, Working Group on Air Quality Objectives and Guidelines.

∗Centre for Research in Earth and Space Technology.

^†Network for Risk Assessment and Management.