764
Views
0
CrossRef citations to date
0
Altmetric
Articles

Electric utility valuations of investments to reduce the risks of long-duration, widespread power interruptions, part I: Background

, ORCID Icon, , &
Pages 311-322 | Received 14 Sep 2022, Accepted 12 Nov 2022, Published online: 19 Dec 2022
1

ABSTRACT

Power industry stakeholders are devoting increasing attention to the risks of long-duration, widespread interruptions (LDWIs) in electricity service. There is concern that these risks are heightening due to more frequent and severe extreme weather events. Numerous studies have examined various aspects of the problem, primarily from an engineering and conceptual perspective. This is the first of two papers reporting the results of a study of LDWIs that focuses on their economic aspects, takes an empirical approach, and includes consideration of institutional factors affecting utilities’ efforts to reduce their vulnerabilities to these disruptions. This paper presents background on the problem, including cost concepts relevant to economic valuation of measures to reduce the risks of LDWIs, valuation methods, and the role of the concept of ‘resilience’ in shaping analysis in this area. This material provides context and motivation for the second paper, which reports on a series of case studies.

This article is part of the following collections:
Adaptive Pathways for Resilient Infrastructure

1. Introduction

In recent years, policymakers and regulators at all levels of government, the electric power industry, and other stakeholders have devoted increasing attention to the risks of long-duration, widespread interruptions (LDWIs) in electricity service. There is widely shared concern that these risks are increasing because of increases in extreme weather and a greater potential for deliberate physical and cyber-attacks on the power system, among other threats. Numerous studies and reports have examined aspects of the problem, generally focusing on the bulk power system and emphasizing anticipatory planning, policy, and regulatory needs, as well as preventive measures to reduce the power system’s vulnerability to external threats.

A number of policy and technical studies on LDWIs have focused on truly catastrophic events that affect multi-state regions or even the entire country and that might last for many weeks or months, such as Campbell (Citation2012) and The President’s National Infrastructure Advisory Council (NIAC; Citation2018). These have been termed ‘black sky’ disruptions (Stockton, Citation2014). Research on this topic has generally concentrated on physical and engineering aspects of the electricity system and its vulnerabilities and has been organized around the concept of the system’s ‘resilience’ – its capacity to withstand disruptive external events without interrupting service to customers, or to recover expeditiously when interruptions occur.

This is the first of two papers that complement such work in several waysFootnote1:

First, we focus on economic aspects of efforts to prevent or mitigate the severity of LDWIs. Broadly speaking, precipitating events, including natural disasters, result in three types of economic impacts related to electric power systems: (1) these events damage or destroy parts of the system’s physical infrastructure, thereby imposing costs on utilities (and their customers and shareholders) for repair and recovery; (2) the power interruptions caused by these events impose costs on the utility’s customers as a result of loss of electricity; and (3) interruptions of sufficient scope and duration can have indirect impacts on the local or regional economy. The observed or estimated magnitudes of these costs can in turn provide a basis for valuing investments and measures to reduce the likelihood of their occurrence in the future – that is, to minimize power systems’ vulnerabilities to future events and to facilitate rapid restoration of service when power interruptions occur. We examine the methods and information used by electric utilities to make such valuations.

Second, we take an empirical approach to this topic, in the form of five case studies based on past power interruptions caused by several different types of extreme weather events in utility service territories around the U.S., and how they affect planning for future events. This pragmatic orientation contrasts with the more conceptual and speculative approach taken in many studies. Accordingly, we are able to fill a gap in knowledge about the actual effects of such disruptions and the means by which utilities and regulators have sought to avoid them in the future.

Third, we examine what could be called ‘dark sky’ events, interruptions that affect large areas and last from several days to a few weeks, but are not in the category of a nation-wide, months-long catastrophe. These intermediate-scale interruptions are historically the most common type of major electric power disruption. They extend beyond the short outages addressed in utility reliability but are not as long as major catastrophic interruptions that last for months. Our examination of these intermediate types of events fills a gap between conventional electricity reliability analysis and recent research into catastrophic disruptions. In addition, we focus on electricity distribution infrastructure at the utility service territory level rather than on bulk power long-distance transmission systems, disruption of which may be involved in catastrophic, multi-region or national power interruptions.

Fourth, we examine the concept of power system resilience in pragmatic terms, investigating its application by utilities and regulators. Although the term is widely used, its practical meaning and operational implications remain elusive. Studying how utilities and regulators address resilience may provide valuable insights that have largely been missing in the literature on this topic.

Finally, we take into account regulatory and other institutional influences on utilities’ preparations for, and responses to, extreme weather events and associated power interruptions. Utilities’ actions are based on laws, regulations, and established practices. These factors may affect the response to past events and preparations for future threats, but to-date have received very little attention in the research literature. We investigate their role and importance.

This first paper begins with background on electricity system vulnerabilities to extreme events and provides key examples of U.S. federal government as well as academic research on this topic. It goes on to discuss the concept of resilience of power systems. Next, basic concepts and methods pertaining to economic valuation of reliability and resilience are reviewed, including cost-effectiveness and cost-benefit analysis, direct and indirect costs of power interruptions to utility customers, and regional economic modeling. The paper ends with a conclusion and remarks on policy implications.

This material provides a foundation for the research described in the second, companion paper, which addresses three questions:

(1) How do utilities or others estimate the costs and benefits of investments to reduce power system vulnerabilities to future extreme weather events?

(2) How do utilities and regulators use the concept of resilience in estimating the economic value of investments to prevent or mitigate the severity of LDWIs?

(3) How do regulatory processes and other institutional factors influence utilities’ economic analysis of power interruptions?

The premise of the work discussed in these papers is that studying examples of actual practice can complement existing, primarily theoretical, conceptual, or simulation model-based work on the economics of electric power system vulnerabilities and may provide valuable lessons for utilities, regulators, policymakers, and other stakeholders grappling with the challenges of estimating the costs and benefits of measures to reduce these vulnerabilities. Moreover, the utility service-territory-level focus can help link regional and national bulk power system planning to the efforts of utilities and state regulators to increase the resilience of more local electricity distribution systems.

The remainder of this paper is organized as follows. In Section 2, we provide examples of previous studies of electricity system vulnerabilities, and then discuss the idea of electricity system resilience and the concept of ‘dark sky’ power interruptions. We then provide background on relevant economic concepts, methods, and data, including cost-effectiveness and cost-benefit analysis; avoided, direct, and indirect economic impacts of power interruptions; empirical findings and computational modeling; and regulatory environments for addressing these topics. We then summarize the motivation for this study to provide a precis for the companion Part II paper, which presents results of the five case studies.

2. Background

2.1. Electricity system vulnerabilities to extreme events

A report by the U.S. National Academies of Science (National Academies of Sciences (NAS), Citation2017) examines the increasing vulnerability of the national electric power system to extreme weather as well as to cyber threats, physical attack, and human error. The Academies’ report gives a technical and operational overview of the power system; the causes of grid failure and power interruptions; and strategies to anticipate, mitigate, and facilitate recovery from disruptions. Although taking a prospective view, the report briefly mentions several examples of past events and interruptions, including lessons to be drawn from them in planning for future disruptions. The report recommends a number of actions including, but not limited to:

  • improving planning and coordination, emphasizing inter-agency, inter-governmental, governmental-electric utility engagement;

  • increasing the practical and empirical knowledge base on physical/engineering impacts of triggering events and prevention and mitigation of resulting power interruptions.

The National Academies also made recommendations specifically to the U.S. Department of Energy (DOE), which include:

  • ‘[developing] comprehensive studies to assess the value to [utility] customers of improved reliability and resilience,’

  • ‘[conducting] a coordinated assessment of the numerous resilience metrics being proposed for transmission and distribution systems and [seeking to operationalize] these metrics within the utility setting, with “engagement with key stakeholders [being] essential”.’

A 2018 report by the U.S. President’s National Infrastructure Advisory Council (The President’s National Infrastructure Advisory Council (NIAC), Citation2018) assessed and analyzed the risks of future LDWIs and how to improve the nation’s capacity to prepare for, mitigate, and rapidly recover from them. NIAC’s study focused on catastrophic disruptions outside the range of historical experience, in which power might be out for months or longer, affecting tens of millions of customers across multiple states. It addressed prospective technical, policy, and regulatory elements of risk management and of preventing or mitigating such catastrophic events. Although the report was generally forward looking, it briefly reviewed five past large power outages and lessons that should be learned from these experiences.Footnote2

DOE has published several studies on the risks of severe weather and other events to the power system, the physical impacts of these events, and details of resulting power interruptions. U. S. Department of Energy (DOE; Citation2010) examined the impacts of the 2005 and 2008 hurricanes on U.S. energy infrastructure. A subsequent study (U. S. Department of Energy (DOE), Citation2013) focused on the impacts of the 2005 and 2008 hurricanes on energy infrastructure in the Northeast, including utility distribution system-level effects. U. S. Department of Energy (DOE; Citation2015) examined U.S. regional climate change vulnerabilities and the potential impacts of climate change-caused extreme weather events on the energy system. This study used examples of past impacts but did not analyze them in detail. The study also listed examples of ‘resilience solutions’ for potential impacts but did not propose a framework to help utilities, regulators, and other stakeholders evaluate proposed solutions. U. S. Department of Energy (DOE; Citation2016) presented a comprehensive framework for identifying and analyzing potential climate change threats to the electricity system and measures to prevent or mitigate their impacts, and U.S. Department of Energy (DOE; Citation2017) provided a technical overview of the modern power grid, identified risks and threats, and discussed grid operations and management strategies for addressing them. The U.S. DOE Grid Modernization Laboratory Consortium (GMLC) has published reference documents that discuss defining ‘metrics for the purpose of monitoring and tracking system properties of the electric infrastructure as it evolves over time’ (Grid Modernization Laboratory Consortium (GMLC), Citation2017).Footnote3

Other relevant work sponsored by the federal government includes a 2014 report by the U.S. Government Accountability Office (United States Government Accountability Office (GAO), Citation2014) on the increasing vulnerabilities of U.S. energy infrastructure to severe weather. GAO found that actions necessary to address these vulnerabilities generally fall into two broad categories: (1) hardening – measures that reduce the vulnerability of infrastructure, and (2) resiliency – measures that speed recovery of the power system in the event of a service disruption. GAO also offered suggestions for how selected federal entities could address these vulnerabilities and summarized various federal roles, by entity, in relation to energy infrastructure. The report mentioned a series of examples of past impacts. More recently, an assessment by the U.S. Global Change Research Program found that U.S. energy infrastructure is becoming increasingly vulnerable to climate-change-caused severe weather and water shortages (U. S. Global Climate Research Program (USGCRP), Citation2017). There has also been a considerable amount of academic research on the vulnerability of power systems to extreme weather and other disruptions, with examples including Maliszewski and Perrings (Citation2012), Kwasinski (Citation2016), and Shen et al. (Citation2021).

2.2. Definition of resilience

Resilience is a widely used rubric in studies of electric power system vulnerability and other related types of analysis. It has been broadly defined by the U.S. Executive Office of the President (EOP) as:

“the ability to prepare for and adapt to changing conditions and to withstand and recover rapidly from disruptions. … Resilience includes the ability to withstand and recover from deliberate attacks, accidents, or naturally-occurring threats or accidents” (Executive Office of the President of the United States. (EOP), Citation2013a).

The generality of this definition has motivated efforts to clarify the underlying concept of resilience, its relationship to electricity reliability, and the relatively new challenges associated with operationalizing it. In a study for the National Association of Regulatory Utility Commissioners, Keogh and Cody (Citation2013) review the concept, problems with defining it precisely, and its practical application. They highlight distinctions between problems associated with resilience and those associated with conventional reliability issues. Stockton (Citation2014) provides suggestions for how utility regulators can conceptualize and operationalize resilience in order to better prepare for these risks. A recent paper by the National Regulatory Research Institute discusses the question of determining appropriate levels of investment in measures to prevent or mitigate severe power disruptions. The author notes that the value of such investments has increased as a result of more severe weather and greater risk of cyber-attacks but also highlights the fact that there is a large amount of uncertainty in such investments, particularly with regard to their benefits, and that cost-benefit analysis (CBA) is therefore challenging.

Notwithstanding these and similar studies, however, the practical meaning of resilience as a distinct category continues to be unclear despite a considerable body of research attempting to make it precise. This research has been disproportionately conceptual and prospective rather than pragmatic and empirical, and it has generally not focused on valuation per se. Many proposals have been made for ‘metrics’ to gauge resilience at varying levels of specificity. With few exceptions, such work has been mostly of an academic nature, and regulators and utilities have not made practical applications of it. As Anderson et al. (Citation2019) put it, ‘ … while many metrics have been proposed …, most remain immature, and no generally-agreed upon resilience metrics are widely used today.’ In addition, a number of prospective ‘frameworks’ have been proposed for making decisions on investments to increase resilience; this work has also been conceptual with little or no grounding in what has actually been accomplished to date by regulators and utilities.

A recent multi-stakeholder expert panel review observed that:

“There is a strong need for robust metrics, methods, and associated planning methodologies to quantify risk within an overall framework for grid resilience to weigh resilience improvements against other goals and investments” (Institute of Electrical and Electronics Engineers (IEEE), Citation2020, p. 3).

What is noteworthy about this statement is that it reveals implicitly that, in the judgment of an authoritative body of experts, years of research on these topics at an array of both public and private, academic and non-academic institutions has failed to meet this need.

Notwithstanding this state-of-affairs, measures to mitigate power system impacts of extreme events, including utility storm-hardening and related efforts (e.g., undergrounding distribution lines, managing vegetation), are often called ‘resilience investments’ in that they can reduce weather-caused physical impacts and power interruptions and/or facilitate faster system recovery from these impacts. The use of this term can convey the notion that such measures are somehow novel. However, many such investments and similar actions are, in practice, already a focus of utilities and regulators in their existing and ongoing efforts to improve electricity reliability (Finster et al., Citation2016). LaCommare et al. (Citation2017) interview public utility commission staff in three jurisdictions – California, Florida, and the District of Columbia – to understand how they assess the economics of investments in reliability and resilience that are proposed to them by utilities. Key findings from this study include: a) little or no distinction is made between reliability and resilience in reviewing proposed projects; b) the costs of investments in reliability or resilience are well understood by utilities and regulators; c) the benefits of such investments are difficult to monetize; and d) there is a need for improved information on the costs that power interruptions impose on utility customers (which can be used to monetize the benefits of investments). LaCommare et al. (Citation2017) also find that most utility requests for regulatory approval of the costs of reliability-resilience investments are made during proceedings around general rate cases.

2.3. Estimating the economic value of reliability and resilience investments

2.3.1. Cost-effectiveness and cost-benefit analysis

Electric utilities operate under a compact with state regulators to provide safe, reliable electricity service at rates that are considered just and reasonable. Standards for reliability are not set by law; instead, individual states and utilities have over time evolved understandings about how to balance spending on reliability with expectations of the level of reliability that will be provided. Evaluation of future investments must consider impacts on electricity rates and affordability. In this context, regulatory oversight focuses on ensuring that ratepayer dollars are spent prudently; the common economic criterion for assessing reliability measures is cost-effectiveness. Cost-effectiveness analysis (CEA) is used to identify efficient options for meeting specific goals or targets. In electricity reliability, for example, some states have requirements defined in terms of numerical limits on allowable numbers and durations of customer power interruptions. Utilities are expected to maintain their systems so that these limits are not exceeded. A utility may have multiple options of different costs for meeting its reliability obligation, such as pole replacement, undergrounding, or vegetation management measures. CEA can be used to determine the least-cost combination of measures that will meet a particular reliability target.

Following an extreme event, regulated utilities are generally required to report any costs that they incur in the course of restoring power to their customers and repairing or replacing damaged distribution, transmission, and generation infrastructure. In recent decades, however, severe electric power interruptions caused by natural disasters have led some commissions and state governments to initiate extraordinary (non-routine) proceedings to review utility response and preparedness for future large, disruptive events, as a result of the expectation that the frequencies and magnitudes of these large events will increase. Such proceedings are primarily driven by the significant costs that LDWIs in particular jurisdictions have imposed upon utility customers. The proceedings aim to develop and implement strategies to prevent such costs in the future by reducing the system’s vulnerability to these types of interruptions.

In most cases, these proceedings review the formal or informal standardsFootnote4 that were used in the original construction and/or maintenance of electricity infrastructure that has been compromised or damaged, and discussion centers around what standards should apply to rebuilding or redesigning the replacement infrastructure. Sometimes the adequacy (or applicability) of the standard itself is examined, and there is an assessment of whether it should be increased or strengthened to ensure that utilities are ‘building it back better.’

Two problems have arisen in this type of process. First, utilities and regulators have not necessarily had experience in preparing for storms, hurricanes, and other events of unprecedented magnitude. Thus, they may not have had sufficient information to determine with complete confidence how much a more stringent standard will reduce their system’s vulnerability. Second, higher levels of storm preparedness invariably require greater expenditures than expenditures that are routinely made to maintain acceptable levels of reliability. One implication of the first problem is that utilities’ and regulators’ established understandings of how to set standards and apply CEA to meeting those standards may not readily apply in this context. An implication of the second problem is that estimates of the potential benefits of the greater investments in preparedness become much more salient in the regulatory process. That is, the primary benefits come from reduced customer exposure to future power disruptions. But exactly how much reduction will be achieved for given increases in investment, and whether a given level of reduction is, in colloquial terms, ‘worth it,’ becomes a challenging problem.

This issue can be illustrated by the example of evaluating measures to reduce customer outage times after a major storm. Suppose a utility estimates that some type of storm-hardening investment costing $X will reduce the average frequency and/or duration of customer outages by Y%. Should these investments be made? Under the approach of an a priori target for reduced outage times (or frequency) and a cost-effectiveness test, the question would be whether Y%/$X is the least-cost option. However, if this is an instance of seeking to increase protection against outages to a greater-than-historical level, setting the target may be part of the problem. It may be the case that $X in expenditures would be considered large in terms of the jurisdiction’s historical experience and would be recovered by increasing customer electricity rates. For this reason, assessing the value – i.e., the economic benefits – of these incremental investments becomes important. Estimates of the benefits can be compared to the capital and operations and maintenance (O&M) costs to determine whether the investments are worthwhile. Cost-benefit rather than cost-effectiveness analysis can be an appropriate methodology for evaluating potential investments in storm-hardening and related measures, especially if the investments are larger than past spending on blue or gray sky reliability measures. Cost-benefit analysis (CBA) is used to determine the optimal levels of, e.g., investments in reliability or resilience, in terms of both the costs and the resulting benefits of these investments when there are no pre-determined standards or requirements. In this case, the costs of measures such as pole replacement, undergrounding, and vegetation management are compared to their monetized benefits in terms of avoided power interruptions caused by extreme weather or other precipitating events. The CBA criterion is to invest in these measures up to the level at which their incremental benefits equal their incremental costs.

Accordingly, defining and monetizing these benefits is an important step in assessing investments to improve power system resilience in order to avoid or minimize damage from future extreme events. In the next subsection, we review concepts, methods, and empirical information pertaining to this topic.

2.3.2. Direct costs to utility customers

The costs that power interruptions impose on utility customers are defined in several ways, depending in part on the type of customer affected. Interruptions affect commercial and industrial (C&I) utility customers by impeding or curtailing their production of goods and services. Commercial firms may need to close their office facilities because of a lack of lighting and air conditioning, and industrial firms may be unable to run machinery and other systems that manufacture their products.Footnote5 These impacts are generally measured as losses in normalized dollars as functions of the length of the power interruption, time of day and the season, facilities affected, presence of backup generation, current inventory, and other factors.Footnote6 Although terminology varies, these can be described as direct costs to these customers.

For residential customers, the standard approach to defining interruption costs is to first define and estimate the economic ‘utility’ (i.e., well-being or worth) that customers derive from using electricity services such as lighting, heating, and cooling (usually as a function of kilowatt-hours [kWh]). Residential customer costs from a power interruption are then defined in terms of the utility the customer loses as a consequence of electricity curtailments. Another approach to estimating this type of economic value is defining it directly in terms of the value to electricity users of improvements in the reliability of their electricity service, such as reducing the number and/or frequency of power interruptions over the course of a year.

Although the metrics described above for residential customers represent a form of direct cost, they are more often referred to as ‘avoided costs.’ This terminology also applies to C&I firms. These quantities are also referred to, both in the research literature and in practice (by utilities and regulators), as the value of lost load (VOLL) or the value of [electricity] service.

The terms ‘avoided’ and ‘value’ indicate the primary significance and use of these cost concepts as measures of the potential benefits of investments and other measures aimed at avoiding the power interruptions that result in these costs. That is, the worth of such investments is gauged in terms of the customer economic losses that they may prevent. This framing of costs and benefits is central to the analysis presented in this two-part paper. Historically, most work on the economics of power interruptions has addressed the direct or avoided costs of momentary or short-term interruptions, i.e., those lasting seconds, minutes, or hours, but generally no more than 16 hours (e.g., see, Larsen et al., Citation2019). The standard methodology for estimating the economic impacts of such disruptions is to survey utility customers self-reporting these impacts either retrospectively – actual costs incurred due to a previous interruption – or prospectively – anticipated costs in the event of a hypothetical future interruption(s) of specified severity and particular duration. Residential and small C&I customer surveys are often administered by mail, phone, or online. Large C&I surveys are typically administered by trained auditors who interview customers in person to facilitate understanding of the question and ensure accuracy of the responses.

Numerous studies have focused on estimating direct or avoided costs of power interruptions to utility customers regardless of cause, such as Swaminathan and Sen (Citation1998), Primen/EPRI (Citation2001), and LaCommare and Eto (Citation2006).Footnote7 For example, Larsen (Citation2016) proposes a framework for evaluating the costs and benefits of one type of undergrounding transmission and distribution lines, the benefits of which include the avoided, direct interruption costs from less frequent and/or shorter-duration power interruptions.Footnote8 Richter and Weeks (Citation2016) discuss econometric methods for estimating electricity customers’ willingness to pay for improved resilience, using United Kingdom data. Sagebiel (Citation2017) also discusses such methods and estimated willingness to pay for improved reliability, among a sample of urban customers in India. Morrissey et al. (Citation2018) estimate the welfare costs of power outages in a region of England.

The Lawrence Berkeley National Laboratory (Berkeley Lab) Interruption Cost Estimate (ICE) Calculator is an online tool based on 34 utility customer interruption cost surveys from across the U.S. This tool is designed for use by utilities, regulators, and others in estimating interruption costs or the avoided costs resulting from investments in power system reliability (Sullivan et al., Citation2018, Citation2015). Campbell (Citation2012) and the Executive Office of the President of the United States. (EOP; Citation2013a) use the ICE estimated customer damage functions for power outages (LaCommare & Eto, Citation2004; Sullivan et al., Citation2009). Rosales-Asensio et al. (Citation2021) used ICE Calculator data to estimate the resilience benefits of distributed energy at several sites in New York City. LaCommare et al. (Citation2018) review improvements in avoided cost data over time and using these data estimate that the total cost of all power interruptions in the U.S. is roughly $44 billion per year. Campbell finds that severe-weather-related power interruptions cost the U.S. economy $20-$55 billion each year, and the EOP finds a range of $5-$75 billion.

A recent report for National Association of Regulatory Utility Commissioners examines methods for valuing the resilience benefits of distributed energy resources (National Association of Regulatory Utility Commissions and Converge Strategies LLC (NARUC/Converge), Citation2019), a topic that has received much attention in electricity policy circles. Zamuda et al. (Citation2019) survey benefit categories used to justify investments that reduce the vulnerability of power systems to extreme weather and climate change. In addition to avoided customer interruption costs, the paper identifies a number of benefit streams that have been included in formal cost-benefit analyses of investments in distribution system resilience with respect to extreme events, including avoided costs to utilities and injuries and fatalities prevented. However, the Zamuda et al. (Citation2019) study finds a limited number of examples of avoided costs being used in formal cost-benefit analyses of power interruptions lasting more than 24 hours. Perhaps most importantly, Zamuda et al. (Citation2019) finds no examples of avoided, indirect economic impacts on the broader economy being included as benefits in a formal cost-benefit analysis. We discuss these types of regional economic impacts in the following subsection.Footnote9

2.3.3. Direct and indirect regional economic impacts

In addition to causing direct costs, power interruptions disrupt the flow of commerce among businesses, industries, and entire sectors of the economy. That is, the loss of power to a supplier of some good or service and the resulting reduction (or cessation) of production of that business’s products will in turn affect both its suppliers and its purchasers (customers), whose upstream or downstream operations, respectively, may be impeded. For example, loss of power at a steel plant will reduce or stop the production of steel, resulting in a direct cost to the producer from reduction in sales revenue, and potentially causing customers who purchase the plant’s steel to slow or shut down their operations (i.e., their manufacture of steel products). The consequent losses of business to the customers result in indirect economic impacts of the power interruption on the suppliers. As discussed earlier, avoided economy-wide costs also implicitly define the benefits of investments that might prevent future LDWIs.

There is significantly less literature on the economic impacts of large-scale, long-duration interruptions than on short-term outages (Larsen et al., Citation2019). Most of the existing literature detailing these types of impacts is based on computer simulation models of local or regional economies, depending on the scale of the modeled interruptions.Footnote10 These include models that represent supply and demand in markets for goods and services, and how these markets interact, as well as models that focus on the relationship between employment and overall economic output. In the current context, these models are designed or augmented to provide information on electricity use, how it contributes to the functioning of an economy, and the costs resulting from its interruption.

There are a number of reasons for using these types of models. First, they are able to represent and analyze both direct and indirect impacts of power interruptions. The latter are those that propagate through an economy through market interconnections, such as ‘upstream’ effects – firms experiencing input shortages due to the interruption’s impacts on their suppliers – and ‘downstream’ effects, firms incurring losses because of its impact on their customers. This is an important advantage of the models over customer surveys, which focus on the direct costs to customers. Second, customer surveys are usually targeted at residential, commercial, and industrial customers whereas the models are comprehensive and can in principle analyze power interruption impacts on all sectors of an economy. Third, some types of models incorporate customers’ and businesses’ adaptive actions to reduce the impacts of interruptions (e.g., rescheduling or relocating production following a power disruption). Fourth, these models can represent effects on economies over time (days, weeks, months) even after the power has been restored. Computational economic modeling has been applied to estimate costs of LDWIs that result from several types of triggering events, both actual historical cases and hypothetical scenarios.

For example, Rose et al. (Citation1997) use input-output modeling and linear programming to estimate the potential costs of electricity outages caused by a hypothetical earthquake in the New Madrid Seismic Zone in Memphis, Tennessee. Rose et al. (Citation2005) use a computable general equilibrium (CGE) model to retrospectively study the effect on the metropolitan Los Angeles economy of rolling blackouts due to power supply shortages that lasted for one to eight hours over several days in California in 2001, during the state’s energy crisis. Those authors find that the combined direct and indirect costs amounted to 1.3% of this economy’s annual gross output, and that adaptive responses considerably reduced the economic impacts of the blackouts. Greenberg et al. (Citation2007) use a macro-econometric model to analyze the economic effects of a power interruption resulting from a hypothetical terrorist attack in New Jersey under various assumptions about the magnitude and duration of the interruption and the speed of recovery. In this study, the most important factor in determining the ultimate economic losses is the impact on state employment. In the most pessimistic scenario, the researchers simulate a loss of 5.5% of power with full restoration after two weeks but a 1.6% reduction in employment relative to the baseline in the first year after the interruption, and a continuing 1.5% reduction after five years. In this scenario, state annual gross domestic product would be reduced by 1.6% ($397 billion) in the first year after the event, 3.3% ($400 billion) in the second year, and 1.8% ($458 billion) in the fifth. There have also been several economic modeling studies of direct and indirect costs of tropical storm Sandy in 2012. For example, Boero and Edwards (Citation2017) use a CGE model to estimate the direct and indirect impacts on the U.S. East Coast economy of the Sandy-caused power interruptions to have been roughly 0.83% of baseline regional economic output in 2012, or about $53 billion. Sue Wing and Rose (Citation2020) use a CGE model to estimate the costs of electric power interruptions caused by a severe earthquake in the San Francisco Bay Area.

2.3.4. Other impacts that can be monetized

In addition to disruptions of economic activity, other indirect societal impacts of power interruptions can be monetized. For example, a long-duration power interruption at a hospital that has insufficient backup generation capabilities may increase mortality/morbidity rates. A significant amount of literature documents how changes in mortality/morbidity rates can be monetized using value of statistical life estimates and other approaches (e.g., Executive Office of the President (EOP), Citation2013b). There may be other co-benefits of hardening infrastructure. Larsen’s (Citation2016) cost-benefit analysis of an undergrounding mandate includes avoided aesthetic costs, which are based on improvements in property value that result from removing overhead transmission or distribution lines from a property owner’s line of sight.

2.3.5. Strengths and limitations of methods

As with any economic tools or models, survey methods for valuing direct customer costs of power interruptions and computational modeling for estimating economy-wide direct and indirect impacts both have strengths and limitations. Sullivan et al. (Citation2018) comprehensively describe and discuss the limitations of survey methods and applications for direct cost estimation. Sanstad (Citation2016) provides a conceptual and theoretical overview of computational economic models and their application to estimating the costs of LDWIs, discusses several examples, reviews methodological issues and the models’ advantages and limitations, and identifies research directions for improving them. A March 2018 expert workshop was convened to understand the advantages and limitations of survey methods and computational modeling and to recommend future research areas to improve economic estimates of LDWIs (Larsen et al., Citation2019). At that workshop, a number of leading researchers from across the U.S. reported on computational models, survey data, and methods, and offered ideas for further research. Eyer and Rose et al. (Citation2019) presented a modeling framework for analyzing the economic tradeoffs between resilience investments and post-interruption recovery, and Sue Wing and Rose et al. (Citation2019) developed a simple economic model and applied it to studying an earthquake-caused power interruption in the San Francisco Bay Area in order to study the underlying drivers of cost estimates. Shawhan et al. (Citation2019) discussed survey methods for direct cost estimation, and Baik et al. (Citation2019) described a new methodology for estimating residential customers’ cost of long- (as opposed to short-) duration interruptions. Schellenberg et al. (Citation2019) surveyed the current data landscape for interruption cost estimation and potential avenues for increasing both the quantity and quality of data for this purpose. Perhaps most importantly, Larsen et al. (Citation2019) identified (1) concerns about using existing survey-based techniques to elicit VOLL for power interruptions lasting longer than 24 hours and (2) the challenges with interpreting and including regional economic model output in formal decision-making processes involving resilience.

3. Conclusion and policy implications

In this concluding section, we summarize the rationale for this study and its potential policy implications.

First, there have been a number of regional and national studies of the physical and engineering aspects of electricity system vulnerabilities and catastrophic power interruptions, but few studies on the economics of power interruptions lasting several days up to several weeks. The economic aspects include utilities’ assessments of potential benefits of investments and other measures to reduce system vulnerability and the likelihood of severe electricity service disruptions. These impacts are increasing. Since the year 1900, four of the five costliest tropical cyclones (including hurricanes) to hit the U.S. mainland in the South, Southeast, and East occurred during the past 15 years.Footnote11 In the West, 15 of the 20 most destructive California wildfires of the past century have occurred since 2000 (CAL FIRE Citation2019a; Citationb); in Northern California, the Pacific Gas & Electric company has recently resorted to the unprecedented strategy of ‘public safety power shutoffs’ – de-energizing sections of its distribution system to prevent its equipment from igniting fires under extreme weather conditions.Footnote12

Second, although system resilience has been discussed in many conceptual studies, the meaning of this term and its practical application remain uncertain. As discussed earlier, state regulators make little or no distinction between reliability and resilience in assessing proposed projects, and many of what have been called ‘resilience investments’ are, in practical terms, measures that are already included in utility storm-hardening activities. Because utility proposals for reliability or resilience investments are made and evaluated in regulatory proceedings – and ultimately approved, amended, or disapproved by regulators – the content of regulatory proceedings is important for understanding how reliability and resilience are addressed in practice.

Third, regulators need information to estimate the monetary/financial benefits of such measures, whether they are deemed investments in reliability or resilience. Although we have not discussed it explicitly, this in part reflects the fact that utilities themselves may not have comprehensive and defensible information on the avoided economic impacts of these types of investments.

Fourth, CEA is a common technique for economic evaluation of proposed investments in reliability, but CBA is a more appropriate tool for evaluating investments in infrastructure that will help avoid (or reduce) the economic impact of LDWIs. There are additional benefit categories relevant to investments in resilience, including (1) avoided costs to customers and (2) avoided local or regional economy-wide losses. There may also be other types of indirect societal benefits from these types of investments.

Fifth, although only a few modeling studies have estimated state- or regional-level economic losses from power interruptions, the magnitude of these impacts is extremely large, on the order of tens or hundreds of billions of dollars. It follows that avoiding these significant losses is a critical, yet under-studied, benefit that may help justify additional investments in power system resilience.

These points highlight the policy implications of this work. Utilities and regulators are facing new challenges and complexities in the decision environment for addressing electricity system vulnerabilities to extreme events. Among the most important is assessing the economic aspects of investments to mitigate these vulnerabilities – in particular, their potential benefits in terms of reducing LDWI costs to their customers and to the regional economies in which they operate. In addition, given the emphasis devoted to ‘resilience’ in much research and policy analysis related to LDWIs, it is important to determine the extent to which this concept has pragmatic usefulness for decision-makers. The case studies summarized in the companion Part II paper can improve our understanding of how these issues are being dealt with in actual practice, and will help guide development of data, models, and other tools that can support utilities and regulators in economic analysis to reduce the risks of severe electricity service disruptions.

Acknowledgments

The work described in this paper was funded by the U.S. Department of Energy’s Office of Electricity under Lawrence Berkeley National Laboratory Contract No. DE-AC02-05CH11231. We are grateful to the U.S. Department of Energy for supporting this project and acknowledge the constructive feedback provided by a number of individuals in academia, industry, and government. All errors and omissions are the responsibility of the authors. Any views expressed in this document are those of the authors and do not necessarily represent those of their employers or sponsors.

Disclosure statement

The Coalition for Disaster Resilient Infrastructure (CDRI) reviewed the anonymised abstract of the article, but had no role in the peer review process nor the final editorial decision.

Additional information

Funding

The Article Publishing Charge (APC) for this article is funded by the Coalition for Disaster Resilient Infrastructure (CDRI).

Notes on contributors

Alan H. Sanstad

Alan H. Sanstad is a Staff Scientist in the Energy Technologies Area at the Lawrence Berkeley National Laboratory. He is also an affiliate researcher of the Energy & Resources Group at the University of California, Berkeley and of the NSF-sponsored Center for Robust Decision-Making on Climate and Energy Policy at the University of Chicago. His research interests include decision-making pertaining to energy system transitions, energy-economic modeling, integrated assessment modeling, and greenhouse gas abatement. Dr. Sanstad received M.S. and Ph.D. degrees in Operations Research and an A.B. in Applied Mathematics from the University of California, Berkeley.

B. D. Leibowicz

B. D. Leibowicz is an Associate Professor in the Operations Research and Industrial Engineering graduate program at The University of Texas at Austin. He is also affiliated with the Walker Department of Mechanical Engineering, Lyndon B. Johnson School of Public Affairs (by courtesy), and Energy and Earth Resources graduate program. His applied research interests include energy system modeling, energy and environmental policy analysis, sustainable cities, technological change, and infrastructure systems. Dr. Leibowicz earned M.S. and Ph.D. degrees in Management Science and Engineering from Stanford University and an A.B. in Physics with a minor in Economics from Harvard University.

Q. Zhu

Q. Zhu is currently an Engineer/Scientist II in the Energy Systems and Climate Analysis Group at the Electric Power Research Institute. Her research interests include energy system modeling, energy and environmental policy analysis, and decision-making related to climate resilience. Dr. Zhu obtained M.S. and Ph.D. degrees in Operations Research and Industrial Engineering from The University of Texas at Austin (where she was while working on this paper) and B.S. degrees in Mathematics and Economics from Penn State.

P. H. Larsen

P. H. Larsen is a Staff Scientist and Leader of the Electricity Markets and Policy Department at the Lawrence Berkeley National Laboratory. He is also a Research Fellow at the University of Montana Bureau of Business and Economic Research. His research interests include economic analysis of electricity reliability and resilience, long-term electric utility planning, the energy service company industry, and risks to infrastructure from extreme events. Dr. Larsen earned M.S. and Ph.D. degrees in Management Science and Engineering from Stanford University, an M.S. in Natural Resource Economics from Cornell University, and a B.A. in Economics from the University of Montana at Missoula.

J. H. Eto

J. H. Eto is a Staff Scientist in the Energy Technologies Area at the Lawrence Berkeley National Laboratory. From 1999 to 2020, Mr. Eto led the program office for the Consortium for Electric Reliability Technology Solutions, which was a national laboratory-university-industry R&D consortium that conducted research and analysis on electricity reliability and transmission technologies. His research interests include electricity reliability, transmission planning and operations, demand response, distributed generation, and utility integrated resource planning. Mr. Eto obtained an M.S. in Energy and Resources and an A.B. in Philosophy of Science from the University of California, Berkeley.

Notes

1. These papers are based on a longer report by Lawrence Berkeley National Laboratory (Sanstad et al., Citation2020).

2. National Association of Regulatory Utility Commissions and Converge Strategies LLC (NARUC/Converge; Citation2021) provides a review of the ‘black sky’ literature.

3. The U.S. Department of Defense has a number of energy resilience initiatives under way (Rickerson et al., Citation2018).

4. ‘Standards’ refer not only to design standards for physical infrastructure, but also standards for preparedness, such as vegetation management and staffing requirements for emergency response.

5. In some cases this can result in physical damage to facilities, caused by the interruption rather than the precipitating event. An example is hardening of molten aluminum in pots as a result of loss of power. We thank Carl Pechman for pointing this out.

6. Normalization may be cost per individual interruption for an average customer, cost per average kilowatt, or per unserved kilowatt-hour (Sullivan et al., Citation2015).

7. U.S. Department of Energy (DOE; Citation2017) provides a more comprehensive list of citations.

8. Larsen et al. (Citation2018) use this framework to project potential future interruption costs to U.S. electric utility customers.

9. We also note that there is a technical engineering literature on optimal determination of resilience investments in power systems, such as Fang and Sansavini (Citation2017) and Pierre et al. (Citation2018).

10. This subsection draws upon Sanstad (Citation2016).

11. Measured in inflation-adjusted dollars – see, National Hurricane Center (NHC; Citation2018).

12. It is also important to point out that the circumstances surrounding some recent large power disruptions – including Hurricane Harvey in Texas and several wildfires in California – have entailed deaths and extensive property damage and loss. Although these may not have been direct consequences of the power interruptions themselves, the information in this report can contribute to efforts to prevent such tragic impacts of future extreme events on both energy and non-energy infrastructure and on human systems more generally.

References

  • Anderson, K., Li, X., Dalvi, S., Ericson, S., Barrows, C., Murphy, C., & Hotchkiss, E. (2019). Integrating the value of electricity resilience in energy planning and operations decisions. IEEE Systems Journal, DOI https://doi.org/10.1109/JSYST.2019.2961298.
  • Baik, S., Sirinterlikci, S., Park, J. W., Davis, A., Morgan, M. G. , et al. (2019). Estimating residential customers’ costs of large, long duration blackouts. In P.H. Larsen, Sanstad, A.H., LaCommare, K.H., Eto, J.H. (Eds.), Frontiers in the Economics of Widespread Long-Duration Power Interruptions. Berkeley, CA: Lawrence Berkeley National Laboratory.
  • Boero, R., & Edwards, B. (2017). Hurricane sandy economics impact assessment: A computable general equilibrium approach and validation. Los Alamos National Laboratory Report LA-UR-17-27053. Los Alamos, NM: Los Alamos National Laboratory.
  • California Department of Forestry and Fire Protection (CAL FIRE). (2019a). Top 20 Largest California Wildfires. Table.
  • California Department of Forestry and Fire Protection (CAL FIRE). (2019b.) Top 20 Most Destructive California Wildfires. Table.
  • Campbell, R. (2012). Weather-related power outages and electric system resiliency. (Report R42696). U.S. Congressional Research Service.
  • Costello, K. (2018). Challenges surrounding electric power resiliency. Public Utilities Fortnightly, 84–91. https://www.fortnightly.com/fortnightly/2018/04/challenges-surrounding-electric-power-resiliency
  • Executive Office of the President (EOP). (2013b). Draft report to congress on the benefits and costs of federal regulations. The White House.
  • Executive Office of the President of the United States. (EOP). (2013a). Economic benefits of increasing electric grid resilience to weather outages. The White House.
  • Eyer, J., Rose, A. (2019). Mitigation and resilience tradeoffs in electricity outages. In P. H. Larsen, Sanstad, A.H., LaCommare, K.H., Eto, J.H. (Eds.), Frontiers in the economics of widespread long-duration power interruptions. Berkeley, CA: Lawrence Berkeley National Laboratory.
  • Fang, Y., & Sansavini, G. (2017). Optimizing power system investments and resilience against attacks. Reliability Engineering and System Safety, 159, 161–173. https://doi.org/10.1016/j.ress.2016.10.028
  • Finster, M., Philips, J., & Wallace, K. (2016). Front-line resilience perspectives: The electric grid. Global Security Sciences Division, Argonne National Laboratory, Report ANL/GSS-16/2.
  • Greenberg, M., Mantell, N., Lahr, M., Felder, F., & Zimmerman, R. (2007). Short and intermediate economic impacts of a terrorist-initiated loss of electric power: Case study of New Jersey. Energy Policy, 35(1), 722–733. https://doi.org/10.1016/j.enpol.2006.01.017
  • Grid Modernization Laboratory Consortium (GMLC). (2017). Grid modernization: Metrics analysis (GMLC 1.1). Pacific Northwest National Laboratory Report 26541. https://gridmod.labworks.org/sites/default/files/resources/GMLC1%201_Reference_Manual_2%201_final_2017_06_01_v4_wPNNLNo_1.pdf
  • Institute of Electrical and Electronics Engineers (IEEE). (2020). Resilience framework, methods, and metrics for the electricity sector. IEEE Power & Energy Society Technical Report PES-TR83.
  • Keogh, M., & Cody, C. (2013). Resilience in Regulated Utilities. The National Association of Regulatory Utility Commissioners (NARUC). http://www.ncsl.org/Portals/1/Documents/forum/Forum_2014/ResilienceRegulatedUtilities.pdf
  • Kwasinski, A. (2016). Quantitative model and metrics of electrical grids’ resilience evaluated at a power distribution level. Energies, 9(2), 93. https://doi.org/10.3390/en9020093
  • LaCommare, K., & Eto, J. (2004). Understanding the cost of power interruptions to US electricity consumers. Lawrence Berkeley National Laboratory.
  • LaCommare, K., & Eto, J. (2006). Cost of power interruptions to electricity consumers in the United States (US). Energy, 31(12), 1845–1855. https://doi.org/10.1016/j.energy.2006.02.008
  • LaCommare, K. H., Eto, J. H., Dunn, L. N., & Sohn, M. D. (2018). Improving the estimated cost of sustained power interruptions to electricity customers. Energy, 153, 1038–1047. https://doi.org/10.1016/j.energy.2018.04.082
  • LaCommare, K. H., Larsen, P., & Eto, J. (2017). Evaluating proposed investments in power system reliability and resilience: Preliminary results from interviews with public utility commission staff. Lawrence Berkeley National Laboratory, January.
  • Larsen, P. H. (2016). A method to estimate the costs and benefits of undergrounding electricity transmission and distribution lines. Energy Economics, 60, 47–61. http://dx.doi.org/10.1016/j.eneco.2016.09.011
  • Larsen, P., B. Boehlert, J. Eto, K. Hamachi-LaCommare, J. Martinich, and L. Rennels. 2018. Projecting future costs to U.S. electric utility customers from power interruptions. Energy, 147, 1256–1277. https://doi.org/10.1016/j.energy.2017.12.081
  • Larsen, P., Sanstad, A. H., LaCommare, K. H., & Eto, J. H . 2019). Frontiers in the economics of widespread long-duration power interruptions. Berkeley, CA: Lawrence Berkeley National Laboratory. https://eta-publications.lbl.gov/sites/default/files/long_duration_interruptions_workshop_proceedings.pdf
  • Maliszewski, P. J., & Perrings, C. (2012). Factors in the resilience of electrical power distribution infrastructures. Applied Geography, 32(2), 668–679. https://doi.org/10.1016/j.apgeog.2011.08.001
  • Morrissey, K., Plater, A., & Dean, M. (2018). The cost of electric power outages in the residential sector: A willingness to pay approach. Applied Energy, 212, 141–150. https://doi.org/10.1016/j.apenergy.2017.12.007
  • National Academies of Sciences (NAS), Engineering, and Medicine. (2017). Enhancing the resilience of the Nation’s electricity system. Washington, DC: National Academies Press. https://doi.org/10.17226/24836
  • National Association of Regulatory Utility Commissions and Converge Strategies LLC (NARUC/Converge). (2019). The value of resilience for distributed energy resources: An overview of current analytical practices.
  • National Association of Regulatory Utility Commissions and Converge Strategies LLC (NARUC/Converge). (2021). Regulatory considerations for utility investments in defense energy resilience.
  • NHC. January. 2018. Costliest U.S. tropical cyclones tables updated. Miami, FL: U.S. National Hurricane Center (NHC).
  • Pierre, B. J., Arguello, B., Staid, A., & Guttromson, R. T. Investment optimization to improve power system resilience. 2018 IEEE International Conference on Probabilistic Methods Applied to Power Systems (PMAPS), IEEE.
  • The President’s National Infrastructure Advisory Council (NIAC). (2018). Surviving a Catastrophic Power Outage. https://www.dhs.gov/sites/default/files/publications/NIAC%20Catastrophic%20Power%20Outage%20Study_508%20FINAL.pdf
  • Primen/EPRI. (2001). The cost of power disturbances to industrial and digital economy companies.
  • Richter, -L.-L., & Weeks, M. (2016). Flexible mixed logit with posterior analysis: exploring willingness-to-pay for grid resilience. Cambridge Working Papers in Economics 1631, Faculty of Economics, University of Cambridge.
  • Rickerson, W., Wu, M., & Pringle, M. (2018). Beyond the Fence Line: Strengthening military capabilities through energy resilience partnerships. Washington, DC: Association of Defense Communities.
  • Rosales-Asensio, E., de Simon-Martin, M., Rosales, A. E., & Colmenar-Santos, A. (2021). Solar-plus-storage benefits for end-users placed at radial and meshed grids: An economic and resiliency analysis. International Journal of Electrical Power and Energy Systems, 128, 106675. https://doi.org/10.1016/j.ijepes.2020.106675
  • Rose, A., Benavides, J., Chang, S. E., Szczesniak, P., & Lim, D. (1997). The regional economic impact of an earthquake: direct and indirect effects of electricity lifeline disruptions. Journal of Regional Science, 37(3), 437–458. https://doi.org/10.1111/0022-4146.00063
  • Rose, A., Oladosu, G., & Salvino, D. 2005). Economic impacts of electricity outages in Los Angeles: The importance of resilience and general equilibrium effects. In M. A. Crew & M. Spiegel (Eds.), Obtaining the best from regulation and competition, Vol.47. (pp. 179-211). Springer Science.
  • Sagebiel, J. (2017). Preference heterogeneity in energy discrete choice experiments: A review on methods for model selection. Renewable and Sustainable Energy Reviews, 69, 804–811. https://doi.org/10.1016/j.rser.2016.11.138
  • Sanstad, A. H. (2016). Regional economic modeling of electricity supply disruptions: A review and recommendations for research; Lawrence Berkeley National Laboratory Report LBNL-1004426.
  • Sanstad, A. H., Zhu, Q., Leibowicz, B. D., Larsen, P. H., & Eto, J. H. (2020). Case studies of the economic impacts of power interruptions and damage to electricity system infrastructure from extreme events. Lawrence Berkeley National Laboratory report.
  • Schellenberg, J., Collins, M., Sullivan, M., Hees, S., Bieler, S. (2019). Data landscape: Challenges and opportunities. In P. H. Larsen, Sanstad, A.H., LaCommare, K.H., Eto, J.H. (Eds,). Frontiers in the economics of widespread long-duration power interruptions. Berkeley, CA: Lawrence Berkeley National Laboratory.
  • Shawhan, D. 2019). Using stated preferences to estimate the value of avoiding power outages: A commentary with input from six continents. In P. H. Larsen, Sanstad, A.H., LaCommare, K.H., Eto, J.H. (Eds.), Frontiers in the economics of widespread long-duration power interruptions. Berkeley, CA: Lawrence Berkeley National Laboratory.
  • Shen, L., Tang, Y., & Tang, L. (2021). Understanding key factors affecting power systems resilience. Reliability Engineering and System Safety, 212, 107621. https://doi.org/10.1016/j.ress.2021.107621
  • Stockton, P. (2014). Resilience for Black Sky Days. The National Association of Regulatory Utility Commissioners (NARUC).
  • Sue Wing, I., Rose, A. Z. (2019). Economic consequences of electric power infrastructure disruptions: an analytical general equilibrium model. In P. H. Larsen, Sanstad, A.H., LaCommare, K.H., Eto, J.H., (Eds.), Frontiers in the economics of widespread long-duration power interruptions. Berkeley, CA: Lawrence Berkeley National Laboratory.
  • Sue Wing, I., & Rose, A. Z. (2020). Economic consequence analysis of electric power infrastructure disruptions: General equilibrium approaches. Energy Economics, 89, 104756. https://doi.org/10.1016/j.eneco.2020.104756
  • Sullivan, M., Collins, M. T., Schellenberg, J., & Larsen, P. H. (2018). Estimating power system interruption costs - A guidebook for electric utilities. Lawrence Berkeley National Laboratory.
  • Sullivan, M., Mercurio, J. M., & Schellenberg, J. (2009). Estimated values of service reliability for electric utility customers in the United States. Berkeley, CA: Lawrence Berkeley National Laboratory Report LBNL-2132E.
  • Sullivan, M., Schellenberg, J., & Blundell, M. (2015). Updated value of service reliability estimates for electric utility customers in the United States. Berkeley, CA: Lawrence Berkeley National Laboratory Report LBNL-6941E.
  • Swaminathan, S., & Sen, R. (1998). Review of power quality applications of energy storage systems. Sandia National Laboratory.
  • United States Government Accountability Office (GAO). (2014). Climate change – energy infrastructure risks and adaptation efforts. Report to Congressional Requestors, GAO-14-74.
  • U. S. Department of Energy (DOE). (2010). Hardening and Resiliency U.S. Energy Industry Response to Recent Hurricane Seasons. https://www.oe.netl.doe.gov/docs/HR-Report-final-081710.pdf
  • U. S. Department of Energy (DOE). (2013). Comparing the Impacts of Northeast Hurricanes on Energy Infrastructure. https://www.energy.gov/sites/prod/files/2013/04/f0/Northeast%20Storm%20Comparison_FINAL_041513b.pdf
  • U. S. Department of Energy (DOE). (2015). Climate Change and the U.S. Energy Sector: Regional Vulnerabilities and Resilience Solutions. http://energy.gov/sites/prod/files/2015/10/f27/Regional_Climate_Vulnerabilities_and_Resilience_Solutions_0.pdf
  • U. S. Department of Energy (DOE). (2016). Climate Change and the U.S. Energy Sector: Guide for Climate Change Resilience Planning. U.S. Department of Energy. https://www.energy.gov/sites/prod/files/2016/10/f33/Climate%20Change%20and%20the%20Electricity%20Sector%20Guide%20for%20Climate%20Change%20Resilience%20Planning%20September%202016_0.pdf
  • U.S. Department of Energy (DOE). (2017). Transforming the Nation’s Electricity Sector: The Second Installment of the Quadrennial Energy Review (QER). https://www.energy.gov/sites/prod/files/2017/01/f34/Chapter%20IV%20Ensuring%20Electricity%20System%20Reliability%2C%20Security%2C%20and%20Resilience.pdf
  • U. S. Global Climate Research Program (USGCRP). (2017). Climate science special report: fourth national climate assessment, Volume I; D. J. Wuebbles, D. W. Fahey, K. A. Hibbard, D. J. Dokken, B. C. Stewart, & T. K. Maycock, Eds., pp. 470. U.S. Global ChangeResearch Program.
  • Zamuda, C., Larsen, P., Collins, M., Bieler, S., Bieler, S., & Hees, S. (2019). Monetization methods for evaluating investments in electricity system resilience to extreme weather and climate change. The Electricity Journal, 32(9), 106641. https://doi.org/10.1016/j.tej.2019.106641