Response to comments by Gurney et al. regarding “Carbon dioxide emission tallies for 210 U.S. coal-fired power plants: A comparison of two accounting methods”

Dear Editor,

I appreciate the opportunity to respond to Gurney and colleagues (Gurney et al., 2014) who disagree with my conclusion (Quick, 2014) that the annual CO₂ emission tallies for 210 U.S. coal-fired power plants were more accurately calculated from fuel consumption and quality measurements (U.S. Energy Information Administration [EIA], 2012) than from flue gas volume and CO₂ concentration measurements (Clean Air Markets Division [CAMD], 2012).

On the Use of Attenuation Bias to Identify Measurement Error

Examination of figure 3 of Quick (2014) unequivocally shows attenuation bias (diminished slope and standard error) where CAMD CO₂ emission rates are the independent variables and the EIA CO₂ emission rates are the dependent variables in an ordinary least squares regression. The significance of this observation should be noncontroversial; it is well known that attenuation bias (also called regression dilution) is caused by random measurement error associated with the independent variable. However, Gurney et al. (2014) suggest that in this instance the conventional explanation does not apply and argue:

In this case, the change in slope and standard error are nothing more than a necessary outcome of two data sets in which one has more variability. The greater variability, in and of itself, is not indicative of more measurement error, as we have no knowledge of the true variation.

Gurney and colleagues correctly observe (and even illustrate in their figure 3) that the cause of the attenuation bias is the greater variation of the CAMD emission rates, but claim that this observation is not diagnostic of greater CAMD measurement error because we lack knowledge of the true variation of CO₂ emissions from these power plants. However, figure 1 of Quick (2014) shows that the differences between the EIA and CAMD CO₂ emission measurements are normally distributed around a mean near zero, which is clear evidence of random measurement error. Importantly, there is no evidence of systematic error, which would be indicated if these differences were not normally distributed about a mean near zero (Hutcheon et al., 2010). Moreover, Bland-Altman plots (figures 3S and 4S of Quick, 2014) also showed that these differences are consistent with random rather than systematic measurement error. So we do have knowledge of why these CO₂ emission tallies differ; the differences are largely the result of random error (measurement precision) rather than systematic error (measurement accuracy). Consequently, as illustrated in Figure 1, the greater variability of the CAMD emission rates is a necessary and inevitable outcome of greater random CAMD measurement error. As explained by Hutcheon et al. (2010) in their paper, instructively entitled “Random Measurement Error and Regression Dilution Bias,” random measurement error associated with the independent variable causes attenuation bias in regression models.

Figure 1. Illustration showing that poor measurement precision due to random measurement error inevitably increases the range of measurement variation. (a) Measurements with good accuracy and good precision show a small range of variation. (b) Measurements with good accuracy but poor precision show a large range of variation.

On the Transformation of Power Plant Emissions (tons CO₂) to Emission Rates (lbs CO₂/MWhr)

Gurney et al. (2014) correctly observe that I divided the annual CO₂ emission tallies for each plant (tons CO₂) by their annual gross electric generation (MWhr) to calculate the plant emission rates (lbs CO₂/MWhr). As discussed in the supplemental material for my paper, this transformation allows valid comparison of the data sets. However, they question my calculation because “the gross electricity generation from the CAMD data set is physically related to the CAMD CO₂ emissions amount,” and consequently “transformation by the gross electricity generation from one data set is certainly a poor choice. To demonstrate this, we utilize a physically neutral log transformation.”

Gurney and colleagues point out that the gross electric generation (MWhr) is physically related to the CAMD CO₂ emissions, but fail to mention that the MWhr values are also physically related to the EIA CO₂ emissions (tons CO₂ are proportional to MWhr in both data sets). They also fail to recognize that the gross electric generation is an independent measurement, which has no influence on either the EIA emission tallies or the CAMD emission tallies. That the gross generation is reported with the CAMD data, but not the EIA data, is irrelevant. The same results are obtained if EIA and CAMD data are transformed using the net electric generation values that are reported with the EIA data, but not the CAMD data (see supplemental material). Gurney and colleagues also suggest that I should have chosen to use a log transformation. However, the suggested log transformation is not useful for the purpose of my study, which was to determine why the EIA and CAMD CO₂ emission tallies differ. Comparison of Figure 2 with figure 1 of Quick (2014) shows that the differences between the EIA and CAMD CO₂ emission rates are the same as the differences between the corresponding EIA and CAMD CO₂ emission tallies; indeed, the underlying data for these figures are identical (see supplemental material). Conversely, comparison of Figure 3 with figure 1 of Quick (2014) shows that the differences between the log-transformed CO₂ emission tallies are not the same as the differences between the corresponding EIA and CAMD CO₂ emission tallies. Consequently, the log-transformed values are not useful proxies to determine why the EIA and CAMD CO₂ emission tallies differ.

Figure 2. Annual CO₂ emission rates calculated from flue gas volume and CO₂ concentration (CAMD data) and fuel consumption and quality (EIA data) for 210 US power plants differ by ±10.8%. This distribution is identical to that shown in figure 1 of Quick (2014) for the corresponding EIA and CAMD CO₂ emission tallies.

Figure 3. The log of the annual CO₂ emission tallies calculated from the EIA and CAMD data differ by ±0.7%. The distribution for these log-transformed values is much different from that shown in figure 1 of Quick (2014) for the corresponding EIA and CAMD CO₂ emission tallies.

Uncertainty and Propagation of Errors

Finally, Gurney et al. (2014) question the uncertainty values that I used in a propagation of error analysis to show an average ±1.6% minimum error associated with the EIA CO₂ emission tallies, and argue:

Although a number of potential uncertainty sources are described, and the nominal uncertainties itemized, there are no independent data to test whether or not these nominal uncertainties are in fact achieved.

However, the EIA and CAMD data originate from independent measurements collected by different agencies. So we do have independent data to test whether the ±1.6% EIA measurement error that I calculated using these uncertainty values was achieved. Conservatively rounding the minimum EIA error up to ±2%, and knowing that the ±10.8% difference between the EIA and CAMD CO₂ emission tallies is due to random error, the equation

1

shows that a ±2% EIA measurement error requires a ±10.6% CAMD measurement error. These errors are consistent with figure 3 of Quick (2014), which also shows a large CAMD measurement error. So the concerns raised by Gurney et al. (2014) are misplaced; we should be more concerned about the uncertainty of the CAMD flue gas measurements than the EIA fuel measurements.

Supplemental Material

Supplemental data for this article can be accessed on the publisher’s website.