Estimation of analysis and forecast error variances

Abstract

Accurate estimates of error variances in numerical analyses and forecasts (i.e. difference between analysis or forecast fields and nature on the resolved scales) are critical for the evaluation of forecasting systems, the tuning of data assimilation (DA) systems and the proper initialisation of ensemble forecasts. Errors in observations and the difficulty in their estimation, the fact that estimates of analysis errors derived via DA schemes, are influenced by the same assumptions as those used to create the analysis fields themselves, and the presumed but unknown correlation between analysis and forecast errors make the problem difficult. In this paper, an approach is introduced for the unbiased estimation of analysis and forecast errors. The method is independent of any assumption or tuning parameter used in DA schemes. The method combines information from differences between forecast and analysis fields (‘perceived forecast errors’) with prior knowledge regarding the time evolution of (1) forecast error variance and (2) correlation between errors in analyses and forecasts. The quality of the error estimates, given the validity of the prior relationships, depends on the sample size of independent measurements of perceived errors. In a simulated forecast environment, the method is demonstrated to reproduce the true analysis and forecast error within predicted error bounds. The method is then applied to forecasts from four leading numerical weather prediction centres to assess the performance of their corresponding DA and modelling systems. Error variance estimates are qualitatively consistent with earlier studies regarding the performance of the forecast systems compared. The estimated correlation between forecast and analysis errors is found to be a useful diagnostic of the performance of observing and DA systems. In case of significant model-related errors, a methodology to decompose initial value and model-related forecast errors is also proposed and successfully demonstrated.

• uncertainty of analysis
• forecast verification
• estimation methods
• data assimilation
• ensemble forecasts

6. Acknowledgements

Yuejian Zhu provided the hemispheric analysis and forecast data used in this study. Discussions with Yuejian Zhu, Mozheng Wei, Dingchen Hou and Roman Krzysztofowicz are gratefully acknowledged. We acknowledge Eugenia Kalnay and two other anonymous reviewers who provided valuable comments that improved the original manuscript.

Notes

¹ ρ _i could be estimated through Observing System Simulation Experiments (OSSE). OSSEs, however, are expensive, would need to be repeated as data assimilation and numerical modelling systems evolve, and are based on various assumptions that may introduce unknown errors into the estimations.