Estimating correlated observation error statistics using an ensemble transform Kalman filter

References

• Anderson J. L . An ensemble adjustment Kalman filter for data assimilation. Mon. Weather Rev. 2001; 129: 2884–2903.
   Web of Science ®Google Scholar
• Bannister R. N . A review of forecast error covariance statistics in atmospheric variational data assimilation. I: characteristics and measurements of forecast error covariances. Q. J. Roy. Meteorol. Soc. 2008; 134: 1951–1970.
   Web of Science ®Google Scholar
• Bickel P. J. , Levina E . Regularized estimation of large covariance matrices. Ann. Stat. 2008; 36: 199–227.
   Web of Science ®Google Scholar
• Bishop C. , Etherton B. , Majumdar S . Adaptive sampling with the ensemble transform Kalman filter. Part I: theoretical aspects. Mon. Weather Rev. 2001; 129: 420–436.
   Web of Science ®Google Scholar
• Bishop C. H. , Hodyss D . Ensemble covariances adaptively localized with ECO-RAP. Part 1: tests on simple error models. Tellus A. 2009; 61: 84–96.
   Google Scholar
• Bormann N. , Bauer P . Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. I: methods and application to ATOVS data. Q. J. Roy. Meteorol. Soc. 2010; 136: 1036–1050.
   Web of Science ®Google Scholar
• Bormann N. , Collard A. , Bauer P . Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. II: application to AIRS and IASI data. Q. J. Roy. Meteorol. Soc. 2010; 136: 1051–1063.
   Web of Science ®Google Scholar
• Bormann N. , Saariene S. , Kelly G. , Thepaut J . The spatial structure of observation errors in atmospheric motion vectors from geostationary satellite data. Mon. Weather Rev. 2002; 131: 706–718.
   Web of Science ®Google Scholar
• Buehner M . Lahoz W. , Khattatov B. , Mènard R . Error statistics in data assimilation: estimation and modelling. Data Assimilation Making Sense of Observations .
   Google Scholar
• Buehner M. , Houtekamer P. , Charette C. , Mitchell H. , He B . Intercomparison of variational data assimilation and the ensemble Kalman filter for global deterministic NWP. Part II: one-month experiments with real observations. Mon. Weather Rev. 2010; 138: 1567–1586.
   Web of Science ®Google Scholar
• Clayton A. M. , Lorenc A. C. , Barker D. M . Operational implementation of a hybrid ensemble/4D-Var global data assimilation system at the Met Office. Q. J. Roy. Meteorol. Soc. 2013; 139(675): 1445–1461.
   Web of Science ®Google Scholar
• Cox S. M. , Matthews P. C . Exponential time differencing for stiff systems. J. Comput. Phys. 2000; 176: 430–455.
   Web of Science ®Google Scholar
• Daley R . Atmospheric Data Analysis. 1991; Cambridge: Cambridge University Press.
   Google Scholar
• Desroziers G. , Berre L. , Chapnik B . Objective validation of data assimilation systems: diagnosing sub-optimality. Proceedings of ECMWF Workshop on Diagnostics of Data Assimilation System Performance. 2009; 15–17. June 2009.
   Google Scholar
• Desroziers G. , Berre L. , Chapnik B. , Poli P . Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. Roy. Meteorol. Soc. 2005; 131: 3385–3396.
   Web of Science ®Google Scholar
• Dormand J. , Prince P . A family of embedded Runge-Kutta formulae. J. Comput. Appl. Math. 1980; 6: 19–26.
   Google Scholar
• Eguíluz V. M. , Alstrøm P. , Hernández-García E. , Piro O . Average patterns of spatiotemporal chaos: a boundary effect. Phys. Rev. E. 1999; 59: 2822–2825.
   Web of Science ®Google Scholar
• Evensen G . The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn. 2003; 53: 343–367.
   Google Scholar
• Fertig E. J. , Harlim J. , Hunt B. R . A comparative study of 4D-VAR and a 4D ensemble Kalman filter: perfect model simulations with Lorenz-96. Tellus A. 2007; 59: 96–100.
   Google Scholar
• Golub G. H. , Van Loan C. F . Matrix Computations. 1996; Baltimore: The Johns Hopkins University Press. 3rd ed.
   Google Scholar
• Gustafsson J. , Protas B . Regularization of the backward-in-time Kuramoto-Sivashinsky equation. J. Comput. Appl. Math. 2010; 234: 398–406.
   Web of Science ®Google Scholar
• Hamill T. M . Interpretation of rank histograms for verifying ensemble forecasts. Mon. Weather Rev. 2000; 129: 550–560.
   Web of Science ®Google Scholar
• Hamill T. M. , Whitaker J. S. , Snyder C . Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon. Weather Rev. 2001; 129: 2776–2790.
   Web of Science ®Google Scholar
• Healy S. B. , White A. A . Use of discrete Fourier transforms in the 1D-Var retrieval problem. Q. J. Roy. Meteorol. Soc. 2005; 131: 63–72.
   Web of Science ®Google Scholar
• Hilton F. , Atkinson N. C. , English S. J. , Eyre J. R . Assimilation of IASI at the Met Office and assessment of its impact through observing system experiments. Q. J. Roy. Meteorol. Soc. 2009; 135: 495–505.
   Web of Science ®Google Scholar
• Hollingsworth A. , Lönnberg P . The statistical structure of short-range forecast errors as determined from radiosonde data. Part I: the wind field. Tellus A. 1986; 38: 111–136.
   Google Scholar
• Houtekamer P. L. , Mitchell H. L . Ensemble Kalman filtering. Q. J. Roy. Meteorol. Soc. 2005; 133: 3260–3289.
   Google Scholar
• Janjic T. , Cohn S. E . Treatment of observation error due to unresolved scales in atmospheric data assimilation. Mon. Weather Rev. 2006; 134: 2900–2915.
   Web of Science ®Google Scholar
• Jardak M. , Navon I. M. , Zupanski M . Comparison of sequential data assimilation methods for the Kuramoto-Sivashinsky equation. Int. J. Numer. Meth. Fluid. 2010; 62: 374–402.
   Web of Science ®Google Scholar
• Kassam A. , Trefethen L . Fourth-order time-stepping for stiff PDEs. SIAM J. Sci. Computing. 2005; 25: 1214–1233.
   Google Scholar
• Kuramoto Y . Diffusion-induced chaos in reaction systems. Prog. Theor. Phys. 1978; 64: 346–367.
   Google Scholar
• Li H. , Kalnay E. , Miyoshi T . Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Q. J. Roy. Meteorol. Soc. 2009; 128: 1367–1386.
   Google Scholar
• Livings D. M. , Dance S. L. , Nichols N. K . Unbiased ensemble square root filters. Phys. D. 2008; 237: 1021–1028.
   Web of Science ®Google Scholar
• Lorenz E . Predictability: a problem partly solved. Predictability. Proceedings of the 1995. ECMWF Seminar. 1996; 1–18.
   Google Scholar
• Lorenz E. , Emanuel K . Optimal sites for supplementary weather observations: simulations with a small model. J. Atmos. Sci. 1998; 55: 399–414.
   Web of Science ®Google Scholar
• Mènard R. , Yang Y. , Rochon Y . Convergence and stability of estimated error variances derived from assimilation residuals in observation space. Proceedings of ECMWF Workshop on Diagnostics of Data Assimilation System Performance. 2009; 15–17. June 2009.
   Google Scholar
• Miyoshi T. , Kalnay E. , Li H . Estimating and including observation-error correlations in data assimilation. Inverse Probl. Sci. Eng. 2013; 21: 387–398.
   Web of Science ®Google Scholar
• Miyoshi T. , Sato Y. , Kadowaki T . Ensemble Kalman filter and 4D-Var intercomparison with the Japanese operational global analysis and prediction system. Mon. Weather Rev. 2010; 138: 2846–2866.
   Web of Science ®Google Scholar
• Ott E. , Hunt B. , Szunyogh I. , Zimin A. , Kostelich E. , co-authors . A local ensemble Kalman filter for atmospheric data assimilation. Tellus A. 2004; 56: 415–428.
   Google Scholar
• Protas B . Adjoint-based optimization of PDE systems with alternative gradients. J. Comput. Phys. 2008; 227: 6490–6510.
   Web of Science ®Google Scholar
• Sivashinsky G . Non-linear analysis of hydrodynamic instability in laminar flames. Acta Astronautica. 1977; 4: 1177–1206.
   Web of Science ®Google Scholar
• Stewart L. M. PhD Thesis, University of Reading. Correlated Observation Errors in Data Assimilation. 2010. Online at: http://www.reading.ac.uk/maths-and-stats/research/theses/maths-phdtheses.aspx.
   Google Scholar
• Stewart L. M., Cameron J., Dance S. L., English S., Eyre J. R., co-authors. Technical report, University of Reading. Mathematics reports series. Observation Error Correlations in IASI Radiance Data. 2009. Online at: http://www.reading.ac.uk/web/FILES/maths/obs_error_IASI_radiance.pdf.
   Google Scholar
• Stewart L. M. , Dance S. L. , Nichols N. K . Correlated observation errors in data assimilation. Int. J. Numer. Meth. Fluid. 2008; 56: 1521–1527.
   Web of Science ®Google Scholar
• Stewart L. M. , Dance S. L. , Nichols N. K . Data assimilation with correlated observation errors: experiments with a 1-D shallow water model. Tellus A. 2013; 65: 19546.
   Google Scholar
• Stewart L. M. , Dance S. L. , Nichols N. K. , Eyre J. R. , Cameron J . Estimating interchannel observation-error correlations for IASI radiance data in the Met Office system. Q. J. Roy. Meteorol. Soc. 2014; 140: 1236–1244.
   Web of Science ®Google Scholar
• Thiebaux H . Anisotropic correlation functions for objective analysis. Mon. Weather Rev. 1976; 104: 994–1002.
   Web of Science ®Google Scholar
• Waller J. A. PhD Thesis, Department of Mathematics and Statistics, University of Reading. Using Observations at Different Spatial Scales in Data Assimilation for Environmental Prediction. 2013. Online at: http://www.reading.ac.uk/maths-and-stats/research/theses/maths-phdtheses.aspx.
   Google Scholar
• Waller J. A. , Dance S. L. , Lawless A. S. , Nichols N. K. , Eyre J. R . Representativity error for temperature and humidity using the Met Office high-resolution model. Q. J. Roy. Meteorol. Soc. 2014; 140: 1189–1197.
   Web of Science ®Google Scholar
• Weston P . Technical report, Met Office, UK. Progress Towards the Implementation of Correlated Observation Errors in 4D-Var. 2011. Forecasting Research Technical Report 560.
   Google Scholar
• Weston P. P. , Bell W. , Eyre J. R . Accounting for correlated error in the assimilation of high resolution sounder data. Q. J. Roy. Meteorol. Soc. 2014. Early View.
   Web of Science ®Google Scholar
• Whitaker J. S. , Hamill T. M. , Wei X. , Song Y. , Toth Z . Ensemble data assimilation with the NCEP global forecast system. Mon. Weather Rev. 2008; 136: 463–482.
   Web of Science ®Google Scholar
• Wilks D. S . Statistical Methods in the Atmospheric Sciences. 1995; San Diego: Academic Press.
   Google Scholar