Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods

References

• Anderson, J. 2001. An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev. 142, 2884–2903.
   Web of Science ®Google Scholar
• Bellsky, T., Berwald, J. and Mitchell, L. 2014. Nonglobal parameter estimation using local ensemble Kalman filtering. Mon. Wea. Rev. 142, 2150–2164.
   Web of Science ®Google Scholar
• Bishop, C. 2006. Pattern Recognition and Machine Learning Springer.
   Google Scholar
• Briers, M., Doucet, A. and Maskell, S. 2010. Smoothing algorithms for state-spacemodels. Ann. Inst. Stat. Math. 62, 61–89.
   Web of Science ®Google Scholar
• Cappé, O., Moulines, E. and Rydén, T. 2005. Inference in Hidden Markov Models. Springer, New York, NY.
   Google Scholar
• Carrassi, A., Bocquet, M., Hannart, A. and Ghil, M. 2017. Estimating model evidence using data assimilation. Q. J. R. Meteorol. Soc. 143, 866–880.
   Web of Science ®Google Scholar
• Carrassi, A. and Vannitsem, S. 2011. State and parameter estimation with the extended Kalman filter: an alternative formulation of the model error dynamics. Q. J. R. Meteorol. Soc. 137, 435–451.
   Web of Science ®Google Scholar
• Christensen, H., Moroz, I. M. and Palmer, T. N. 2015. Stochastic and perturbed parameter representations of model uncertainty in convection parameterization. J. Atmos. Sci. 72, 2525–2544.
   Web of Science ®Google Scholar
• Cosme, E., Verron, J., Brasseur, P., Blum, J. and Auroux, D. 2012. Smoothing problems in a Bayesian framework and their linear gaussian solutions. Mon. Weather Rev. 140, 683–695.
   Web of Science ®Google Scholar
• Delsole, T. and Yang, X. 2010. State and parameter estimation in stochastic dynamical models. Physica D 239, 1781–1788.
   Web of Science ®Google Scholar
• Dempster, A., Laird, N. and Rubin, D. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 9, 1–38.
   Google Scholar
• Dreano, D., Tandeo, P., Pulido, M., Ait-El-Fquih, B., Chonavel, T. and co-authors. 2017. Estimation of error covariances in nonlinear state-space models using the expectation maximization algorithm. Q. J. R. Meteorol. Soc. 142, 1877–1885.
   Web of Science ®Google Scholar
• Evensen, G. 1994. Sequential data assimilation with a nonlinear quasi-geostrophic model using monte carlo methods to forecast error statistics. J. Geophys. Res. 99, 10143–10162.
   Web of Science ®Google Scholar
• Evensen, G. 2003. The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn. 53, 343–367.
   Google Scholar
• Ghahramani, Z. and Roweis, S. 1999. Learning nonlinear dynamical systems using an EM algorithm. In: Advances in Neural Information Processing Systems, MIT Press, pp. 431–437.
   Google Scholar
• Hamilton, F., Berry, T., Peixoto, N. and Sauer, T. 2013. Real-time tracking of neuronal network structure using data assimilation. Phys. Rev. E 88, 052715.
   Web of Science ®Google Scholar
• Hamilton, F., Berry, T. and Sauer, T. 2016. Ensemble Kalman filtering without a model. Phys. Rev. X 6, 011021.
   Web of Science ®Google Scholar
• Hannart, A., Carrassi, A., Bocquet, M., Ghil, M., Naveau, P. and co-authors. 2016. Dada: data assimilation for the detection and attribution of weather and climate-related events. Clim. Change 136, 155–174.
   Web of Science ®Google Scholar
• Hansen, J. and Penland, C. 2006. Efficient approximate techniques for integrating stochastic differential equations. Mon. Wea. Rev. 134, 3006–3014.
   Web of Science ®Google Scholar
• Hunt, B., Kostelich, E. J. and Szunyogh, I. 2007. Efficient data assimilation for spatio-temporal chaos: a local ensemble transform Kalman filter. Physica D 77, 437–471.
   Google Scholar
• Jazwinski, A. H. 1970. Stochastic and Filtering Theory. Mathematics in Sciences and Engineering Series, Vol. 64. Academic Press, London and New York, p. 376.
   Google Scholar
• Kalnay, E. 2002. Atmospheric Modeling, Data Assimilation, and Predictability Cambridge University Press, Cambridge.
   Google Scholar
• Katsoulakis, M., Majda, A. and Vlachos, D. 2003. Coarse-grained stochastic processes for microscopic lattice systems. Proc. Nat. Acad. Sci. 100, 782–787.
   Web of Science ®Google Scholar
• Kondrashov, D., Ghil, M. and Shprits, Y. 2011. Lognormal Kalman filter for assimilating phase space density data in the radiation belts. Space Weather 9, 11.
   Web of Science ®Google Scholar
• Lguensat, R., Tandeo, P., Fablet, R., Pulido, M. and Ailliot, P. 2017. The analog ensemble-based data assimilation. Mon. Wea. Rev. 145, 4093–4107.
   Web of Science ®Google Scholar
• Lorenz, E. (1996). Predictability–A Problem Partly Solved. Reading: ECMWF. (pp. 1–18)
   Google Scholar
• Lott, F., Guez, L. and Maury, P. 2012. A stochastic parameterization of nonorographic gravity waves: formalism and impact on the equatorial stratosphere. Geophys. Res. Lett. 39, L06807.
   Web of Science ®Google Scholar
• Majda, A. and Gershgorin, B. 2011. Improving model fidelity and sensitivity for complex systems through empirical information theory. Proc. Nat. Acad. Sci. 100, 10044–10049.
   Web of Science ®Google Scholar
• Mason, P. and Thomson, D. 1992. Stochastic backscatter in large-eddy simulations of boundary layers. J. Fluid Mech. 242, 51–78.
   Web of Science ®Google Scholar
• Neal, R. and Hinton, G. 1999. A View of the EM Algorithm that Justifies Incremental, Sparse and other Variants. Springer, Dordrecht.
   Google Scholar
• Nicolis, N. 2004. Dynamics of model error: the role of unresolved scales revisited. J. Atmos. Sci. 61, 1740–1753.
   Web of Science ®Google Scholar
• Palmer, T. 2001. A nonlinear dynamical perspective on model error: a proposal for non-local stochastic-dynamic parameterization in weather and climate prediction models. Q. J. R. Meteorol. Soc. 127, 279–304.
   Web of Science ®Google Scholar
• Powell, M. 2006. The NEWUOA software for unconstrained optimization without derivatives. In: Large-Scale Nonlinear Optimization. Springer, Boston, MA, pp. 255–297.
   Google Scholar
• Pulido, M. and Rosso, O. 2017. Model selection: using information measures from ordinal symbolic analysis to select model sub-grid scale parameterizations. J. Atmos. Sci. 74, 3253–3269.
   Web of Science ®Google Scholar
• Pulido, M., Scheffler, G., Ruiz, J., Lucini, M. and Tandeo, P. 2016. Estimation of the functional form of subgrid-scale schemes using ensemble-based data assimilation: a simple model experiment. Q. J. R. Meteorol. Soc. 142, 2974–2984.
   Web of Science ®Google Scholar
• Raanes, P. 2016. On the ensemble Rauch-Tung-Striebel smoother and its equivalence to the ensemble Kalman smoother. Q. J. R. Meteorol. Soc. 142, 1259–1264.
   Web of Science ®Google Scholar
• Ruiz, J., Pulido, M. and Miyoshi, T. 2013a. Estimating parameters with ensemble-based data assimilation a review. J. Meteorol. Soc. Jpn. 91, 79–99.
   Web of Science ®Google Scholar
• Ruiz, J., Pulido, M. and Miyoshi, T. 2013b. Estimating parameters with ensemble-based data assimilation parameter covariance treatment. J. Meteorol. Soc. Jpn. 91, 453–469.
   Web of Science ®Google Scholar
• Santitissadeekorn, N. and Jones, C. 2015. Two-stage filtering for joint state-parameter estimation. Mon. Wea. Rev. 143, 2028–2042.
   Web of Science ®Google Scholar
• Shaman, J., Karspeck, A., Yang, W., Tamerius, J. and Lipsitch, M. 2013. Real-time influenza forecasts during the 2012–2013 season. Nat. Commun. 4, 2837.
   Web of Science ®Google Scholar
• Shumway, R. and Stoffer, D. 1982. An approach to time series smoothing and forecasting using the EM algorithm. J. Time Ser. Anal. 3, 253–264.
   Google Scholar
• Shutts, G. 2015. A stochastic convective backscatter scheme for use in ensemble prediction systems. Q. J. R. Meteorol. Soc. 141, 2602–2616.
   Web of Science ®Google Scholar
• Stensrud, D. 2009. Parameterization Schemes: Keys to Understanding Numerical Weather Prediction Models Cambridge University Press, Cambridge.
   Google Scholar
• Tandeo, P., Pulido, M. and Lott, F. 2015. Offline estimation of subgrid-scale orographic parameters using EnKF and maximum likelihood error covariance estimates. Q. J. R. Meteorol. Soc. 141, 383–395.
   Web of Science ®Google Scholar
• van Leeuwen, P. J. 2009. Particle filtering in geophysical systems. Mon. Wea. Rev. 407, 4089–4114.
   Web of Science ®Google Scholar
• Wei, G. and Tanner, M. A. 1990. A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. J. Amer. Stat. Assoc. 85, 699–704.
   Web of Science ®Google Scholar
• West, M. and Liu, J. 2001. Combined parameter and state estimation in simulation-based filtering. In: Sequential Monte Carlo Methods in Practice. Springer, New York, pp. 197–223.
   Google Scholar
• Wikle, C. and Berliner, L. 2007. A Bayesian tutorial for data assimilation. Physica D 230, 1–16.
   Web of Science ®Google Scholar
• Wilks, D. S. 2005. Effects of stochastic parametrizations in the Lorenz 96 system. Q. J. R. Meteorol. Soc. 131, 389–407.
   Web of Science ®Google Scholar
• Wu, C. 1983. On the convergence properties of the EM algorithm. Ann. Stat. 11, 95–103.
   Web of Science ®Google Scholar