Ensemble clustering in deterministic ensemble Kalman filters

ABSTRACT

Ensemble clustering (EC) can arise in data assimilation with ensemble square root filters (EnSRFs) using non-linear models: an M-member ensemble splits into a single outlier and a cluster of M–1 members. The stochastic Ensemble Kalman Filter does not present this problem. Modifications to the EnSRFs by a periodic resampling of the ensemble through random rotations have been proposed to address it. We introduce a metric to quantify the presence of EC and present evidence to dispel the notion that EC leads to filter failure. Starting from a univariate model, we show that EC is not a permanent but transient phenomenon; it occurs intermittently in non-linear models. We perform a series of data assimilation experiments using a standard EnSRF and a modified EnSRF by a resampling though random rotations. The modified EnSRF thus alleviates issues associated with EC at the cost of traceability of individual ensemble trajectories and cannot use some of algorithms that enhance performance of standard EnSRF. In the non-linear regimes of low-dimensional models, the analysis root mean square error of the standard EnSRF slowly grows with ensemble size if the size is larger than the dimension of the model state. However, we do not observe this problem in a more complex model that uses an ensemble size much smaller than the dimension of the model state, along with inflation and localisation. Overall, we find that transient EC does not handicap the performance of the standard EnSRF.

• non-linear models
• ensemble square root filters
• non-Gaussianity

Acknowledgements

The authors gratefully acknowledge two anonymous reviewers for their positive and constructive comments and suggestions that helped improve the quality of the manuscript. The support of NASA grants NNX07AM97G and NNX08AD40G, DOE grant DEFG0207ER64437, NOAA grant NA09OAR4310178 and ONR grants N000140910418 and N000141010557 are gratefully acknowledged. Craig H. Bishop acknowledges support from the Office of Naval Research grant with project element number 0602435N and document number N0001411WX20871.

Notes

^†Now at: Department of Meteorology, University of Reading, Reading, UK