Online monitoring of high-dimensional binary data streams with application to extreme weather surveillance

Abstract

With the rapid development of modern sensor technology, high-dimensional data streams appear frequently nowadays, bringing urgent needs for effective statistical process control (SPC) tools. In such a context, the online monitoring problem of high-dimensional and correlated binary data streams is becoming very important. Conventional SPC methods for monitoring multivariate binary processes may fail when facing high-dimensional applications due to high computational complexity and the lack of efficiency. In this paper, motivated by an application in extreme weather surveillance, we propose a novel pairwise approach that considers the most informative pairwise correlation between any two data streams. The information is then integrated into an exponential weighted moving average (EWMA) charting scheme to monitor abnormal mean changes in high-dimensional binary data streams. Extensive simulation study together with a real-data analysis demonstrates the efficiency and applicability of the proposed control chart.

Keywords:

• High-dimensional monitoring
• pairwise correlation
• binary data streams
• EWMA
• thresholding

Acknowledgments

The authors sincerely acknowledge the efforts of the Editor, the Associate Editor and three anonymous referees that have resulted in significant improvements of this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This paper is partly supported by National Natural Science Foundation of China [grant number 12071144], [grant number 71931004], [grant number 11871324], [grant number 11771145], [grant number 71931006], [grant number 11801210], RGC GRF [grant number 16201718], [grant number 16216119], National Science Foundation of Shanghai [grant number 19ZR1414400], China Postdoctoral Science Foundation [grant number 2020M671064], National Bureau of Statistics of China [grant number 2020LD03], and the Fundamental Research Funds for the Central Universities.