A change-point–based control chart for detecting sparse mean changes in high-dimensional heteroscedastic data

Abstract

Because of the “curse of dimensionality,” high-dimensional processes present challenges to traditional multivariate statistical process monitoring (SPM) techniques. In addition, the unknown underlying distribution of and complicated dependency among variables such as heteroscedasticity increase the uncertainty of estimated parameters and decrease the effectiveness of control charts. In addition, the requirement of sufficient reference samples limits the application of traditional charts in high-dimension, low-sample-size scenarios (small n, large p). More difficulties appear when detecting and diagnosing abnormal behaviors caused by a small set of variables (i.e., sparse changes). In this article, we propose two change-point–based control charts to detect sparse shifts in the mean vector of high-dimensional heteroscedastic processes. Our proposed methods can start monitoring when the number of observations is a lot smaller than the dimensionality. The simulation results show that the proposed methods are robust to nonnormality and heteroscedasticity. Two real data examples are used to illustrate the effectiveness of the proposed control charts in high-dimensional applications. The R codes are provided online.

Keywords:

• change-point
• heteroscedasticity
• high-dimensional control chart
• sparse changes
• statistical process monitoring (SPM)

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Zezhong Wang

Zezhong Wang received her doctoral degree from the Department of Systems Engineering, City University of Hong Kong. She is a postdoc researcher at the Hong Kong Industrial Artificial Intelligence and Robotics Centre (FLAIR). Her research interests include statistical process monitoring, data science, and artificial intelligence applications. Her email address is [email protected].

Inez Maria Zwetsloot

Inez M. Zwetsloot is an associate professor in the Department of Business Analytics at the University of Amsterdam, The Netherlands. Previously she was assistant professor at the Department of Systems Engineering, City University of Hong Kong. She is the recipient of the Young Statistician Award (ENBIS, 2021) and the Feigenbaum Medal (ASQ, 2021). Her research focuses on using statistics and analytics for solving business challenges using data. This includes work on statistical process monitoring, network analytics, quality engineering, and data science. Her email address is [email protected].