Outlier detection in general profiles using penalized regression method

Abstract

Profile monitoring is a technique for checking the stability of functional relationships between a response variable and one or more explanatory variables over time. The presence of outliers has seriously adverse effects on the modeling, monitoring, and diagnosis of profile data. This article proposes a new outlier detection procedure from the viewpoint of penalized regression, aiming at identifying any abnormal profile observations from a baseline dataset. Profiles are treated as high-dimension vectors and the model is reformulated into a specific regression model. A group-type regularization is then applied that favors a sparse vector of mean shift parameters. Using the classic hard penalty yields a computationally efficient algorithm that is essentially equivalent to an iterative approach. Appropriately choosing a sole tuning parameter in the proposed procedure enables Type-I error to be controlled and robust detection ability to be delivered. Simulation results show that the proposed method has an outstanding performance in identifying outliers in various situations compared with other existing approaches. This methodology is also extended to the case where within-profile correlations exist.

Keywords:

• High-dimensional inference
• iterative approach
• masking effect
• outlier detection
• profile monitoring
• sparsity
• statistical process control