Modified minimum covariance determinant estimator and its application to outlier detection of chemical process data

Abstract

To overcome the main flaw of minimum covariance determinant (MCD) estimator, i.e. difficulty to determine its main parameter h, a modified-MCD (M-MCD) algorithm is proposed. In M-MCD, the self-adaptive iteration is proposed to minimize the deflection between the standard deviation of robust mahalanobis distance square, which is calculated by MCD with the parameter h based on the sample, and the standard deviation of theoretical mahalanobis distance square by adjusting the parameter h of MCD. Thus, the optimal parameter h of M-MCD is determined when the minimum deflection is obtained. The results of convergence analysis demonstrate that M-MCD has good convergence property. Further, M-MCD and MCD were applied to detect outliers for two typical data and chemical process data, respectively. The results show that M-MCD can get the optimal parameter h by using the self-adaptive iteration and thus its performances of outlier detection are better than MCD.

Keywords:

• outlier detection
• robust mahalanobis distance
• minimum covariance determinant
• chi-squared distribution
• chemical process

Acknowledgements

The authors gratefully acknowledge the financial support from the following foundations: National Natural Science Foundation of China (20776042), National High-Tech Research and Development Program of China (863 Program: 2007AA04Z164), Doctoral Fund of Ministry of Education of China (20090074110005) and ‘Shu Guang’ project supported by Shanghai Municipal Education Commission and Shanghai Education Development Foundation.