A regression sequences based method for high dimensional outlier detection

Abstract

In recent years, high dimensional space outlier detection has attracted wide attention. High dimensional space outlier detection plays a very important role in many fields such as network intrusion detection, credit card fraud, electronic commerce crime, medical diagnosis and anti-terrorism. The purpose of high-dimensional outlier detection is to find a small number of objects in a high-dimensional dataset, which has a special behavior or abnormal behavior compared with most of the rest of the objects in the high-dimensional dataset. The existing high dimensional space outlier detection method cannot effectively deal with the problem of uncertain and incomplete data. In this paper, the concept of rough set boundary and the high dimensional space outlier detection method based on regression distance proposed by Knorr et al are combined together. We propose a new method for the definition and detection of high dimensional space outliers based on the regression sequence in the framework of rough set. For this method, we design a corresponding high dimensional space outlier detection algorithm (HDSOD) and verify the validity of the algorithm HDSOD by experiments on high dimensional data set. The experimental results show that the proposed method provides a new method to deal with the uncertain and incomplete data in high dimensional space outlier detection.

Keywords:

• Data mining
• High dimensional space outlier detection
• Rough set
• Uncertain and incomplete data
• Regression sequence