Dimension Reduction for Outlier Detection Using DOBIN

Abstract

This article introduces DOBIN, a new approach to select a set of basis vectors tailored for outlier detection. DOBIN has a simple mathematical foundation and can be used as a dimension reduction tool for outlier detection tasks. We demonstrate the effectiveness of DOBIN on an extensive data repository, by comparing the performance of outlier detection methods using DOBIN and other bases. We further illustrate the utility of DOBIN as an outlier visualization tool. The R package dobin implements this basis construction. Supplementary materials for this article are available online.

KEYWORDS:

• Basis vectors
• Dimension reduction
• Outlier detection
• Outlier visualization

Supplementary Materials

R package dobin: This package contains the DOBIN basis construction.

Datasets: Datasets discussed in Section 5 are available at Kandanaarachchi et al. (2019). The character network from the novel Les Misérables was taken from the R package SOMbrero (Vialaneix et al. 2019). The book vectors dataset used in Section 4.5 is available at the github repository (https://github.com/sevvandi/Outlier-Basis).

Scripts: The script Figures_For_Paper_1.R contains the R code used to conduct experiments and produce graphs in Section 3. The script Figures_For_Paper_2.R contains the R code used in Section 4. And finally, the script Figures_For_Paper_3.R contains the R code used to produce the graphs in Section 5. The original computation on the data repository detailed in Section 5 was conducted using the MonARCH HPC Cluster. The results of this computation are available at the github repository (https://github.com/sevvandi/Outlier-Basis).

Other R-packages: In addition to the R package dobin we have also used the R packages OutliersO3 (Unwin 2019b), HDoutliers (Fraley 2018), isolationForest (Liu 2009), DMwR (Torgo 2010), pROC (Robin et al. 2011), ggplot2 (Wickham 2016), mbgraphic (Grimm 2019), gridExtra (Auguie 2017), tsutils (Kourentzes 2019), dplyr (Wickham et al. 2019), and tidyr (Wickham and Henry 2020).

Fraley, C. (2018), “HDoutliers: Leland Wilkinson’s Algorithm for Detecting Multidimensional Outliers,” R Package Version 1.0, available at https://CRAN.R-project.org/package=HDoutliers.

Additional information

Funding

Funding was provided by the Australian Research Council through the Linkage Project LP160101885. This research was also supported by the Monash eResearch Centre and eSolutions-Research Support Services through the MonARCH HPC Cluster.