Resistant and Test-Based Outlier Rejection: Effects on Gaussian One- and Two-Sample inference

Abstract

Resistant and sequential test-based procedures for the detection of multiple outliers are compared in Gaussian one- and two-sample problems. Subsequent to statistical rejection of putative outliers, operating characteristics of hypothesis tests applied to samples that are outlier-free may differ from their nominal values. Outlier rejection rules may be calibrated, however, to control the rate of erroneous outlier labeling in pure Gaussian samples. The small-sample behavior of resistant location and scale estimators used in outlier detection is examined through simulation, yielding new formulas for calibration of boxplot-based and shorth-based outlier-rejection rules. Effects of different outlier criteria and calibration rubrics on postrejection hypothesis tests are compared in a variety of contamination settings and real-world examples. Tests conducted subsequent to calibrated rejection have approximately nominal operating characteristics whether or not outliers are present. The selection of an outlier criterion can have an impact on inference and thus becomes a potential component of model uncertainty. Properly calibrated tools should be widely available so that this component of uncertainty becomes better understood; we describe public-domain software implementing a variety of detection techniques.

KEY WORDS:

• Extreme studentized deviate
• Model uncertainty
• Outlier
• Resistant statistic
• Shorth