Efficient Robust Regression via Two-Stage Generalized Empirical Likelihood

Abstract

Large- and finite-sample efficiency and resistance to outliers are the key goals of robust statistics. Although often not simultaneously attainable, we develop and study a linear regression estimator that comes close. Efficiency is obtained from the estimator's close connection to generalized empirical likelihood, and its favorable robustness properties are obtained by constraining the associated sum of (weighted) squared residuals. We prove maximum attainable finite-sample replacement breakdown point and full asymptotic efficiency for normal errors. Simulation evidence shows that compared to existing robust regression estimators, the new estimator has relatively high efficiency for small sample sizes and comparable outlier resistance. The estimator is further illustrated and compared to existing methods via application to a real dataset with purported outliers.

KEY WORDS:

• Asymptotic efficiency
• Breakdown point
• Consistency
• Constrained optimization
• Distributional robustness
• Efficient estimation
• Exponential tilting
• Least trimmed squares
• Weighted least squares

Acknowledgments

The authors are grateful to the editor, an associate editor, and three anonymous referees for their valuable comments. The authors’ research was partially supported by NSF grant DMS 1005612 and NIH grants P01-CA-142538, R01-MH-084022 (Bondell); and NSF grant DMS 0906421 and NIH grants R01-CA-085848, P01-CA-142538 (Stefanski).