Robust Variable Selection

Abstract

The prevalence of extreme outliers in many regression data sets has led to the development of robust methods that can handle these observations. While much attention has been placed on the problem of estimating regression coefficients in the presence of outliers, few methods address variable selection. We develop and study robust versions of the forward selection algorithm, one of the most popular standard variable selection techniques. Specifically we modify the VAMS procedure, a version of forward selection tuned to control the false selection rate, to simultaneously select variables and eliminate outliers. In an alternative approach, robust versions of the forward selection algorithm are developed using the robust forward addition sequence associated with the generalized score statistic. Combining the robust forward addition sequence with robust versions of BIC and the VAMS procedure, a final model is obtained. Monte Carlo simulation compares these robust methods to current robust methods like the LSA and LAD-LASSO. Further simulation investigates the relationship between the breakdown point of the estimation methods central to each procedure and the breakdown point of the final variable selection method.