Model Selection and Estimation in Additive Regression Models

Abstract

We propose a method of simultaneous model selection and estimation in additive regression models (ARMs) for independent normal data. We use the mixed model representation of the smoothing spline estimators of the nonparametric functions in ARMs, where the importance of these functions is controlled by treating the inverse of the smoothing parameters as extra variance components. The selection of important nonparametric functions is achieved by maximizing the penalized likelihood with an adaptive LASSO. A unified EM algorithm is provided to obtain the maximum penalized likelihood estimates of the nonparametric functions and the residual variance. In the same framework, we also consider forward selection based on score tests, and a two stage approach that imposes an early stage screening using an individual score test on each induced variance component of the smoothing parameter.

For longitudinal data, we propose to extend the adaptive LASSO and the two-stage selection with score test screening to the additive mixed models (AMMs), by introducing subject-specific random effects to the additive models to accommodate the correlation in responses. We use the eigenvalue-eigenvector decomposition approach to approximate the working random effects in the linear mixed model presentation of the AMMs, so as to reduce the dimensions of matrices involved in the algorithm while keeping most data information, hence to tackle the computational problems caused by large sample sizes in longitudinal data.

Simulation studies are provided and the methods are illustrated with data applications.