Please use this identifier to cite or link to this item:
Title: A note on the Lasso and related procedures in model selection
Authors: Leng, C. 
Lin, Y.
Wahba, G.
Keywords: Consistent model selection
Forward Stagewise regression
Variable selection
Issue Date: Oct-2006
Citation: Leng, C.,Lin, Y.,Wahba, G. (2006-10). A note on the Lasso and related procedures in model selection. Statistica Sinica 16 (4) : 1273-1284. ScholarBank@NUS Repository.
Abstract: The Lasso, the Forward Stagewise regression and the Lars are closely related procedures recently proposed for linear regression problems. Each of them can produce sparse models and can be used both for estimation and variable selection. In practical implementations these algorithms are typically tuned to achieve optimal prediction accuracy. We show that, when the prediction accuracy is used as the criterion to choose the tuning parameter, in general these procedures are not consistent in terms of variable selection. That is, the sets of variables selected are not consistently the true set of important variables. In particular, we show that for any sample size n, when there are superfluous variables in the linear regression model and the design matrix is orthogonal, the probability that these procedures correctly identify the true set of important variables is less than a constant (smaller than one) not depending on n. This result is also shown to hold for two-dimensional problems with general correlated design matrices. The results indicate that in problems where the main goal is variable selection, prediction-accuracy-based criteria alone are not sufficient for this purpose. Adjustments will be discussed to make the Lasso and related procedures useful/consistent for variable selection.
Source Title: Statistica Sinica
ISSN: 10170405
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

Google ScholarTM


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.