Variable Selection via the Sparse Net

Variable selection is an important problem when the model includes many noisy variables. For years, the sparse penalized approaches have been proposed for the problem. Examples are the least absolute selection and shrinkage operator (LASSO) and smoothly clipped absolute deviation penalty. The sparse penalized approaches are known to have more stable sampling properties compared with the traditional best subset selection approaches, keeping higher computational efficiency. In this paper, we propose a new penalty, sparse net (SNET), that can be used for variable selection and parameter estimation in the linear regression model. The SNET consists of two penalties: the LASSO and the negative ridge. The SNET shares the same spirit with the elastic net but the ridge effect is negatively imposed on the regression coefficients to avoid unnecessary selection and biases. In general, the SNET penalized sum of squared residuals is non-convex, depending on the size of negative ridge effect, which requires non-convex optimization algorithm. For this issue, we introduce an efficient algorithm for implementing the SNET penalized estimator. Several numerical examples show that the SNET can be a nice competitor with existing penalties.