ACOMPARATIVE STUDY OF SOME VARIABLES SELECTION  METHODS IN HIGH DIMENSIONAL MULTIPLE LINER  REGRESSION VIA SIMULATION

MEDIA SHAMSADDIN  BARI; HUSSEIN ABDULRAHMAN  HASHEM

MEDIA SHAMSADDIN BARI Dept. of Mathmetic ,College of Basic Education,University of Duhok,Kurdistan Region,Iraq
HUSSEIN ABDULRAHMAN HASHEM Dept. of Mathematics, College of Sciences, University of Duhok, Kurdistan Region, Iraq

Keywords: High-dimensional regression; Lasso; Split regularization

Abstract

In this study, we surveyed many strategies for picking relevant variables in high-dimensional MLR analyses. Parameters in linear regression are often estimated using traditional approaches like the Ordinary Least Squares (OLS) methodology. However, OLS estimates do not fare well when the dataset contains outliers or when the assumption of normality is broken, as in the case of heavy-tailed errors. Huber Lasso (Rosset and Zhu, 2007) and quantile regression (Koenker and Bassett, 1978) are two examples of resilient regularized regression techniques presented as solutions to this issue. This study examines the differences between the Whitening Lasso (WLasso) estimates, adaptable Huber Lasso (HLasso) estimates, adaptive LAD Lasso (ALasso) estimates, genLasso (generative least squares) estimates, gamma (gam) estimates, and Split Regularized Regression (SRR) estimates

Downloads

Download data is not yet available.

References

Arnold, T. B., and Tibshirani, R. J. (2014).Efficient implementations of the generalized Lasso dual-path algorithm. Journal of Computational and Graphical Statistics, 25(1):1–27, 2016.
Bradic, J. and J. Fan (2011). Penalized composite quasi-likelihood for ultrahigh dimensional variable selection. Journal of the Royal Statistical Society, B 73 (3), 325–349.
Christidis, A.-A., Lakshmanan, L., Smucler, E., and Zamar, R. (2020). Split regularized regression. Technometrics 62.3, pp. 330–338.
Fan, J. and R. Li (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96(456), 1348–1360.
Fujisawa, H. and Eguchi, S. (2008). Robust parameter estimation with a small bias against heavy contamination, Journal of Multivariate Analysis, 99(9), 2053-2081.
Koenker, R. and G. W. Bassett (1978). Regression quantiles. Econometrica 46, 33–50.
Lambert-Lacroix, S. and Zwald, L. (2011). Robust regression through Huber’s criterion and adaptive Lasso penalty. Electronic Journal of Statistics 5,
Qin, Y., Li, S. and Yu, Y. (2017). Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection.https://arxiv.org/pdf/1708.05439.pdf.
Rosset, S. and Zhu, J. ( 2007). Piecewise linear regularized solution paths. The Annals of Statistics 35 (3), 1012–1030.
Taddy, M. (2017). One-step estimator paths for concave regularization, Journal of Computational and Graphical Statistics pp. 1–12.
Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B 58, 267–288.
Tibshirani, R. J., and Taylor, J. (2011). The solution path of the generalized Lasso. Ann.Stat., 39(3), 1335-1371.
Wang, H., Li, G., and Jiang, G. (2007). Robust regression shrinkage and consistent variable selection through the LAD-Lasso. Journal of Business & Economic Statistics 25, 347 - 355.
Yi, C. Huang, J. (2016). Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression. Journal of Computational and Graphical Statistics 3. 547–557
Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society B, 67, 301–320.
Zou, H. (2006). The adaptive Lasso and its oracle properties. Journal of the American Statistical Association 101, 1418–1429.
Zhu, W., L´evy-Leduc, C., and Tern`es, N. (2021). A variable selection approach for highly correlated predictors in high-dimensional genomic data. Bioinformatics, 37(16), 2238– 2244.