Estimating Models and Evaluating their Efficiency under Multicollinearity in Multiple Linear Regression: A Comparative Study
DOI:
https://doi.org/10.21271/zjhs.28.5.17%20Keywords:
فرە پەيوەيستى هێڵەكى ، لاريبوونى هێڵی فرەیی ، تۆڕی دەماری دەستکرد، پاشەکشەی ڕیجی، شیکاری پێکهاتە سەرەکییەکانAbstract
Multicollinearity between independent variables occurs in multiple linear regression analysis characterized by high correlations, which complicates discerning individual variable effect, impacting model accuracy, stability, and interpretation of relationships. The research aims to diagnose the multicollinearity problem between explanatory variables in the linear regression model and identifying the variables causing this problem based on the variance inflation factor (VIF), then estimation and evaluate the performance of three alternative methods, which are Ridge regression, principal components analysis, and Feedforward Neural Networks (FFNN) models with one and two hidden layers and application of the models to compressive strength data for high-performance concrete. The results showed that Ridge regression and PCA effectively addressed multicollinearity problem, but the single hidden layer model FFNN showed superior predictive accuracy in estimating the compressive strength of high-performance concrete when comparing RMSE, MAE, and R2 values.
References
- Al-Rawi, K.M., 1987. Introduction to Regression Analysis. Dar Al-Kutub for Printing and Publishing, Mosul University.
- Blbas, T.A., Mahmood, H. & Omer, A., 2017. A Comparison results of factor analysis and cluster analysis to the migration of young people from the Kurdistan Region to Europe. ZANCO Journal of Pure and Applied Sciences, 29(4), pp.44-55.
- Chan, Y.-L. et al., 2022. Mitigating the Multicollinearity Problem and Its Machine Learning Approach: A Review. Mathematics, 10(1283), pp.1-17.
- Cohen, , Cohen, , West, S.G. & Aiken, L.S., 2002. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Third edition. Routledge.
- Doane, D.P. & Seward, L., 2021. Applied Statistics in Business and Economics. 7th edition.McGraw-Hill Education.
- Duzan, , 2020. A comparison between the method of least squares and ridge regression in the Presence of Multicollinearity in regression analysis. International Science and Technology Journal, 23, pp.182-201.
- Ekiugbo, A., Amiolemhen, P. & Ariavie, G.O., 2021. Performance of Multiple Linear Regression and Artificial Neural Network in Predicting Risk Index. Journal of Science and Technology Research, 3(4), pp.233-44.
- Fausett, L., 1994. Fundamentals of Neural Networks: Architectures, Algorithms, and Applications. Prentice-Hall.
- Gujarati, D.N. & Porter, D.C., 2009. Basic Econometrics. 5th ed. McGraw-Hill Irwin.
- Haykin, S.S., 1999. Neural Networks: A Comprehensive Foundation. 2nd ed. Prentice Hall.
- Hoerl, A. & Kennard, R., 1970. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12, pp.55-67.
- Johnson, & Wichern, , 2014. Applied Multivariate Statistical Analysis. 6th ed. Pearson Education Limited.
- King, S.L., 1999. Neural Networks vs. Multiple Linear Regression for Estimating previous diameter. In Stringer, & Loftis, D.L., eds. 12th Central Hardwood Forest Conference. Kentucky, 1999,pp159-166
- Kutner, M.H., Nachtsheim, C.J., Neter, J. & Li , , 2005. Applied Linear Statistical Models, 5 Edition. McGraw-Hill Irwin.
- Li , M. & Wang, , 2019. An Empirical Comparison of Multiple Linear Regression and Artificial Neural Network for Concrete Dam Deformation Modelling. Mathematical Problems in Engineering, 1(1), pp.1-13.
- Montgomery, D.C., Peck, E.A. & Vining, G., 2012. Introduction to Linear Regression Analysis. 5th ed. John Wiley & Sons.
- Pati, K.D., 2020. Estimate The Parameters In Presence Of Multicollinearity And Outliers Using Bisquare Weighted Ridge Least Median Squares Regression (WRLMS). Journal of University of Duhok, 32(2), pp.9-24.
- Rashid, M.J. & Tofiq, O., 2022. Application of Principal Component Analysis for Steel Material Components. Kurdistan Journal of Applied Research, 7(2), pp.85-94.
- Rawlings, O., Pantula, G. & Dickey, D.A., 1998. Applied Regression Analysis: A Research Tool, 2nd Edition. Springer.
- Rencher , A.C. & Schaalje, G., 2008. Linear models in statistics. 2nd ed. John Wiley & Sons.
- Samarasinghe, , 2006. Neural Networks for Applied Sciences and Engineering: From Fundamentals to Complex Pattern Recognition. Auerbach Publications.
- Sharma, A., & Chopra, A., 2011, Artificial Neural Networks: Applications In Management, IOSR Journal of Business and Management (IOSR-JBM), Volume 12, Issue 5, PP 32-40
- Shrestha, N., 2020. Detecting Multicollinearity in Regression Analysis. American Journal of Applied Mathematics and Statistics, 8(2), pp.39-42.
- Yeh, I.C., 1998. Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete Research. Elsevier, 28(12), pp.1797-1808.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Saman Hussein Mahmood
This work is licensed under a Creative Commons Attribution 4.0 International License.
