Improving random forest algorithm by selecting appropriate penalized method

Abstract

This article is improved the random forest algorithm by selecting the most appropriate penalized regression methods, and it is tried to improve the post-selection boosting random forest (PBRF) algorithm using elastic net regression. The proposed method with the highest efficiency is called Reducing and Aggregating Random Forest Trees by Elastic Net (RARTEN). The introduced method consists of three steps. In the first step, the random forest algorithm is used as a predictor. In the second step, Elastic Net, as a penalized regression method, is applied to reduce the number of trees and improve the random forest and PBRF. In the last step, selected trees are aggregated. The obtained results of the real data and Monte Carlo simulation are evaluated using various statistical performance criteria. The simulation study shows that the RARTEN with 7%, 5%, and 8.5% reduction in the linear, nonlinear, and noise model, respectively improve the accuracy of the traditional random forest and the proposed method by Wang. In addition, this method has a significant reduction compared to other penalized regression methods. Moreover, the real data results show that the proposed method in our study with a reduction of almost 16% confirms the validity of the proposed model.

Keywords:

• Adaptive lasso
• Elastic Net
• Group lasso
• Lasso
• Machine learning
• Random forest

Acknowledgment

The authors are thankful to the reviewers for the insightful comments and suggestions that have resulted in a much improved version of this article.