Cross-Validation, Risk Estimation, and Model Selection: Comment on a Paper by Rosset and Tibshirani

Click to increase image sizeClick to decrease image size

Acknowledgments

I am grateful for several helpful conversations with Brad Efron.

Notes

1 Note

For grf, I used the function regression_forest with option tune.parameters = TRUE. For xgboost, I used the function cv.xgb, which cross-validates the number of trees used for boosting. I set nrounds = 1000, early_stopping_rounds = 10 and max_depth = 3, with other parameters set to default. More extensive cross-validation with random search over other parameters, such as eta, max_depth and gamma, did not improve the performance of boosting here.

Additional information

Funding

This work was supported by National Science Foundation10.13039/100000001 grant DMS-1916163 and a Facebook Faculty Award.