Response versus gradient boosting trees, GLMs and neural networks under Tweedie loss and log-link

Abstract

Thanks to its outstanding performances, boosting has rapidly gained wide acceptance among actuaries. To speed up calculations, boosting is often applied to gradients of the loss function, not to responses (hence the name gradient boosting). When the model is trained by minimizing Poisson deviance, this amounts to apply the least-squares principle to raw residuals. This exposes gradient boosting to the same problems that lead to replace least-squares with Poisson Generalized Linear Models (GLM) to analyze low counts (typically, the number of reported claims at policy level in personal lines). This paper shows that boosting can be conducted directly on the response under Tweedie loss function and log-link, by adapting the weights at each step. Numerical illustrations demonstrate similar or better performances compared to gradient boosting when trees are used as weak learners, with a higher level of transparency since responses are used instead of gradients.

Keywords:

• Risk classification
• boosting
• gradient boosting
• regression trees
• GLM
• neural networks

Acknowledgements

The authors thank anonymous Referees and the Editor for their constructive comments which helped to improve this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Chollet, F., et al. (2015). https://github.com/rstudio/keras/

2 Notice that the 1 Neuron NN differs from the GLM by the fact that covariates pass through a first layer with hyperbolic tangent activation function.