Improving econometric prediction by machine learning

ABSTRACT

We present a Machine Learning (ML) toolbox to predict targeted econometric outcomes improving prediction in two directions: (i) by cross–validated optimal tuning, (ii) by comparing/combining results from different learners (meta–learning). In predicting woman wage class based on her characteristics, we show that all our ML methods’ predictions highly outperform standard multinomial logit ones, both in terms of mean accuracy and its standard deviation. In particular, we set out that a regularized multinomial regression obtains an average prediction accuracy almost 60% larger than that of an unregularized one. Finally, as different learners may behave differently, we show that combining them into one ensemble learner proves to preserve good predictive accuracy lowering the variance more than stand-alone approaches.

KEYWORDS:

• Machine learning
• ensemble methods
• optimal prediction

JEL CLASSIFICATION:

• C53
• C61
• C63

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

¹ The use of, let’s say, a tree-based propensity score for estimating the selection equation opens up the problem of correctly estimating the standard error of the average treatment effect in the second-step (outcome) equation. What is the asymptotic distribution of the average treatment effect estimator when the first-step propensity score is estimated via a highly non-parametric procedure? This is still an open question that could forage a new stream of research in causal inference. A possible empirical solution could be the use of the bootstrap, although one should prove that bootstrap is correct in this context.

² The main reference on the statistics of meta–learning can be found in Van der Laan and Rose (2011).

³ A challenging stream of research aims at understanding the relationship between data structure and ML models’ prediction ability. We know so far that when data present a strong inner ordering, some methods tend to outperform others. For instance, for image recognition purposes, deep neural networks are surprisingly accurate compared to other classification algorithms. This has to do with the inner ordering of images, such a human faces. In particular, convolutional neural networks are highly suited for this task.

⁴ All algorithms implementation and graphing have been programmed in Python 3.7, using the Stata/Python integrated interface available in Stata 16. All codes are available on request.

⁵ We assume that the observed learner–specific accuracies ${\hat{\theta }}_j$, $j=1,\dots ,M$, represent a random sample from a population that is normally distributed with mean $\theta$ and variance ${\tau }^2$. The weights are thus obtained considering a random-effects model where the ${\hat{\theta }}_j={\theta }_j+{\in }_j=\theta +u_j+{\in }_j$ where ${\in }_j$ and $u_j$ are assumed to be independent with ${\in }_j\sim N(0,{\hat{\sigma }}_j^2)$ and $u_j=N(0,{\tau }^2)$. The weights are thus calculated as ${\hat{w}}_j=1/({\hat{\sigma }}_j^2+{\hat{\tau }}^2)$, with ${\hat{\sigma }}_j^2$ obtained by cross–validation, and ${\hat{\tau }}^2$ by the random–effects maximum likelihood.