A model averaging approach for the ordered probit and nested logit models with applications

ABSTRACT

This paper considers model averaging for the ordered probit and nested logit models, which are widely used in empirical research. Within the frameworks of these models, we examine a range of model averaging methods, including the jackknife method, which is proved to have an optimal asymptotic property in this paper. We conduct a large-scale simulation study to examine the behaviour of these model averaging estimators in finite samples, and draw comparisons with model selection estimators. Our results show that while neither averaging nor selection is a consistently better strategy, model selection results in the poorest estimates far more frequently than averaging, and more often than not, averaging yields superior estimates. Among the averaging methods considered, the one based on a smoothed version of the Bayesian Information criterion frequently produces the most accurate estimates. In three real data applications, we demonstrate the usefulness of model averaging in mitigating problems associated with the ‘replication crisis’ that commonly arises with model selection.

Keywords:

• Hit rate
• model averaging
• model selection
• Monte Carlo
• nested logit
• ordered probit
• screening

Acknowledgements

The authors thank the associate editor and two referees for thoughtful review of the manuscript. The usual disclaimer applies.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. The subject of replication crisis has attracted enormous attention among scientists in recent years. See [13] for a high-level introduction. Andrew Gelman's blog (http://andrewgelman.com/) provides links to many interesting articles written on this subject.

2. Model averaging is a fundamentally different approach from boosting used extensively in machine learning. In contrast to model averaging, boosting adds new models to the model ensemble sequentially, creating a new model space that is more complex than the original. Davidson and Fan [10] showed that when there exists considerable uncertainty in the original model space, model averaging is often preferred to boosting which is perceived as building an overly complex model out of insufficient data.

3. The Matlab codes for computing the FMA estimates are available for download from the corresponding author's website:http://personal.cb.cityu.edu.hk/msawan/researchprofile.htm.

4. With the exception of the Jackknife methods, the FMA methods considered in this paper are not computationally demanding. To give an idea, in our simulations, under the nested logit model with ($n_1,n_2)=(300,100)$, it takes 7–9 s to complete one round of replication if the JMA estimators are excluded from the set; however, if the JMA estimators are also included, then the corresponding computing time increases to 120–140 s. It is also observed that the time required for computing the JMA estimates increases with the sample size. The scalability of the model averaging methods in relation to computing time is not an issue except for the JMA methods. For model averaging in a high-dimensionality setup, see [2].

5. The data are available online at www.asiabarometer.org/. Inoguchi et al. [22] provided a detailed discussion of the survey.

Additional information

Funding

Wan's work was supported by a strategic grant from the City University of Hong Kong (Project no. 7004985). Zhang's research was supported by the National Science Foundation of China (Project nos. 71522004, 71463012, 71631008 and 11471324) and a grant from the Ministry of Education of China (Project no. 17YJC910011).