Simple use of BIC to Assess Model Selection Uncertainty: An Illustration using Mediation and Moderation Models

Abstract

The Bayesian information criterion (BIC) has been used sometimes in SEM, even adopting a frequentist approach. Using simple mediation and moderation models as examples, we form posterior probability distribution via using BIC, which we call the BIC posterior, to assess model selection uncertainty of a finite number of models. This is simple but rarely used. The posterior probability distribution can be used to form a credibility set of models and to incorporate prior probabilities for model comparisons and selections. This was validated by a large scale simulation and results showed that the approximation via the BIC posterior is very good except when both the sample sizes and magnitude of parameters are small. We applied the BIC posterior to a real data set, and it has the advantages of flexibility in incorporating prior, addressing overfitting problems, and giving a full picture of posterior distribution to assess model selection uncertainty.

Keywords:

• BIC posterior
• mediation and moderation
• model selection
• posterior distribution

Article information

Conflicts of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This research was partially supported by a grant from the Natural Science Foundation of Fujian Province of China (Grant No. 2018J05002); and partially supported by a Multi-Year Research Grant from the University of Macau (Grant No. MYRG2017-00032-CCE).

Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The authors would like to acknowledge Prof Kelvin Yuen's (Department of Civil and Environmental Engineering, University of Macau) input at the very beginning to use BIC as an approximation. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institutions or the Natural Science Foundation of Fujian Province of China is not intended and should not be inferred.