Postselection Inference in Structural Equation Modeling

Abstract

Most statistical inference methods were established under the assumption that the fitted model is known in advance. In practice, however, researchers often obtain their final model by some data-driven selection process. The selection process makes the finally fitted model random, and it also influences the sampling distribution of the estimator. Therefore, implementing naive inference methods may result in wrong conclusions—which is probably a prime source of the reproducibility crisis in psychological science. The present study accommodates three valid state-of-the-art postselection inference methods for structural equation modeling (SEM) from the statistical literature: data splitting (DS), postselection inference (PoSI), and the polyhedral (PH) method. A simulation is conducted to compare the three methods with the commonly used naive procedure under selection events made by L₁-penalized SEM. The results show that the naive method often yields incorrect inference, and that the valid methods control the coverage rate in most cases with their own pros and cons. Real world data examples show the practical use of the valid inference methods.

Keywords:

• Structural equation modeling
• factor analysis
• lasso
• postselection inference
• polyhedral lemma

Article Information

Conflict of interest disclosures: The author signed a form for disclosure of potential conflicts of interest. The author did report any financial or other conflicts of interest in relation to the work described.

Ethical principles: The author affirms having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This work was supported by Grant MOST106-2410-H-006-038 from the Ministry of Science and Technology in Taiwan.

Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The ideas and opinions expressed herein are those of the authors alone, and endorsement by the author’s institution or the Ministry of Science and Technology in Taiwan is not intended and should not be inferred.

Notes

1 In linear regression problems with S covariates to be chosen, the Scheffe’s PoSI constant is $k_{\alpha }=\sqrt{S\times F_{S,N-S,1-\alpha }},$ where $F_{S,N-S,1-\alpha }$ is the $1-\alpha$-quantile of the F distribution with degrees of freedom S and N – S. In SEM settings, we should consider $S\times F_{S,N-S,1-\alpha }$ with $N\to \infty ,$ which converges to ${\chi }_{S,1-\alpha }^2.$

2 Despite that we used” large” to qualify the value of 0.3, a standardized regression coefficient $\beta =0.3$ only represents a medium effect according to Cohen (1992). Thus, the values of the non-null effects are considered only small to medium. We just used the terms “small”, “medium”, and “large” for convenience. When we adopted a large effect defined by Cohen here (e.g., $\beta =0.5$), the resulting R² became larger than one, which is impossible.

3 The five-number summary for the rates of convergent solutions was 27%, 93%, 100%, 100%, and 100%. We found that nonconvergent solutions mainly occurred when both the model size and the number of non-null effects were large and the sample size was 100. For other conditions, the rate of convergent solutions exceeded 93%.

4 Recall that the population target is model-dependent. Suppose that there are Q potential causes. Without the duplicated cases, there are $Q·2^{Q-1}$ model-dependent parameters since: (1) Q path coefficients are considered; and (2) each potential cause appears in $2^{Q-1}$ candidate models.