Smoothed Quantiles for χ2 Type Test Statistics with Applications

Abstract

Chi-square type test statistics are widely used in assessing the goodness-of-fit of a theoretical model. The exact distributions of such statistics can be quite different from the nominal chi-square distribution due to violation of conditions encountered with real data. In such instances, the bootstrap or Monte Carlo methodology might be used to approximate the distribution of the statistic. However, the sample quantile may be a poor estimate of the population counterpart when either the sample size is small or the number of different values of the replicated statistic is limited. Using statistical learning, this article develops a method that yields more accurate quantiles for chi-square type test statistics. Formulas for smoothing the quantiles of chi-square type statistics are obtained. Combined with the bootstrap methodology, the smoothed quantiles are further used to conduct equivalence testing in mean and covariance structure analysis. Two real data examples illustrate the applications of the developed formulas in quantifying the size of model misspecification under equivalence testing. The idea developed in the article can also be used to develop formulas for smoothing the quantiles of other types of test statistics or parameter estimates.

Keywords:

• Bootstrap simulation
• smoothed quantile
• statistical learning
• equivalence testing

Article information

Conflict of interest disclosures

Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles

The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Role of the funders/sponsors

The funder or sponsor of this research did not have any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments

The authors would like to thank the Action Editor Dr. Michael Edwards and three anonymous reviewers for their comments on prior versions of this manuscript. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors' institutions or the NSFC is not intended and should not be inferred.

Notes

1 Many test statistics asymptotically follow chi-square distributions under idealized conditions. But their true distributions are unknown in practice due to violation of conditions. We call such statistics ${\chi }^2$ type statistics.

2 The value of N_r plays the role of the number of replications in operation, while we reserve N for the sample size based on which statistic T is evaluated.

3 We might include more covariates in the regression model but the complication can make the prediction more difficult, since we need to solve x corresponding to $\widehat{T}=c_{\alpha }$ in a future step.

4 But a large value of N_r can be costly in computation and may create the problem of non-convergence in estimating the model parameters.

5 We use Q_ml instead of F_ml for the normal-distribution-based discrepancy function because F has been used for the cumulative distribution function of T.

6 We divided the nine variables in the original dataset respectively by 6.0, 4.0, 8.0, 3.0, 4.0, 7.0, 23.0, 20.0, 36.0, making all the standard deviations of the resulting variables between 1.0 and 2.0. Such a rescaling facilitates the speed of convergence in parameter estimation but does not affect the analyses in any substantive way.

7 Although the values of h and Q_mlh are the output of solving Equation (8)(8) $$T_{ml}=c_{\alpha }({\widehat{ϵ}}_t),$$(8) using the first smoothed quantile, they will be the same if solving Equation (8)(8) $$T_{ml}=c_{\alpha }({\widehat{ϵ}}_t),$$(8) using any of the other 4 smoothed quantiles for this example.

8 In theory, RMSEA is defined at the population level. In operation, one has to estimate it via the observed statistic.

9 Raw observations were transformed using ${\hspace{0.17em}\text{log}\hspace{0.17em}}_{10}(·)$ in Fears et al. (1996), which are what we used in the analysis.

Additional information

Funding

This work was supported by Grant 31971029 from the Natural Science Foundation of China (NSFC).