A New Bayesian Test to Test for the Intractability-Countering Hypothesis

ABSTRACT

We present a new test of hypothesis in which we seek the probability of the null conditioned on the data, where the null is a simplification undertaken to counter the intractability of the more complex model that the simpler null model is nested within. With the more complex model rendered intractable, the null model uses a simplifying assumption that capacitates the learning of an unknown parameter vector given the data. Bayes factors are shown to be known only up to a ratio of unknown data-dependent constants—a problem that cannot be cured using prescriptions similar to those suggested to solve the problem caused to Bayes factor computation, by noninformative priors. Thus, a new test is needed in which we can circumvent Bayes factor computation. In this test, we undertake generation of data from the model in which the null hypothesis is true and can achieve support in the measured data for the null by comparing the marginalized posterior of the model parameter given the measured data, to that given such generated data. However, such a ratio of marginalized posteriors can confound interpretation of comparison of support in one measured data for a null, with that in another dataset for a different null. Given an application in which such comparison is undertaken, we alternatively define support in a measured dataset for a null by identifying the model parameters that are less consistent with the measured data than is minimally possible given the generated data, and realizing that the higher the number of such parameter values, less is the support in the measured data for the null. Then, the probability of the null conditional on the data is given within a Markov chain Monte Carlo (MCMC)-based scheme, by marginalizing the posterior given the measured data, over parameter values that are as, or more consistent with the measured data, than with the generated data. In the aforementioned application, we test the hypothesis that a galactic state-space bears an isotropic geometry, where the (missing) data comprising measurements of some components of the state-space vector of a sample of observed galactic particles are implemented to Bayesianly learn the gravitational mass density of all matter in the galaxy. In lieu of an assumption about the state-space being isotropic, the likelihood of the sought gravitational mass density given the data is intractable. For a real example galaxy, we find unequal values of the probability of the null—that the host state-space is isotropic—given two different datasets, implying that in this galaxy, the system state-space constitutes at least two disjoint sub-volumes that the two datasets, respectively, live in. Implementation on simulated galactic data is also undertaken, as is an empirical illustration on the well-known O-ring data, to test for the form of the thermal variation of the failure probability of the O-rings. Supplementary materials for this article are available online.

KEYWORDS:

• Bayes factors
• Bayesian p-values
• Hypothesis testing
• Markov chain Monte Carlo

Supplementary Materials

Details of the Bayesian learning of the gravitational mass density and state-space $\text{pdf}$ of the galaxy are provided in Section S-1 of the attached supplementary material. Section S-2 discusses details of the fully Bayesian significance test.

Acknowledgments

The author gratefully acknowledge the comments of the reviewers that helped improve the article.