Semiparametric Inference in a Genetic Mixture Model

ABSTRACT

In genetic backcross studies, data are often collected from complex mixtures of distributions with known mixing proportions. Previous approaches to the inference of these genetic mixture models involve parameterizing the component distributions. However, model misspecification of any form is expected to have detrimental effects. We propose a semiparametric likelihood method for genetic mixture models: the empirical likelihood under the exponential tilting model assumption, in which the log ratio of the probability (density) functions from the components is linear in the observations. An application to mice cancer genetics involves random numbers of offspring within a litter. In other words, the cluster size is a random variable. We wish to test the null hypothesis that there is no difference between the two components in the mixture model, but unfortunately we find that the Fisher information is degenerate. As a consequence, the conventional two-term expansion in the likelihood ratio statistic does not work. By using a higher-order expansion, we are able to establish a nonstandard convergence rate N^{− 1/4} for the odds ratio parameter estimator $\widehat{\beta }$. Moreover, the limiting distribution of the empirical likelihood ratio statistic is derived. The underlying distribution function of each component can also be estimated semiparametrically. Analogously to the full parametric approach, we develop an expectation and maximization algorithm for finding the semiparametric maximum likelihood estimator. Simulation results and a real cancer application indicate that the proposed semiparametric method works much better than parametric methods. Supplementary materials for this article are available online.

KEYWORDS:

• Cluster mixture data
• Degenerate Fisher information
• Exponential tilting model
• Genetic mixture model

Supplementary Materials

The online web appendix contains more simulation studies, more details for the EM-algorithm in Section 2.2, and detailed proofs of Theorems 1–3.

Acknowledgments

The authors thank the joint editors, the associate editor, and the two referees for constructive comments and suggestions that led to a significant improvement. The first two authors contributed equally to this work. Liu is the corresponding author.

Funding

Dr. Li’s research is supported in part by NSERC Grant RGPIN-2015-06592. Dr. Liu’s research is supported by grants from the National Natural Science Foundation of China (Numbers 11371142, 11171112, 11101156, 11501208, and 11501354), the Program of Shanghai Subject Chief Scientist (14XD1401600), and the 111 Project (B14019).