Permutation Tests for Assessing Potential Non-Linear Associations between Treatment Use and Multivariate Clinical Outcomes

Abstract

In many psychometric applications, the relationship between the mean of an outcome and a quantitative covariate is too complex to be described by simple parametric functions; instead, flexible nonlinear relationships can be incorporated using penalized splines. Penalized splines can be conveniently represented as a linear mixed effects model (LMM), where the coefficients of the spline basis functions are random effects. The LMM representation of penalized splines makes the extension to multivariate outcomes relatively straightforward. In the LMM, no effect of the quantitative covariate on the outcome corresponds to the null hypothesis that a fixed effect and a variance component are both zero. Under the null, the usual asymptotic chi-square distribution of the likelihood ratio test for the variance component does not hold. Therefore, we propose three permutation tests for the likelihood ratio test statistic: one based on permuting the quantitative covariate, the other two based on permuting residuals. We compare via simulation the Type I error rate and power of the three permutation tests obtained from joint models for multiple outcomes, as well as a commonly used parametric test. The tests are illustrated using data from a stimulant use disorder psychosocial clinical trial.

Keywords:

• Chi-square distribution
• joint tests
• likelihood ratio test
• linear mixed effects model
• multiple outcomes

Article information

Conflict of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. BR, SL and GF have no conflict of interest. RW received consulting fees from Alkermes and Astellas.

Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: BR, SL and GF are supported by the National Institutes of Health (NIDA R33 DA042847) and RW is supported by the National Institutes of Health (NIDA UG1 DA015831).

Role of the funders/sponsors: None of the funders or sponsors of this research had any further role in study design, analysis and interpretation of data; or in the writing of the report.