A sensitivity analysis of the Children’s Treatment Network trial: a randomized controlled trial of integrated services versus usual care for children with special health care needs

Abstract

Background

The value of integrated care through comprehensive, coordinated, and family-centered services has been increasingly recognized for improving health outcomes of children with special health care needs (CSHCN). In a randomized controlled trial (RCT), the integrated care provided through the Children’s Treatment Network (CTN) was compared with usual care in improving the psychosocial health of target CSHCN. In this paper, we aimed to estimate the effect of CTN care by conducting multiple analyses to handle noncompliance in the trial.

Methods

The trial recruited target children in Simcoe County and York Region, ON, Canada. Children were randomized to receive CTN or usual care and were followed for 2 years. The CTN group received integrated services through multiple providers to address their specific needs while the usual care group continued to receive care directed by their parents. The outcome was change in psychosocial quality of life at 2 years. We conducted intention-to-treat, as-treated, per-protocol, and instrumental variable analyses to analyze the outcome.

Results

The trial randomized 445 children, with 229 in the intervention group and 216 in the control group. During follow-up, 52% of children in the intervention group did not receive complete CTN care for various reasons. At 2 years, we did not find a significant improvement in psychosocial quality of life among the children receiving CTN care compared with usual care (intention-to-treat mean difference 1.50, 95% confidence interval −1.49 to 4.50; P = 0.32). Other methods of analysis yielded similar results.

Conclusion

Although the effect of CTN care was not significant, there was evidence showing benefits of integrated care for CSHCN. More RCTs are needed to demonstrate the magnitude of such an effect. The CTN study highlights the key challenges in RCTs when assessing interventions involving integrated care, and informs further RCTs including similar evaluations.

Keywords:

• children with special health care needs
• chronically ill
• family-centered care
• randomized controlled trial
• noncompliance

Supplementary materials

Propensity score methods

The propensity score (PS) is the probability of receiving a treatment conditional on observed characteristics. Balancing on PS mimics randomization and produces comparable groups that are balanced on prognosis and observed confounding. We collected baseline variables that included child’s age, sex, admission diagnosis, parent’s age, marital status, education, distress, parenting style, family income, social support, and family functioning. Using those variables, we calculated the PS for receiving complete Children’s Treatment Network (CTN) integrated care by a logistic regression model. Four PS methods were used to balance between treated and untreated children in the AT and PP analyses, ie, matching, stratification, weighting, and covariate adjustment.

In PS matching, we created matched pairs of treated and untreated children by matching them within 0.2 of the standard deviation of the logit of the PS. We used a 1:1 ratio to match the nearest children without replacement. This matching algorithm produced the least bias.¹ We then used the generalized estimating equation model to analyze the matched data. An exchangeable correlation structure was used.

In PS stratification, children were divided into five equal strata by the quintiles of their PS.² The outcome between treated and untreated children was compared directly in each stratum. The overall treatment effect is given by

${\beta }_{overall}=\frac{{\displaystyle {\sum }_i^s{\beta }_i}}{s},$

where β_i is the estimated treatment effect in stratum i. The variance of β_overall is calculated by

$Var({\beta }_{overall})=\frac{1}{{\displaystyle {\sum }_i^s{w}_i}},$

where w_i is the inverse of the estimated variance of β_i for stratum i.

In the PS-weighted method, we balanced the children by the inverse probability of receiving CTN integrated care. The weight (w_i) for each child can be calculated by

$w_i=\frac{X_i}{e_i}+\frac{(1-X_i)}{1-e_i},$

where X_i is the treatment indicator and e_i is the estimated PS for child i. Confounding between the observed variables and treatment was eliminated in the weighted sample. We then directly compared the outcome between treated and untreated children using a weighted linear regression model.

In the covariate adjustment method, we adjusted the PS as the sole covariate in the regression model for estimating the treatment effect.

Calculating standard error for instrumental variable estimate

Suppose we have two random variables, X and Y. A Taylor series expansion of f(x, y) about the values (x₀, y₀) is given by

$\begin{array}{c}f(x,y)=f(x_0,y_0)+(x-x_0)\frac{\partial f(x,y)}{\partial x}{\left|{}_{(x_0,y_0)}+(y-y_0)\frac{\partial f(x,y)}{\partial y}\right|}_{(x_0,y_0)}+o(\cdot )\\ \approx f(x_0,y_0)+(x-x_0)\times \frac{\partial f(x,y)}{\partial x}{\left|{}_{(x_0,y_0)}+(y-y_0)\frac{\partial f(x,y)}{\partial y}\right|}_{(x_0,y_0)}\end{array}$

where o(⋅) represents the higher order terms which are omitted in this approximation. The instrumental variable (IV) estimator for the exposure X and the outcome Y is given by

${\beta }_{IV}=\frac{{\beta }_{(Z\to Y)}}{{\beta }_{(Z\to X)}}$

where β_(Z→Y) and β_(Z→X) are the parameters of association between Z and Y and between Z and X, respectively. By the Taylor series expansion on β_IV, we have

${\beta }_{IV}\approx \frac{b_{(Z\to Y)}}{b_{(Z\to X)}}+({\beta }_{(Z\to Y)}-b_{(Z\to Y)})\frac{1}{b_{(Z\to X)}}-({\beta }_{(Z\to X)}-b_{(Z\to X)})\frac{b_{(Z\to Y)}}{b_{(Z\to X)}{}^2}$

where b_(Z→Y) and b_(Z→X) are two values at which β_IV is differentiable. The variance of the IV estimator can then be approximated as

$Var({\beta }_{IV})\approx \frac{1}{b_{(Z\to X)}{}^2}Var({\beta }_{(Z\to Y)})+\frac{b_{(Z\to Y)}{}^2}{b_{(Z\to X)}{}^4}Var({\beta }_{(Z\to X)})-\frac{2b_{(Z\to Y)}}{b_{(Z\to X)}{}^3}Cov({\beta }_{(Z\to X)},{\beta }_{(Z\to Y)}).$

Under the assumption that β_(Z→χ) is independent of β_(Z→Y), the variance of the IV estimator is then

$Var({\beta }_{IV})\approx \frac{1}{b_{(Z\to X)}{}^2}Var({\beta }_{(Z\to Y)})+\frac{b_{(Z\to Y)}{}^2}{b_{(Z\to X)}{}^4}Var({\beta }_{(Z\to X)}).$

We substitute b_(Z→Y) and b_(Z→X) by the estimates of ß_(Z→Y) and ß_(Z→X), respectively, and approximate the $\sqrt{Var({\beta }_{(Z\to Y)})}$ and $\sqrt{Var({\beta }_{(Z\to X)})}$ by the associated standard error. Thus, we can obtain an approximate variance of the IV estimate. In our analysis, we used the least squares estimate of ß_(Z→Y) and associated standard error obtained from the linear regression model. For the exposure X (a binary indicator of whether or not a patient received complete CTN integrated care), the association between X and the IV can be calculated by

${\beta }_{(Z\to X)}=P(X=1|Z=1)-P(X=1|Z=0)$

where P(X = 1|Z = 1) represents the proportion of treated patients in the CTN group; and P(X = 1|Z = 0) is always zero because the children in the usual care group are deemed to be untreated. The variance of β_(Z→X) is

$Var({\beta }_{(Z\to X)})=nP(X=1|Z=1)(1-P(X=1|Z=1))$

where n is the number of children in the CTN group.

References

• AustinPCOptimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studiesPharm Stat201110215016120925139
   PubMed Web of Science ®Google Scholar
• RosenbaumPRRubinDBReducing bias in observational studies using subclassification on the propensity scoreJ Am Stat Assoc198479387516524
   Web of Science ®Google Scholar

Acknowledgments

CY is supported in part by funding from the Father Sean O’Sullivan Research Centre studentship award, the Canadian Institute of Health Research Training award in Bridging Scientific Domains for Drug Safety and Effectiveness, and the Canadian Network and Centre for Trials Internationally program.

Author contributions

CY conceived the sensitivity analysis, proposed statistical methods, performed all analyses, interpreted the results, and drafted and revised the manuscript. GB advised on important intellectual content and revised the manuscript. GB was also commissioned by the CTN to design, carry out, analyze, and report the original trial. LT and JB contributed to the statistical design of the sensitivity analysis and revision of the manuscript. All authors have read and approved the final manuscript.

Disclosure

The authors report no conflicts of interest in this work.