Matching research and practice: Prediction of individual patient progress and dropout risk for basic routine outcome monitoring

Abstract

Objective

Despite evidence showing that systematic outcome monitoring can prevent treatment failure, the practical conditions that allow for implementation are seldom met in naturalistic psychological services. In the context of limited time and resources, session-by-session evaluation is rare in most clinical settings. This study aimed to validate innovative prediction methods for individual treatment progress and dropout risk based on basic outcome monitoring.

Methods

Routine data of a naturalistic psychotherapy outpatient sample were analyzed (N = 3902). Patients were treated with cognitive behavioral therapy with up to 95 sessions (M = 39.19, SD = 16.99) and assessment intervals of 5–15 sessions. Treatment progress and dropout risk were predicted in two independent analyses using the nearest neighbor method and least absolute shrinkage and selection operator regression, respectively.

Results

The correlation between observed and predicted patient progress was r = .46. Intrinsic treatment motivation, previous inpatient treatment, university-entrance qualification, baseline impairment, diagnosed personality disorder, and diagnosed eating disorder were identified as significant predictors of dropout, explaining 11% of variance.

Conclusions

Innovative outcome prediction in naturalistic psychotherapy is not limited to elaborated progress monitoring. This study demonstrates a reasonable approach for tracking patient progress as long as session-by-session assessment is not a valid standard.

Keywords:

• Cognitive behavior therapy
• outcome research
• process research
• statistical methodology

Acknowledgement

The authors would like to thank the reviewers for their helpful feedback and suggestions in the revision of the manuscript.

Data and/or Code Availability

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to restrictions that could compromise research participant privacy.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Ethics

The present retrospective study analyzed routine data from a university psychotherapy outpatient clinic in Mainz, Germany. All procedures were performed in accordance with the 1964 Helsinki Declaration and its later amendments. Written, informed consent allowing anonymized data to be used for research purposes was obtained from all participants.

Supplemental data

Supplemental data for this article can be accessed here https://doi.org/10.1080/10503307.2021.1930244.

Notes

1 Successful completion of practical training I (§ 2 PsychThG-APrV) and at least 280 lessons of theoretical course credit.

2 A LASSO linear regression model for continuous outcome was fitted using a 10-fold cross-validation.

3 The Gower coefficient was used in order to include categorical variables in the prediction.

4 Cut-offs were defined in accordance with Derogatis (1993).

5 A LASSO logistic regression model for binary outcomes was fitted using a 10-fold cross-validated bootstrap ranking procedure (100 bootstraps).

6 A comparison of LASSO versus Elastic Net Regularization resulted in the same selection of variables.

7 Rated by the therapist on a five-point Likert scale ranging from 0 (“non-existent”) to 4 (“very high”).

8 There was no significant difference in model fit (Akaike Information Criterion) between LASSO and Elastic Net Regularization. In terms of prediction accuracy, both models performed equally well.

9 The probability that a predicted dropout, classified above the cut-off, actually drops out of treatment.

10 The probability that a dropout has been identified as such above the cut-off.

Additional information

Funding

This work was supported by a PhD scholarship from the Department of Psychology of the Johannes Gutenberg-University Mainz, Germany.