Threats to external validity in the neuroprediction of substance use treatment outcomes

ABSTRACT

Background: Tools predicting individual relapse risk would invaluably inform clinical decision-making (e.g. level-of-care) in substance use treatment. Studies of neuroprediction – use of neuromarkers to predict individual outcomes – have the dual potential to create such tools and inform etiological models leading to new treatments. However, financial limitations, statistical power demands, and related factors encourage restrictive selection criteria, yielding samples that do not fully represent the target population. This problem may be further compounded by a lack of statistical optimism correction in neuroprediction research, resulting in predictive models that are overfit to already-restricted samples.

Objectives: This systematic review aims to identify potential threats to external validity related to restrictive selection criteria and underutilization of optimism correction in the existing neuroprediction literature targeting substance use treatment outcomes.

Methods: Sixty-seven studies of neuroprediction in substance use treatment were identified and details of sample selection criteria and statistical optimism correction were extracted.

Results: Most publications were found to report restrictive selection criteria (e.g. excluding psychiatric (94% of publications) and substance use comorbidities (69% of publications)) that would rule-out a considerable portion of the treatment population. Furthermore, only 21% of publications reported optimism correction.

Conclusion: Restrictive selection criteria and underutilization of optimism correction are common in the existing literature and may limit the generalizability of identified neural predictors to the target population whose treatment they would ultimately inform. Greater attention to the inclusivity and generalizability of addiction neuroprediction research, as well as new opportunities provided through open science initiatives, have the potential to address this issue.

KEYWORDS:

• Neuroprediction
• generalizability
• substance use disorders
• clinical decision-making
• selection criteria
• translational research
• precision medicine

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by funding from IK2 CX001807/CX/CSRD VA (PI: Forster). SEF was responsible for the concept. NNG designed and executed the literature search strategy. NNG and SEF contributed to the review of search results, as well as the interpretation of relevant findings. SEF and NNG drafted and revised the manuscript and have critically reviewed content and approved the final version. The contents do not represent the views of the Department of Veterans Affairs, Department of Defense, or the United States Government