Abstract
Objectives
To evaluate the safety and efficacy of a digital therapeutic in treatment-seeking individuals with opioid use disorder (OUD) in an analysis of randomized clinical trial (RCT) data (ClinicalTrials.gov identifier: NCT00929253).
Methods
Secondary analysis of an RCT including 170 adults meeting DSM-IV criteria for OUD. Participants were randomized to 12-weeks of treatment-as-usual (TAU) or TAU plus a digital therapeutic providing 67 digital, interactive educational modules based on the Community Reinforcement Approach. TAU consisted of buprenorphine maintenance therapy, 30 min biweekly clinician interaction, and abstinence-based contingency management. Primary endpoints were treatment retention and abstinence (negative urine drug screen) during weeks 9–12 of treatment. Safety was assessed by evaluating adverse events.
Results
Participants randomized to TAU plus a digital therapeutic had significantly greater odds of opioid abstinence during weeks 9–12 compared to TAU: 77.3 versus 62.1%, respectively (p=.02), OR 2.08, 95% CI 1.10–3.95. The risk of patients leaving treatment was significantly lower in the digital therapeutic group (HR 0.49, 95% CI 0.26–0.92). No significant difference was observed in the rate of adverse events between groups (p=.42).
Conclusions
A prescription digital therapeutic (PDT) in combination with buprenorphine therapy improves clinically significant patient outcomes including abstinence from illicit opioids and retention in treatment compared with treatment as usual.
Introduction
In 2019, almost 2 million adults aged 18 and older in the United States (US) had an opioid use disorder (OUD)Citation1. Fatal opioid overdoses have skyrocketed over the past decade, killing 51,574 people in the 12-month period ending February, 2020Citation2. The opioid epidemic in the US and globally has highlighted the need for much wider access to pharmacological and behavioral treatments for OUD.
Medications for opioid use disorder (MOUD) along with behavioral therapy, are standard of care for OUDCitation3. Despite the availability of MOUD, 80–90% of individuals who need treatment do not receive careCitation4. Common reasons for this trend include refusal to seek treatment, high cost of care, stigma, and lack of or limited access to treatmentCitation5. These difficulties are magnified in rural communities, where substance use treatment centers and providers can be difficult to physically reach.
Evidence-based behavioral approaches for OUD are resource-intensive and challenging to implement as they require intensive training and ongoing supervision to ensure correct and consistent deliveryCitation6,Citation7.
A computer-based therapeutic was developed to deliver behavioral therapy based on the Community Reinforcement Approach (CRA). CRA is an evidence-based behavioral therapy designed for patients with substance use disordersCitation8. CRA reinforces abstinence from drug use by encouraging behaviors that improve employment status, family and social relations, and increased recreational activitiesCitation9. The therapeutic (academic name Therapeutic Education System [TES]Citation10) was evaluated in two RCTs that formed the evidence base on which FDA authorization of the reSET-O PDT was basedCitation11,Citation12. TES, delivered via a web browser, used clinical content and a mechanism of action equivalent to reSET-O, which delivers prescribed content via mobile devices (i.e. smart phones and tablets).
The digital therapeutic treats patients with OUD by combining CRA therapy with fluency training (to reinforce concept mastery), and contingency management (CM), an evidence-based form of motivational incentivesCitation13. The combination of CRA and CM has been shown to improve substance use treatment outcomes when delivered either by a clinician or digitallyCitation10–12,Citation14,Citation15.
The efficacy of the digital therapeutic was evaluated in a 2014 RCT of 170 adults with OUD at the Center for Addiction Research, University of Arkansas for Medical SciencesCitation12. This study did not report safety outcomes, however. It also did not evaluate abstinence from individual substances, and it did not characterize abstinence during the last four weeks of the 12-week study treatment period, which is the current approach recommended by the National Institute of Drug AbuseCitation16. The objective of this study was to further evaluate the efficacy of the digital therapeutic in treatment-seeking individuals with OUD by analyzing abstinence data in the last four weeks of treatment for both cocaine use and opioid use in order to ascertain any unique impacts of treatment with TES on these specialized populations of patients with substance use disorders (SUDs). Safety data, which have not been reported previously, were analyzed to evaluate any associations between use of a digital therapeutic and adverse events typical of patients with OUD.
Methods
Participants and setting
The design and conduct of this RCT have been described previouslyCitation12. A total of 206 individuals were consented and assessed for eligibility. After exclusion of 36 individuals for lack of intake assessments or other reasons, 170 individuals with OUD were enrolled in the study. All participants provided written informed consent. Participants were at least 18 years old, in good health, met DSM-IV criteria for opioid dependence, qualified for buprenorphine treatment, had no active psychiatric disorder, no unstable or significant medical illness, were not pregnant, and were not incarcerated. The study was registered as NCT00929253 on ClinicalTrials.gov, approved by University of Arkansas Medical Sciences Investigator Review Board, and conducted according to Good Clinical Practices.
Randomization
All participants were inducted onto buprenorphine (sublingual mono tablet) using procedures described previouslyCitation12. Following induction, randomization to either TAU or TAU plus the digital therapeutic was 1:1 and stratified based on buprenorphine stabilization dose, distance from the clinic, prior treatment status, and past month cocaine use. A designated study coordinator remained blind to the randomization schema until each participant’s assignment was unlocked for implementation.
Treatment-as-usual (TAU)
Participants randomized to TAU received buprenorphine/naloxone and an every-other-week 30-minute meeting with a clinician. During the 30-minute clinician interaction, the patient reviewed his or her treatment progress. Participants provided urine samples in-clinic three times per week: Monday, Wednesday, and Friday. Urine specimens were tested for methadone, opioids, propoxyphene, cocaine, and once per week for benzodiazepines. Oxycontin testing was performed using a single-panel Oxycontin dipstick for a qualitative result. All participants were eligible for CM vouchers based on urine drug screen (UDS) results.
TAU plus digital therapeutic
The program used in the original study consisted of 67 interactive digital modules based on CRA, plus a user guide. Core modules focus on building basic cognitive- behavioral and relapse prevention skills and provide education on preventing infections (i.e. human immunodeficiency virus, hepatitis C virus, and other infections transmitted sexually or via shared needles). Supplemental modules target improvement of psychosocial functioning (e.g. managing relationships, building communication skills, employment status, time management, insomnia), in-depth training on preventing infections, and support for living with infections. Module completion was self-guided with no direct clinician supervision. Participants were asked to complete modules three times per week during weeks without a clinician visit, and twice per week during weeks with a clinician visit. Participants were able to revisit previously completed modules or complete a new module. In this clinical trial patients accessed the web-based therapeutic using in-clinic computers, whereas reSET-O delivers equivalent, prescribed clinical content via an application (app) downloaded to mobile devices (i.e. smartphones, tablets) from an app store with a prescription.
Medication
All participants received buprenorphine pharmacotherapy. Buprenorphine mono tablet was used for induction and buprenorphine/naloxone combination sublingual tablet (4:1 ratio) was used for maintenance and detoxificationCitation12.
During the maintenance phase, participants were administered buprenorphine under observation at each clinic visit. Participants were provided a double-dose of buprenorphine on Mondays and Wednesdays and a triple-dose on Fridays. This dosing schedule is safe and effective, without causing intoxication or clinically significant withdrawal, and is supported by clinical guidelinesCitation17,Citation18. At study completion, participants were either detoxified under supervision of the study physician or referred for continued treatment.
Contingency management (CM)
All participants were eligible for voucher incentives based on abstinence from opioids and cocaine determined via UDS. Vouchers were provided on an escalating scale, where the initial negative UDS was worth 10 points at $0.25 per point. For each subsequent negative UDS, the voucher value was increased by five points, and three consecutively negative UDS resulted in a $10 bonus. In the event of a positive UDS, the voucher value was reduced to the initial value, however after three consecutive negative UDS results the voucher value was increased to the level achieved before it was reduced. Participants could earn up to $997.50 for 12-weeks continuous abstinence.
Outcomes
The primary study endpoints were abstinence and retention in treatment. Abstinence was assessed by UDS throughout the study. The primary endpoint evaluated abstinence during the last four weeks of treatment (weeks 9–12). Each UDS assessment was used to determine a participant’s abstinence from opioids, cocaine or both, three times weekly. Total abstinence was defined as abstinence from both opioids and cocaineCitation12. Participants were considered non-abstinent (i.e. positive) if the UDS indicated cocaine or opioid use for a given third-week time point, or if the sample was missing/not provided, which is a standard, and conservative, approach in the field of addiction researchCitation16.
Retention in treatment was based on the number of days in treatment for each study participant. Dropout was defined as the time of the last face-to-face contact.
A secondary endpoint was the total number of one-third weeks abstinent (opioids and cocaine) for each participant over the 12-week study duration. Exploratory endpoints included likelihood of abstinence from opioids during the last 6–8 weeks of treatment.
Adverse events relevant to patients diagnosed with OUD (e.g. psychiatric events, gastrointestinal problems) were monitored from time of consent throughout the study. Adverse events were retrospectively coded to system organ classification using preferred terms of the Medical Dictionary for Regulatory Activities based on de-identified participant study records.
Statistical analysis
Baseline and demographic characteristics were described using means, standard deviations, and frequencies for the study sample. T-tests and chi-squared tests were used to evaluate differences between treatment groups for continuous and categorical variables, respectively.
The primary endpoint evaluated abstinence during the last four weeks of treatment (weeks 9–12) using a repeated measures logistic generalized estimating equations model with factors for treatment, time, and treatment × timeCitation19,Citation20. Three categories of abstinence were evaluated: abstinence from opioids and cocaine, from opioids alone, and from cocaine alone. Dropout was considered an event and time to withdrawal from treatment was treated as time-to-event data. Values ranged from 0 to 81 days, with a maximum of 81 days for participants completing all 12 weeks of treatment. Retention rate was estimated at weeks 2, 4, 6, 8, 10, and 12, using the Kaplan-Meier method. Retention distribution between the two groups was compared using a log-rank test.
Difference in total one-third weeks abstinent was analyzed by two-sample t-test comparing group means. Descriptive statistics were used to summarize the number of modules completed by the TAU plus digital therapeutic group (mean, standard deviation, range).
Safety was evaluated using Fisher’s exact test for comparison of adverse events.
Statistical analyses were conducted by a study-independent statistician and performed in SAS, version 9.3 or higher (SAS Institute). No adjustments for multiple comparisons were made. The sample size determination for this study has been describedCitation12.
Results
Participant demographics
No significant differences were detected between treatment groups on any demographic variable (). Most participants were male (54.1%) and white (95.3%), with a mean age of 32.9 years. DSM-IV criteria for cocaine dependency was met in 21.5% of the TAU group and 15.4% of the TAU plus digital therapeutic group (difference not statistically significant).
Table 1. Participant demographic information.
Abstinence and retention
As shown in , participants randomized to TAU plus the digital therapeutic had a significant, increased likelihood of abstinence from opioids and cocaine during the last four weeks of treatment, compared to those randomized to TAU (75.9 vs. 60.6%; OR 2.05, 95% CI 1.07–3.90; p=.03). Similar improvements were observed for abstinence from opioids only, with a 77.3% likelihood of abstinence during weeks 9–12 for the TAU plus PDT group vs. 62.1% for the TAU group (OR 2.08, 95% CI 1.10–3.95; p=.02). Likewise, 82.4% of participants randomized to TAU plus digital therapeutic were likely to be abstinent from cocaine only in weeks 9–12 compared to 64.5% for those randomized to TAU (OR 2.58, 95% CI 1.37–4.86; p=.003).
Table 2. Abstinence from substance use.
A significant improvement in retention during the study was observed, with a hazard ratio of 0.49 (95% CI 0.26–0.92; p=.02) in favor of the TAU plus digital therapeutic group (). Treatment dropout rate was lower for the TAU plus digital therapeutic group (17.6%) compared to the TAU group (31.6%).
Figure 1. Kaplan-Meier estimates of time to withdrawal from treatment.
Secondary and exploratory endpoints
Total number of one-third weeks abstinent, and abstinence during the last 6 and 8 weeks of treatment are shown in . Participants randomized to TAU plus digital therapeutic achieved significantly more one-third weeks of abstinence (mean = 27.97, SD = 8.17) over the study duration compared to those randomized to TAU (mean = 24.06, SD = 11.89; p=.02). The TAU plus digital therapeutic group demonstrated a significantly increased likelihood of abstinence from opioids during the last 6 weeks of treatment compared to the TAU group (78.2 vs. 63.9%; OR 2.03, 95% CI 1.09–3.80; p=.03). Similarly, an increase in likelihood of abstinence was observed during the last 8 weeks of treatment, for participants who received TAU plus digital therapeutic compared to TAU (82.4 vs. 68.5%; OR 2.16, 95% CI 1.16–4.01; p=.01).
Participants randomized to TAU plus digital therapeutic completed a mean of 77.3 (SD = 32.36; range = 4–150) therapeutic modules (core + supplemental) over 12 weeks of treatment, including a mean of 42.2 (SD = 15.31; range = 4–78) core modules and a mean of 35.1 (SD = 17.59; range = 0–72) supplemental modules.
Safety
Observed adverse events were of the type and frequency anticipated in a population of patients with OUD (e.g. gastrointestinal, musculoskeletal, and psychiatric events) (). Overall, 112 participants reported an adverse event, including 57 (62.6%) in the TAU plus digital therapeutic group and 55 (69.9%) of the TAU participants. The proportion of participants reporting adverse events in each treatment group did not differ significantly (p=.42). No suicide-related events were reported. None of the adverse events observed were adjudicated to be device-related.
Table 3. Summary of adverse events.
Discussion
In this secondary analysis, participants randomized to the TAU plus digital therapeutic group exhibited significant improvements in abstinence in the last 4 weeks of treatment and retention in treatment. These results were consistent regardless of whether abstinence was defined as opioids alone, cocaine alone, or both opioids and cocaine. Abstinence from opioids was also improved in participants who received TAU plus the digital therapeutic during the last 5–8 weeks of treatment. Time to withdrawal from treatment was extended significantly in the TAU plus digital therapeutic group compared to the TAU group, demonstrating improved treatment retention.
Substance use disorder is a chronic, refractory disease, characterized by lapses and relapses, requiring lifelong managementCitation16. OUD is a particular challenge, given the high rates of relapse and life-threatening consequences associated with opioid overdoseCitation5. Successful OUD treatment has been hampered by issues of access, especially to evidence-based medications and behavioral interventions, as well as stigma and poor treatment outcomesCitation21. The therapeutic program evaluated in this study (and, by extension, reSET-O) may assist providers in addressing the opioid crisis, as it offers effective and accessible behavioral treatment. In particular, the way that the contingency management component of reSET-O is integrated into the therapeutic program and funded via the prescription cost, may address the observation that although CM has been shown to be very effective, it is often not implemented because of funding shortages or logistics at substance use disorder treatment centersCitation22.
The majority of individuals with substance use disorders consider abstinence an important aspect of recoveryCitation23,Citation24. Improvements in short-term abstinence are predictive of long-term outcomes, and each additional day of abstinence may be life-saving given the risk of fatal opioid overdoseCitation25,Citation26. The abstinence data presented here demonstrate that the PDT, when used in combination with buprenorphine MOUD, enabled a substantive improvement of a primary treatment objective, abstinence from opioid use. Furthermore, the PDT demonstrated a positive benefit/risk ratio, as improvement in the primary outcomes was observed without an increase in adverse events. This is seldom the case for pharmacotherapy and highlights the value of PDTs as a safe and effective treatment modality.
High treatment attrition rates (30% or higher) are still a major challenge facing OUD treatment providers and limit the effectiveness of careCitation27–29. Many patients discontinue treatment within the first monthCitation28,Citation30,Citation31. Interventions to improve retention are much needed given the difficulty of keeping patients in treatment and the risks of relapse and overdose for those who discontinue. The finding that a digital therapeutic significantly improved treatment retention further supports the value of incorporating such therapeutics into OUD treatmentCitation21,Citation26,Citation32–34.
The data presented here were the basis for FDA market-authorization of reSET-O as a Class II medical device based on the predicate reSET, the first FDA market-authorized PDT for treating substance use disorderCitation19. FDA market-authorization is important as it highlights independent review of safety and efficacy to support informed decision making by patients, healthcare payers and providers. There were 325,000 health and wellness apps available in 2017Citation35 – dozens related to substance use disorders. The majority of these apps are not clinically validated according to Good Clinical Practices, nor built according to Good Manufacturing Practices to ensure they function as intended. As a result, healthcare providers and patients have little guidance on which apps effectively treat OUD and should be integrated into clinical practice. Market-authorization by a regulatory body is the gold standard by which PDTs should be evaluated and on which labeling that clearly describes the indications and intended use of the therapeutic should be based.
Limitations
The present study has some limitations. The clinical trial was conducted at a single site, with a small sample of primarily Caucasian males. Thus, the sample may not be representative of all individuals seeking treatment for OUD. Abstinence rates were generally high in both treatment groups, which may have resulted from the combination of medication and CM for all participants and may have partially masked the impact of the PDT. The study was open-label, hence all parties were aware of the treatment interventions. However, an unblinded design is consistent with prior randomized studies assessing effectiveness of digital interventions for substance use disorders, in which participants and study staff are aware of the treatment assignmentCitation10,Citation12,Citation15. Finally, no follow-up of the participants was conducted subsequent to the 12-week intervention period, hence it is unclear how long participants may benefit after discontinuing treatment.
Conclusions
A digital therapeutic in combination with buprenorphine MOUD improves clinically significant patient outcomes including abstinence from illicit opioids and retention in treatment compared with treatment as usual. PDTs such as reSET-O may enhance treatment outcomes and clinical care, particularly in geographic regions without ready availability of clinics or trained staff, or in times when in-person access is limited, such as is the case with the current COVID-19 pandemic. Given the accessibility of mobile devices, reSET-O has the potential to meet many of the unmet needs in the current treatment delivery system for patients with OUD.
Transparency
Declaration of funding
Clinical trial (NCT00929253) was sponsored by [grant R01DA012997-10] (Warren K. Bickel) from the National Institute on Drug Abuse and the Wilbur Mills Endowment, without financial support from Pear Therapeutics. The statistical analysis presented herein was funded by Pear Therapeutics, which develops reSET-O.
Declaration of financial/other relationships
YAM and HFL are employees of Pear Therapeutics. WKB is a principal of HealthSim, LLC; NotifiUS, LLC; BEAM Diagnostics, Inc; and Red 5 Group, LLC; serves on the scientific advisory board for Sober Grid, Inc.; Ria Health; and is a consultant for Alkermes, Inc.; Nektar Therapeutics; and Sandoz, Inc. LAM is affiliated with HealthSim, LLC; Red 5 Group, LLC; Square2 Square Systems, Inc. and is a scientific advisory board member of Pear Therapeutics. KG has no conflicts of interest to disclose. JB is a consultant of Pear Therapeutics. Peer reviewers on this manuscript have received an honorarium from CMRO for their review work but have no other relevant financial relationships to disclose.
Acknowledgements
Writing and editorial support was provided by Dr. Nicole Enman and Mr. Stephen Braun (Pear Therapeutics) and Ms. Kate Jones (Global MedCom Consulting, LLC). Data transfer assistance was provided by Eric Shen (Pear Therapeutics).
References
- Substance Abuse and Mental Health Services Administration. 2019 National Survey of Drug Use and Health (NSDUH) releases [cited 2020 Sep 29]. Available from: https://www.samhsa.gov/data/release/2019-national-survey-drug-use-and-health-nsduh-releases.
- Centers for Disease Control and Prevention. Wide-ranging online data for epidemiological research (WONDER). National Center on Health Statistics, Center for Disease Control and Prevention [Published 2018. cited 2019 Jul 29]. https://wonder.cdc.gov.
- The ASAM National Practice Guideline for the Treatment of Opioid Use Disorder: 2020 Focused Update. J Addict Med. 2020;14(2S Suppl 1):1–91. DOI:10.1097/ADM.0000000000000633.
- Center for Behavioral Health Statistics and Quality. 2017 National survey on drug use and health final analytic file codebook. Rockville (MD): Substance Abuse and Mental Health Services Administration; 2018.
- National Academies of Sciences Engineering and Medicine. Medications for opioid use disorder save lives. Washington, DC: The National Academies Press; 2019.
- Marsch LA, Dallery J. Advances in the psychosocial treatment of addiction: the role of technology in the delivery of evidence-based psychosocial treatment. Psychiatr Clin North Am. 2012;35(2):481–493.
- Pincus HA, England MJ. Improving the quality of psychosocial interventions for mental and substance use disorders: a report from the IOM. JAMA.2015;314(12):1227–1228.
- Budney AJ, Higgins ST. A community reinforcement plus vouchers approach: treating cocaine addiction (NIH Publication Number 98-4309). Rockville (MD): National Institute on Drug Abuse; 1998.
- Hunt GM, Azrin NH. A community-reinforcement approach to alcoholism. Behav Res Ther. 1973;11(1):91–104.
- Bickel WK, Marsch LA, Buchhalter AR, et al. Computerized behavior therapy for opioid-dependent outpatients: a randomized controlled trial. Exp Clin Psychopharmacol. 2008;16(2):132–143.
- Campbell AN, Nunes EV, Matthews AG, et al. Internet-delivered treatment for substance abuse: a multisite randomized controlled trial. Am J Psychiatry. 2014;171(6):683–690.
- Christensen DR, Landes RD, Jackson L, et al. Adding an internet-delivered treatment to an efficacious treatment package for opioid dependence. J Consult Clin Psychol. 2014;82(6):964–972.
- Stitzer M, Petry N. Contingency management for treatment of substance abuse. Annu Rev Clin Psychol. 2006;2:411–434.
- Bickel WK, Amass L, Higgins ST, et al. Effects of adding behavioral treatment to opioid detoxification with buprenorphine. J Consult Clin Psychol. 1997;65(5):803–810.
- Marsch LA, Guarino H, Acosta M, et al. Web-based behavioral treatment for substance use disorders as a partial replacement of standard methadone maintenance treatment. J Subst Abuse Treat. 2014;46(1):43–51.
- National Institute on Drug Abuse. Principles of drug addiction treatment. A research based guide. Rockville (MD): National Institutes of Health, US Department of Health and Human Services; 2012. p. 1–75.
- Center for Substance Abuse Treatment. Clinical guidelines for the use of buprenorphine in the treatment of opioid addiction. Treatment improvement protocol (TIP) series 40. DHHS Publication No. (SMA) 04-3939. Rockville (MD): Substance Abuse and Mental Health Services Administration; 2004.
- Marsch LA, Bickel WK, Badger GJ, et al. Buprenorphine treatment for opioid dependence: the relative efficacy of daily, twice and thrice weekly dosing. Drug Alcohol Depend. 2005;77(2):195–204.
- Center for Devices and Radiological Health. De Novo summary for reSET (DEN160018). Washington, DC: US Food and Drug Administration; 2017.
- Clinical Trials Network. Treatment Effect & Assessment Measures (TEAM) Task Force Recommendations. 2010 [cited 2020 Nov 6]. Available from: http://ctndisseminationlibrary.org/PDF/522.pdf
- Hser YI, Evans E, Huang D, et al. Relationship between drug treatment services, retention, and outcomes. Psychiatr Serv. 2004;55(7):767–774.
- Cunningham C, Stitzer M, Campbell AN, et al. Contingency management abstinence incentives: cost and implications for treatment tailoring. J Subst Abuse Treat. 2017;72:134–139.
- Kaskutas LA, Borkman TJ, Laudet A, et al. Elements that define recovery: the experiential perspective. J Stud Alcohol Drugs. 2014;75(6):999–1010.
- Laudet AB. What does recovery mean to you? Lessons from the recovery experience for research and practice. J Subst Abuse Treat. 2007;33(3):243–256.
- Dennis ML, Foss MA, Scott CK. An eight-year perspective on the relationship between the duration of abstinence and other aspects of recovery. Eval Rev. 2007;31(6):585–612.
- Weisner C, Ray GT, Mertens JR, et al. Short-term alcohol and drug treatment outcomes predict long-term outcome. Drug Alcohol Depend. 2003;71(3):281–294.
- Hser YI, Evans E, Huang D, et al. Long-term outcomes after randomization to buprenorphine/naloxone versus methadone in a multi-site trial. Addiction. 2016;111(4):695–705.
- Hser YI, Saxon AJ, Huang D, et al. Treatment retention among patients randomized to buprenorphine/naloxone compared to methadone in a multi-site trial. Addiction. 2014;109(1):79–87.
- Weiss RD, Potter JS, Fiellin DA, et al. Adjunctive counseling during brief and extended buprenorphine-naloxone treatment for prescription opioid dependence: a 2-phase randomized controlled trial. Arch Gen Psychiatry. 2011;68(12):1238–1246.
- Soeffing JM, Martin LD, Fingerhood MI, et al. Buprenorphine maintenance treatment in a primary care setting: outcomes at 1 year. J Subst Abuse Treat. 2009;37(4):426–430.
- Stein MD, Cioe P, Friedmann PD. Buprenorphine retention in primary care. J Gen Intern Med. 2005;20(11):1038–1041.
- Hubbard RL, Craddock SG, Anderson J. Overview of 5-year followup outcomes in the drug abuse treatment outcome studies (DATOS). J Subst Abuse Treat. 2003;25(3):125–134.
- Simpson DD. The relation of time spent in drug abuse treatment to posttreatment outcome. Am J Psychiatry. 1979;136(11):1449–1453.
- Zhang Z, Friedmann PD, Gerstein DR. Does retention matter? Treatment duration and improvement in drug use. Addiction. 2003;98(5):673–684.
- Research2Guidance. mHealth economics 2017—current status and future trends in mobile health [Published 2017; cited 2020 Oct 15]. https://research2guidance.com/product/mhealth-economics-2017-current-status-and-future-trends-in-mobile-health/.