Mechanisms of change in Internet- and mobile-based interventions for PTSD: a systematic review and meta-analysis

ABSTRACT

Background: While Internet- and mobile-based interventions (IMIs) are potential options to increase the access to evidence-based therapies for post-traumatic stress disorder (PTSD), comprehensive knowledge on their working mechanisms is still scarce.

Objective: We aimed to evaluate studies investigating the efficacy and mechanisms of change in IMIs for adults with PTSD.

Method: In this systematic review and meta-analysis (PROSPERO CRD42019130314), five databases were consulted to identify relevant studies, complemented by forward (i.e. citation search) and backward (i.e. review of reference lists from included studies) searches. Randomized controlled trials (RCTs) investigating the efficacy of IMIs compared to active controls, as well as component and mediation studies were included. Two independent reviewers extracted the data and assessed the risk of bias and requirements for process research. Random-effects meta-analyses on PTSD symptom severity as primary outcome were conducted and further information was synthesized qualitatively.

Results: In total, 33 RCTs were included (N = 5421). The meta-analysis comparing IMIs to non-bonafide active controls yielded a significant standardized mean difference (SMD) of −0.36 (95%CI −0.53 to −0.19) favouring IMIs. Although meta-analytic pooling was not possible for the component and mediation studies, evidence suggests no differential effects regarding PTSD symptom reduction between different levels of support and personalization and between different types of exposure. Moreover, mediation studies revealed significant intervening variable effects for self-efficacy beliefs, perceived physical impairment, social acknowledgement, and trauma disclosure.

Conclusions: Results indicate that IMIs for PTSD are superior to active controls. Furthermore, findings may contribute to the development of new interventions by outlining important directions for future research (e.g. regarding requirements for process research) and highlighting potential mechanisms of change (i.e. self-efficacy, perceived physical impairment, social acknowledgement, and trauma disclosure).

HIGHLIGHTS

• Knowledge on mechanisms of change in psychothera-pies is crucial as it contributes to intervention development and improved outcomes. • This review suggests that digital interventions for PTSD are efficacious; yet, more studies are needed to enhance understanding on how they work.

Antecedentes: Si bien las intervenciones basadas en Internet y dispositivos móviles (IMI) son opciones potenciales para aumentar el acceso a terapias basadas en la evidencia para el trastorno de estrés postraumático (TEPT), el conocimiento integral sobre sus mecanismos de acción aún es limitado.

Objetivo: Nos propusimos evaluar estudios que investigan la eficacia y los mecanismos de cambio en los IMI para adultos con TEPT.

Método: En esta revisión sistemática y metanálisis (PROSPERO CRD42019130314), se consultaron cinco bases de datos para identificar estudios relevantes, complementados con búsquedas hacia adelante (es decir, búsqueda de citas) y hacia atrás (es decir, revisión de listas de referencias de estudios incluidos). Se incluyeron ensayos controlados aleatorios (ECA) que investigaban la eficacia de los IMI en comparación con los controles activos, así como los estudios de componentes y mediación. Dos revisores independientes extrajeron los datos y evaluaron el riesgo de sesgo y requerimientos para investigación de proceso. Se realizaron metanálisis de efectos aleatorios sobre la severidad de los síntomas del TEPT como resultado primario y se sintetizó cualitativamente información adicional.

Resultados: En total, se incluyeron 33 ECA (N = 5421). El metanálisis que comparó los IMI con los controles activos no fiables produjo una diferencia de medias estandarizada (DME) significativa de −0,36 (IC del 95%: −0,53 a −0,19) a favour de los IMI. Aunque no fue posible la agrupación metaanalítica para los estudios de componentes y mediación, la evidencia sugiere que no hay efectos diferenciales con respecto a la reducción de los síntomas del TEPT entre diferentes niveles de apoyo y personalización y entre diferentes tipos de exposición. Más aún, los estudios de mediación revelaron efectos variables significativos, para las creencias de autoeficacia, la discapacidad física percibida, el reconocimiento social y la revelación del trauma.

Conclusiones: Los resultados indican que los IMI para el PTSD son superiores a los controles activos. Más aún, los hallazgos podrían contribuir al desarrollo de nuevas intervenciones al delinear direcciones importantes para la investigación futura (por ejemplo, con relación a los requerimientos para la investigación de procesos) y resaltar los potenciales mecanismos de cambio (es decir, la autoeficacia, la discapacidad física percibida, el reconocimiento social y la revelación del trauma).

背景: 尽管基于互联网和移动设备的干预措施 (IMI) 是提高创伤后应激障碍 (PTSD) 循证疗法获得途径的潜在选择, 但仍缺乏其工作机制相关的全面知识。

目的: 我们旨在评估在患有PTSD的成人中考查IMI疗效与改变机制的研究。

方法: 在本系统综述和元分析 (PROSPERO CRD42019130314) 中, 参考了五个数据库来识别相关研究, 并以正向搜索 (即引文检索) 和反向搜索 (即对包含研究的参考文献列表进行回顾) 作为补充。纳入了相较于活性对照考查IMI疗效的随机对照试验 (RCT), 以及成分和中介研究。两名独立的审阅者提取了数据并评估了误差风险和过程研究的要求。进行了以PTSD症状严重程度为主要结果的随机元分析, 并对更多信息进行了定性综合。

结果: 总共纳入了33个RCT (N = 5421) 。对比IMI与非亲和活性对照的元分析得到了IMI −0.36 (95％CI −0.53至-0.19) 的显著标准化均值 (SMD) 优势。尽管元分析合并不可能对成分和中介研究进行, 但证据表明, 在不同程度的支持和个性化之间以及不同类型的暴露之间, PTSD症状减轻的疗效没有差异。此外, 中介研究表明, 自我效能感, 感知到的躯体损伤, 社交认可和创伤披露是显著的中介变量。

结论: 结果表明PTSD的IMI优于活性对照。此外, 通过概述未来研究的重要方向 (例如关于过程研究的要求) 并强调潜在改变机制 (即自我效能, 感知到的躯体损伤, 社交认可和创伤披露), 研究结果可能有助于新兴干预措施的开发。

KEYWORDS:

• Digital health
• trauma
• efficacy
• mediators
• component studies

PALABRAS CLAVE:

• Salud digital
• trauma
• eficacia
• mediadores
• estudios de componentes

关键词:

• 数字健康
• 创伤
• 疗效
• 中介因素
• 成分研究

Post-traumatic stress disorder (PTSD) is a prevalent mental disorder associated with significant personal suffering and substantial disease burden worldwide (Bryant, 2019; McMillen, North, Mosley, & Smith, 2002). Although the evidence-base for effective psychotherapeutic face-to-face treatments for this condition is well-established (Cusack et al., 2016), only a fraction of affected patients receive evidence-based treatments (Finch, Ford, Grainger, & Meiser-Stedman, 2020). Reasons for this include fear of stigmatization, geographical barriers, and a shortage of mental healthcare providers (Bryant, 2019; Finch et al., 2020). Using the Internet as a medium for the delivery of psychological interventions may help to overcome these barriers (Andersson & Titov, 2014).

Internet- and mobile-based interventions (IMIs) have been found to be efficacious both for mental disorders in general (Carlbring, Andersson, Cuijpers, Riper, & Hedman-Lagerlöf, 2018; Domhardt, Steubl, et al., 2018; Domhardt, Geßlein, von Rezori, & Baumeister, 2019) and PTSD in particular (Kuester, Niemeyer, & Knaevelsrud, 2016; Sijbrandij, Kunovski, & Cuijpers, 2016; Simblett, Birch, Matcham, Yaguez, & Morris, 2017). Furthermore, these technology-delivered interventions may be easy to access and offer patients the opportunity to flexibly integrate psychological treatments with their daily lives (Domhardt, Ebert, & Baumeister, 2018; Domhardt, Steubl, & Baumeister, 2018). Providing geographically barrier-free mental healthcare to underserved post-conflict areas as well as to low- and middle-income countries may be particularly relevant in case of PTSD as exposure to potentially traumatic events is a rather common phenomenon in these countries, and only a small number of their citizens with PTSD have access to treatment (Barbui & Tansella, 2013; Wagner, Schulz, & Knaevelsrud, 2012). Moreover, IMIs have been found to be scalable and cost-effective (Andersson & Titov, 2014; Domhardt et al., 2019). Users of IMIs frequently list anonymity, privacy, and confidentiality among the reasons why they prefer to use the Internet for mental health support (Horgan & Sweeney, 2010). However, aside from these presumed advantages, possible risks concerning these interventions (e.g. non-detected negative outcomes, unclear crisis management in case of anonymous users) should be carefully investigated, as this knowledge is largely pending (Andersson & Titov, 2014).

Gaining a better insight on how and whereby these interventions work is especially important to inform mental healthcare practices and intervention development, which strive to enhance augmented outcomes and intervention safety (Domhardt et al., 2019). For this purpose, knowledge on the mechanisms of change (i.e. the processes or events that are causally responsible for treatment change) can be obtained by three research approaches: 1) evaluating the efficacy against different active comparison conditions, 2) assessing the effects of single intervention components, and 3) identifying mediators. Additionally, a closer investigation of adherence rates may complement these approaches.

First, IMIs can be compared with either active or inactive control conditions, to test their general efficacy as a fundamental prerequisite for process research (Kazdin, 2007). However, studies with inactive control conditions yield limited conclusions on the relationship of treatment effects associated with specific components (Karlsson & Bergmark, 2015). Therefore, this systematic review and meta-analysis focuses on studies with active control conditions to underline the role and importance of effective intervention components (i.e. specific factors) in IMIs. Subsequently, active controls can be further distinguished as bonafide (i.e. treatments that were intended to be therapeutic; Wampold et al., 1997) and non-bonafide (i.e. non-therapeutic conditions, e.g. attention/psychological placebos) control conditions. The former condition additionally allows for the crucial investigation of the non-inferiority of the IMIs as compared with other (established evidence-based) interventions.

Second, component studies enable the determination of active components (e.g. human support, exposure) and the assessment of the incremental (add-on) effects of specific components (Bell, Marcus, & Goodlad, 2013). Component studies consist of trial designs that compare the full-treatment package with the same intervention, in which either one of the specific components is left out (i.e. dismantling studies) or added (i.e. additive design studies). To date, there is only evidence for the effect-contributing role of guidance in IMIs, while evidence for the incremental effects of other components is still absent (Domhardt et al., 2019; Sijbrandij et al., 2016). Thus, it is of great clinical significance to gain further insight into other components to facilitate the development of empirically-supported, efficient, and cost-efficient interventions.

Third, the mechanisms of change can be further evaluated by mediator analyses (e.g. change in cognitions; Kazdin, 2007). In line with previous research (Cuijpers, Reijnders, & Huibers, 2019; Domhardt et al., 2021), mediators are conceptualized as intervening variables that may statistically account for the relationship between the independent variable and the outcome. Thereby, it is necessary to perform a well-established mediation analysis with quantitative evaluations of the changes. A recent systematic review investigating the mechanisms of change in face-to-face psychological interventions for PTSD found evidence that improvements in maladaptive trauma-related beliefs and appraisals are core mechanisms of change (Kangaslampi & Peltonen, 2019). The authors found the strongest evidence for this mechanism in well-established and evidence-based cognitive behavioural therapy (CBT) interventions (i.e. cognitive processing therapy, trauma-focused CBT, prolonged exposure). Additionally, the results show preliminary evidence for increases in mindfulness as a mechanism of change in mindfulness and spiritually oriented interventions. Given the results of this review and the heterogeneity of IMIs for PTSD (e.g. executive functioning training, expressive writing, self-efficacy training), there may be a broad range of possible mechanisms of change in IMIs for PTSD (e.g. improvements in maladaptive cognitions, attention processes, coping strategies and emotion regulation, self-efficacy, social acknowledgement, trauma-disclosure, traumatic memory, mindfulness), warranting an explorative approach in the current study.

Besides these three approaches that focus on symptom severity as the central outcome, adherence can be regarded as another important approach considering the high dropout rates in IMIs for PTSD (e.g. 51.2% post-intervention; Cieslak et al., 2016). As dropout is considered a crucial problem in IMIs (Steele, Mummery, & Dwyer, 2007), this systematic review will also concentrate on dropout rates to inform future intervention development by gathering knowledge on components contributing to increased adherence rates.

To our knowledge, three meta-analyses (Kuester et al., 2016; Sijbrandij et al., 2016; Simblett et al., 2017) have assessed Internet-based interventions for PTSD so far and found evidence for their efficacy. However, they did not distinguish between bonafide and non-bonafide active controls and no systematic review on IMIs for PTSD has examined the components in additive or dismantling design studies up to this point. Previous meta-analyses provided mainly indirect results suggesting potentially larger effect sizes of IMIs with therapeutic support compared to those without (Sijbrandij et al., 2016). Besides, they reported no evidence for other relevant active components (Kuester et al., 2016; Simblett et al., 2017) and included predominantly passive control conditions. The same applies to mediators that have neither been systematically reviewed nor meta-analytically integrated so far. Moreover, mobile-based applications were excluded in two of the previous studies (Kuester et al., 2016; Sijbrandij et al., 2016).

Thus, this review and meta-analysis aims to complement and extend the current evidence-base by adhering to the following research questions:

1. Are IMIs for PTSD efficacious in reducing PTSD symptom severity?

   1. Are IMIs for PTSD equally efficacious when compared to bonafide control conditions/interventions?

   2. Are IMIs for PTSD more efficacious than non-bonafide active control conditions?

2. Which components are responsible for the efficacy of IMIs for PTSD?

3. What potential mediators have been examined so far in IMIs for PTSD?

1. Method

1.1. Registration

This review and meta-analysis is registered with PROSPERO (CRD42019130314) and is reported according to the PRISMA guidelines (Liberati et al., 2009). Further detailed information can be found in the study protocol (Steubl, Sachser, Baumeister, & Domhardt, 2019).

1.2. Search strategy and selection criteria

We conducted systematic searches as detailed in the study protocol comprising four different steps (Steubl et al., 2019). The predefined sets of search strings specific for each database are outlined in Supplementary Tables A.1 and A.2. The electronic database search conducted on 8 November 2020 yielded 7093 records. One additional record was identified through backward searches. After removing duplicates, we screened the remaining 4653 titles and abstracts and included 128 articles into our full-text screening, resulting in 33 studies meeting our eligibility criteria (Supplemental material G). In detail, 27 studies were included for assessing efficacy compared to active control conditions, five for assessing components, and three for assessing mediators. One study (Cieslak et al., 2016) was included for both the evaluation of efficacy and mediators, and one (Niles et al., 2020) for both the evaluation of efficacy and components. The ICTRP and Web of Science search yielded no additional included studies. The corresponding flow chart is presented in Figure 1.

Figure 1. Flow chart

Studies were eligible for inclusion, if (a) they focused on an adult target population (≥18 years), (b) they included participants with PTSD or subthreshold PTSD measured by a standardized, clinician-administered instrument or a validated self-report measure of PTSD symptoms, and (c) at least one of their trial arms investigated an IMI for PTSD. In this respect, IMIs are defined as psychological/psychotherapeutic interventions delivered via Internet- and mobile-based communication technologies (Barak, Klein, & Proudfoot, 2009). Additionally, studies had to be written in English or German and either be published in a peer-reviewed journal or classified as an ongoing trial in ICTRP with already available results.

To investigate the efficacy of IMIs for PTSD and active components, only randomized controlled trials (RCTs) were included (Higgins & Green, 2011). To study potential mediator variables, secondary analyses of previous RCTs were eligible as well. These trials had to include repeated measures (i.e. longitudinal) of mediators and outcomes, and either deploy well-established mediation analyses (Baron & Kenny, 1986) or include a quantitative assessment of changes in the examined psychological mediators. Possible mediators had to be measured by validated psychometric instruments.

Eligible comparisons differed by each research question. First, to investigate the efficacy of IMIs for PTSD, studies must have included an active (e.g. IMI, face-to-face, treatment-as-usual (TAU), or placebo) control group. Moreover, to be classified as bonafide (Wampold et al., 1997) the treatment for the control group had to aim at reducing symptom severity (i.e. non-inferiority trials). If the authors postulated the ineffectiveness of the control group, it was classified as non-bonafide. In case the authors did not specify their hypotheses on the efficacy of the comparison group clearly, the allocation was decided depending on existing empirical evidence for the efficacy of the control treatment documented in other studies. Second, to investigate components, eligible trials must have been classified as additive or dismantling design studies. Third, to investigate mediators, studies were eligible for inclusion if they compared an IMI group with either an active or non-active (e.g. wait-list or no intervention) control group. TAU comparison conditions were classified as active control groups following the recommendations of Burns (2009).

To assess participants’ engagement with the online intervention, adherence was operationalized as (a) the mean number of main intervention units completed (e.g. modules), (b) the percentage of participants that completed the whole treatment, and (c) dropout rates at post-treatment.

To evaluate post-intervention symptom reduction, separate analyses for different periods of assessment were performed: (a) short-term (1–3 months after post-treatment), (b) medium-term (>3 ≤ 12 months after post-treatment), and (c) long-term (>12 months after post-treatment) effects.

1.3. Data analysis

Missing standard deviations were computed using the standard deviations per group and Cohen’s (1988) alternative formula for calculating a pooled standard deviation, which contains the individual group sample sizes. Further study-specific details relevant for data-analyses are outlined in Supplemental Material C.

In general, if outcomes were assessed by several instruments (a) the primary outcome instrument of the study was prioritized and (b) in case of multiple outcome instruments of the same hierarchical level, the outcome instrument most often used across all included studies was chosen for the meta-analysis.

In order to evaluate the quality of included studies, pairs of two independent reviewers (LS, CD, AS, CR) assessed the risk of bias using the Cochrane Collaboration’s tool for assessing risk of bias in RCTs (Higgins et al., 2011). Accordingly, each study was rated (i.e. “low’, ‘high’, or ‘unclear’ risk of bias) on the following domains: (a) random sequence generation, (b) allocation concealment, (c) blinding of participants and personnel, (d) blinding of outcome assessment, (e) incomplete outcome data, and (f) other bias (e.g. deviations from the study protocol). It is important to note that the third domain of blinding of participants and personnel is often not feasible in (guided) IMIs. To prevent a distorted rating, this domain was rated as ‘unclear’ in case of missing blinding in guided interventions. The inter-rater reliability was calculated by means of Cohen’s Kappa. A value between 0.60 and 0.80 was considered as substantial and a value >0.80 as (almost) perfect (Landis & Koch, 1977). Additionally, in order to assess the aptitude of included studies to detect mechanisms of change and to approach causality, we rated mediation studies qualitatively as predefined in the study protocol (Steubl et al., 2019), employing the criteria originally proposed by Kazdin (2007), which were adapted to psychotherapy research by Lemmens, Müller, Arntz, and Huibers (2016). Additionally, in the context of psychotherapy process research it is beneficial that assessed mediators are deduced theoretically and nested within a plausible theoretical framework (Kazdin, 2007; Kraemer, Wilson, Fairburn, & Agras, 2002). Thus, the derivation of the theoretical background was rated as a seventh criterion in line with Domhardt et al. (2021).

Meta-analytic pooling was performed using the Review Manager 5.3 software (The Cochrane Collaboration, 2014). Additional statistical analyses were computed employing R (R Core Team, 2017). Standardized Mean Differences (SMDs) and their 95% confidence intervals (CI) were calculated. As Hedges’ g is less biased than Cohen’s d in small samples (Borenstein, Hedges, Higgins, & Rothstein, 2009), SMDs are presented as values of the former measure.

Due to the amount of missing standard deviations and the recommendation to use imputation methods sparingly (Higgins & Green, 2011), post-test data was employed even though some trials in this meta-analysis reported differences in baseline values (de Kleine et al., 2019; Krupnick et al., 2017) or did not test for differences (Clausen et al., 2019; Kahn, Collinge, & Soltysik, 2016). Meta-regression was used to test the influence of excluding baseline data.

Statistical heterogeneity was evaluated with the I2- and Cochran’s Q-Statistics (Higgins & Green, 2011). Forest plots were created and used to visually investigate the presence and nature of heterogeneity. To examine possible publication bias, the trim and fill procedure (Duval & Tweedie, 2000), Egger’s test of funnel plot asymmetry (Egger, Smith, et al., 1997), and visual inspection of funnel plots were utilized.

For further comparisons of intervention and study characteristics (concerning e.g. control conditions, target population), subgroup analyses were performed when feasible. Sensitivity analyses were conducted to test the robustness of the results by comparing the pooled SMD of the different times of outcome assessment, and to examine the effect of including studies at high risk of bias. In case a quantitative synthesis was not appropriate, results are summarized qualitatively.

The strength of the body of included evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (Guyatt et al., 2008), summarizing the following five factors: (a) limitations of the studies’ design or execution, (b) consistency of results, (c) directness of evidence, (d) precision of results, and (e) publication bias. The GRADE system classifies the quality of evidence in four levels ranging from ‘very low’ and ‘low’ to ‘moderate’ and ‘high’ and the overall quality is downgraded by one level, if one of the factors is not met.

2. Results

2.1. Study characteristics

A detailed overview of the study characteristics is outlined in Supplementary Tables B.1-B.3. The majority of the included studies (k = 30; 90.9%) were conducted in western countries and assessed an Internet-based intervention (k = 27; 81.8%). One study evaluated a combined intervention which included Internet- and mobile-based components (Kahn et al., 2016). Most of the included studies evaluated interventions that adhered to the principles of CBT (k = 12; 36.4%), followed by interventions that were based on cognitive training (e.g. neurocognitive training; k = 8; 24.2%), therapeutic writing (k = 6; 18.2%), and self-efficacy (k = 2; 6.1%). The number of intervention modules ranged from 3 to 60 (M = 12.85; SD = 14.89) and their intervention duration from 0.5 to 24.0 weeks (M = 7.40; SD = 5.75).

In total, N = 5421 participants (female: 51.7%) were included in primary studies with a mean sample size of n = 164 (SD = 268.0). The number of participants ranged from n = 20 to 1292 per study. The mean age of participants ranged from 22.0 to 52.7 (M = 38.18; SD = 6.97) years. Recruitment via healthcare providers was the most frequently reported setting (k = 13; 39.4%), followed by (online) advertisements (k = 12; 36.4%) and organizations related to the institution conducting the study (k = 4; 12.1%). The majority of studies (k = 19; 57.6%) were published within the past five years and 45.5% of the studies involved veterans (k = 15).

The most frequently used measurement for PTSD severity was a version of the PTSD Checklist (PCL; k = 16; 48.5%) followed by the revised version of the Impact of Events Scale (IES-R; k = 5; 15.2%) and the Clinician Administered PTSD Scale (CAPS; k = 5; 15.2%). In sum, the majority of studies (k = 28; 84.8%) deployed only self-report measurements. Dropout rates varied from 1.0% to 51.2% (M = 22.13%; SD = 13.74%). Other adherence rates were reported heterogeneously, impeding an overall summary except for dropout rates. However, extracted information are detailed in Supplementary Tables B.1-B.3.

2.2. Risk of bias assessment

In total, 22 studies (66.7%) had at least one domain rated with ‘high’ with risk of performance bias (i.e. flawed blinding of participants and personnel) being the most common. Results of the risk of bias assessments of the included primary studies are summarized in Figure 2. Individual ratings for each study are presented in Supplementary Figure D. The inter-rater reliability showed almost perfect agreement (Cohen’s kappa = 0.91).

Figure 2. Risk of bias graph

2.3. Efficacy studies

A total of 27 eligible efficacy studies were identified. The majority of these studies implemented TAU (e.g. routine psychiatric clinical care, coordinated services provided by multidisciplinary teams; k = 7; 25.9%), placebo training (k = 8; 29.6%), or control writing (i.e. neutral writing tasks; k = 5; 18.5%) as control conditions.

2.3.1. Efficacy compared to bonafide controls

Two studies (Eisma et al., 2015; Littleton, Grills, Kline, Schoemann, & Dodd, 2016) were classified as investigating bonafide active controls. Littleton et al. (2016) compared a self-help website with a therapist-facilitated program and found evidence for differences depending on symptom severity. It was observed that less affected participants benefited most from self-help offers, while the therapist-facilitated interactive program was more suitable for participants with higher symptom severity. The second study (Eisma et al., 2015) classified as a trial with a bonafide control condition compared Internet-delivered exposure with Internet-delivered behavioural activation. The results showed no significant differences in PTSD symptom reduction between the two groups. Meta-analytic pooling was not feasible due to the limited number of studies.

2.3.2. Efficacy compared to non-bonafide active controls

The majority of studies investigating efficacy deployed non-bonafide comparison conditions (k = 25, 92.6%).

Overall efficacy A total of 21 studies reported results on between-group differences in symptom severity at post-intervention. The SMDs ranged from −2.32 to 0.69, resulting in a significant pooled SMD of −0.36 (95%CI −0.53 to −0.19), favouring IMIs when compared to non-bonafide active controls (Figure 3). Statistical heterogeneity was substantial. Based on the GRADE approach, we downgraded the level of evidence two levels, from high to low, to account for the limitations of the included studies (i.e. risk of bias) and heterogeneity in the results.

Figure 3. Forest Plot for overall efficacy IMI vs. Active control

Publication bias In line with the visual inspection of the funnel plot, the regression test for funnel plot asymmetry indicated no significant asymmetry (t(23) = −1.37, p = 0.159). The resulting contour-enhanced funnel plot presented in Supplementary Figure F also suggests no missing studies. The trim and fill method estimated two missing studies (SMD of −0.24, 95%CI −0.43 to −0.06).

Further analyses To test the influence of baseline differences, baseline between-group effects were included as a predictor. Results indicate that they were not significantly associated with the post-test effect sizes (p = 0.419). Both short- (SMD of −0.39, 95%CI −0.67 to −0.10) and medium-term (SMD of −0.20, 95%CI −0.38 to −0.01) follow-up results yielded a pooled SMD significantly favouring IMIs. There were no results available for long-term outcomes. Results remained comparable (SMD of −0.36, 95%CI −0.55 to −0.17) in a sensitivity analysis, excluding the trial by Sayer et al. (2015), for which means and standard deviations were only estimated based on medians and interquartile ranges. Additionally, a sensitivity analysis was also performed with the apparent outlier Beatty, Koczwara, and Wade (2016), resulting in a pooled SMD of −0.29 (95%CI −0.43 to −0.14). Excluding studies with (possible) between-group differences for symptom severity at baseline (Clausen et al., 2019; de Kleine et al., 2019; Kahn et al., 2016; Krupnick et al., 2017) resulted in a pooled SMD of −0.29 (95%CI −0.48 to −0.11), favouring IMIs (Supplementary Figures E.1 and E.2).

When considering different control conditions, pooled SMDs reached significance exclusively for TAU comparisons and the subgroup of CBT interventions was the only one with significant pooled effects when distinguishing between therapeutic backgrounds. Both subgroups, the self-report measurements only and the clinician-administered measurements only, reached significant pooled effects favouring IMIs. The same applies to the SMDs when including only low dropout studies and only studies with a veteran population. Excluding studies with high risk of bias also resulted in a significant pooled effect. Pooled SMDs for subgroup and sensitivity analyses can be found in Table 1. The corresponding forest plots are detailed in Supplementary Figures E.3-E.15.

Table 1. Subgroup and sensitivity analyses

Subgroup		No. of included studies	Pooled SMD^a	95%CI	I^2	Q (df, p)
Control conditions
	IMI vs. TAU	7	−0.27*	−0.51, −0.02	74%	22.82* (6, <0.001)
	IMI vs. Control writing	4	−0.16	−0.53, 0.21	76%	6.71* (3, 0.082)
	IMI vs. Psychoeducation	4	−0.82	−2.21, 0.57	95%	41.55* (2, <0.001)
Therapeutic backgrounds
	IMI (CBT only) vs. Control	6	−0.54*	−0.98, 0.10	92%	60.62* (5, <0.001)
	IMI (writing only) vs. Control	5	−0.26	−0.61, 0.08	74%	15.11* (4, 0.004)
	IMI (training only) vs. Control	5	−0.32	−0.67, 0.03	56%	9.02 (4, 0.06)
Type of symptom severity rating
	IMI vs. Control (self-report measurements only)	17	−0.38*	−0.58, −0.18	86%	111.24* (16, <0.001)
	IMI vs. Control (clinician-administered measurements only)	4	−0.31*	−0.48, −0.14	0%^b	0.95 (3, 0.813)
Adherence
	IMI vs. Control (low dropout only^c)	12	−0.55*	−0.84, −0.26	83%	64.67* (11, <0.001)
Population
	IMI vs. Control (veterans only)	9	−0.27*	−0.51, −0.03	68%	25.29* (8, 0.001)
Sensitivity analyses
	IMI vs. Control (studies with high risk of bias excluded^d)	20	−0.37*	−0.55, −0.22	83%	112.71* (19, <0.001)
	IMI vs. Control (short-term follow-up results^e)	10	−0.39*	−0.67, −0.10	78%	40.48* (9, <0.001)
	IMI vs. Control (medium-term follow-up results^f)	6	−0.20*	−0.38, −0.01	57%	11.57* (5, 0.04)

*Significant SMDs are marked with an asterisk. ^aNegative values characterize effect sizes favouring the intervention group. ^b95%CI 0.0 to 51.8%. ^cStudies with low dropout are defined as ≤20% dropout post-randomization. ^dStudies with high risk of bias are defined as having three or more domains rated with ‘high’. ^eShort-term is defined as up to 3 months or 12 weeks post-randomization. ^fMedium-term is defined as >3 months (or 12 weeks) and <1 year (or 52 weeks) post-randomization.

2.4. Component studies

Five relevant component studies were identified. Three of these were additive and dismantling studies (60.0%), which assessed types of support – namely peer support (Possemato et al., 2019), clinician-administered guidance (Possemato et al., 2016), and daily text messages directing the usage of the mobile phone apps (Roy et al., 2017). None of these studies found significant differences between the conditions, even though Possemato et al. (2016) stated that their results still suggest an improvement of outcomes when adding support. Niles et al. (2020) tested two versions of an Attention-Bias Modification mobile app for PTSD. There were no differences between the personalized and the non-personalized version (Niles et al., 2020). Spence et al. (2014) evaluated exposure and also reported no significant differences in symptom reduction. All of the three studies focusing on support involved veterans, whereas Niles et al. (2020) and Spence et al. (2014) included individuals who self-identified as having PTSD. Due to the limited amount of studies, meta-analytic pooling was not feasible.

2.5. Mediation studies

Three relevant mediation studies investigating different populations were identified. Cieslak et al. (2016) assessed health and human service professionals, who have been indirectly exposed to a traumatic event at work. Stevens, Holmgreen, Walt, Gengler, and Hobfoll (2017) recruited veterans. Xu et al. (2016) included individuals who have experienced any traumatic event. On the whole, the studies evaluated four different potential mediators: self-efficacy beliefs (Cieslak et al., 2016), perceived physical impairment (Stevens et al., 2017), social acknowledgement (i.e. patients’ experience of positive reactions from other individuals or society that acknowledge the traumatic experience), and disclosure of trauma (Xu et al., 2016). The included studies used various statistical methods and tools for the mediation analyses, namely PROCESS, Model 4, with bootstrapping (Cieslak et al., 2016; Hayes, 2013), multiple models in Mplus Version 5.2 (Muthén & Muthén, 1998–2017; Stevens et al., 2017), and Preacher and Hayes’ approach (2004; Xu et al., 2016).

Table 2 indicates the extent to which the RCTs meet the requirement for process research as outlined in the methods section and the study protocol (Steubl et al., 2019). The inter-rater reliabilities yielded perfect agreement (Cohen’s kappa = 1.00). All of the potential mediators reached significance. Given the limited number of eligible studies, meta-analytic pooling was methodologically and conceptually not meaningful.

Table 2. Extent to which mediation studies meet requirements for process research

Author(s) (Year)	RCT	Control	Theoretical background	n ≥ 40 per group	Multiple mediators	Temporality^a	Manipulation
Cieslak et al. (2016)	Yes	Yes	Yes	Yes	No	No	No
Stevens et al. (2017)	Yes	Yes	Yes	Yes	No	No	No
Xu et al. (2016)	Yes	Yes	Yes	Yes	Yes	No	No

^aTemporality is defined as >2 assessments during the treatment phase (including pre- and post-treatment assessment).

3. Discussion

This systematic review and meta-analysis complements and extends previous research by summarizing, synthesizing, and reviewing the literature on IMIs for PTSD. It goes beyond previous reviews (Kuester et al., 2016; Sijbrandij et al., 2016; Simblett et al., 2017), as it is the first meta-analysis distinguishing between bonafide and non-bonafide controls and examining mediators and components in additive or dismantling studies. An additional strength of this study lies in the integration of mobile-based interventions.

Results addressing the first research question provide additional evidence for the efficacy of IMIs for PTSD across different types of outcome measurements in different populations. These results are consistent with the findings of previous meta-analyses (Kuester et al., 2016; Sijbrandij et al., 2016; Simblett et al., 2017) and further expand on these findings by including only active non-bonafide control groups. Additionally, when pooling the studies with low dropout rates, an increase in the effect size was observed. Even though more detailed analyses were not possible due to the heterogeneity in the reporting of the data, this fact points towards the importance of adherence to therapy in IMI studies. Furthermore, evidence on the comparison of IMIs with bonafide control conditions turned out to be limited, with only two studies falling into this category. Hence, no firm conclusion can be drawn in this regard.

Similarly, the current evidence-base in regard to intervention components proved to be weak, with only five eligible studies precluding meta-analytic pooling. Hereby, human support could have been expected to serve as a vital intervention component considering the fact that traumatic events of interpersonal nature are a risk factor for the development of PTSD. Therefore, corrective interpersonal experiences may be also of special significance in IMIs for PTSD (Brewin, Andrews, & Valentine, 2000). Nevertheless, the previously proposed importance of therapeutic guidance (e.g. Baumeister, Reichler, Munzinger, & Lin, 2014; Domhardt et al., 2019) could not be replicated in the present study. Likewise, engagement in exposure exercises did not yield a significant difference (Eisma et al., 2015). This is a finding that stands in sharp contrast to previous results that emphasize the overall importance of exposure in PTSD treatment (Rothbaum & Schwarz, 2002). However, it is also worth noting that the study shows limited power to detect significant differences at all (Eisma et al., 2015). Additional possible active components (e.g. opportunity to talk about the traumatic experience, psychoeducation, teaching of coping skills) that have previously been associated with the success of face-to-face treatments for PTSD (Wampold et al., 2010) have not been assessed in the included studies. Even though this is the first systematic review on intervention components in IMIs for PTSD, the results align with previous meta-analytic reviews on dismantling/additive studies in regard to components in IMIs for anxiety disorders (Domhardt et al., 2019) and in face-to-face psychotherapies (Bell et al., 2013; Cuijpers, Cristea, Karyotaki, Reijnders, & Hollon, 2017). None of these studies found significant pooled effects for other components aside from guidance (Domhardt et al., 2019). However, Bell et al. (2013) found small but significant effects for adding single components to psychotherapeutic interventions, without disclosing which specific components contributed to the increased efficacy across different populations and disorders. Furthermore, Cuijpers et al. (2017) concluded that existing component studies on depression did not have the required statistical power and methodological quality to gain insights on active components, calling for high quality research to disentangle the active ingredients of psychotherapeutic interventions for common mental disorders (Furukawa et al., 2018).

Although mediation studies are of high importance to detect mechanisms of change and may contribute to evidence-based intervention development leading to improved treatments (Domhardt et al., 2021), they are limited in number. With only three eligible RCTs, the intended two-stage structural equation modelling (TSSEM) approach (Cheung, 2015; Cheung & Chan, 2009) was not feasible. However, evidence from individual studies point towards the potential mediating roles of both cognitive and emotional processes such as self-efficacy beliefs, changes in perceived physical health impairment, social acknowledgement, and disclosure of trauma. Especially social acknowledgement and disclosure of trauma may be associated with the human support component, as both of them relate to social and interpersonal processes. Other proposed mediators in the context of PTSD treatment (e.g. hopelessness, habituation; Gallagher & Resick, 2012) were not assessed in the included studies. The methodological assessment of the requirements for process research (Domhardt et al., 2021; Kazdin, 2007; Lemmens et al., 2016) revealed substantial shortcomings in mediation studies, especially in regard to the assessment of multiple mediators, the consideration of temporality, and the experimental manipulation of the mediator variable itself. Yet, it is crucial to assess more than one mediator (i.e. to depict interactions between different processes and test competing hypotheses of rivalling mechanisms), establish the essential timeline (i.e. to ensure that changes in the mediator variable occurred before changes in the outcome, thereby pointing to causality) and employ an experimental approach (i.e. to eliminate alternative explanations for the mediator-outcome-relation; Kazdin & Nock, 2003; Lemmens et al., 2016). However, all of the included studies met the criteria of classification as RCTs, employing a control group, including a theoretical background, and using an appropriate sample size (i.e. n ≥ 40 per group).

3.1. Limitations of the current study

There are several limitations that should be considered when interpreting the findings of this systematic review and meta-analysis which mirror the GRADE rating ‘low’. The first important limitation results from the broad inclusion of different samples (e.g. type of trauma) and interventions (e.g. number of modules, duration of treatment, theoretical background), giving rise to clinical, methodological and (in large part substantial) statistical heterogeneity between the included studies, which limits the generalizability of findings. Likewise, previous meta-analyses on IMIs for PTSD have reported relatively high heterogeneity while suggesting different reasons for this (e.g. variety of employed therapeutic techniques, level of guidance; Simblett et al., 2017). The second important limitation is that the included studies reported high attrition rates, which is a frequently observed problem in IMIs, especially in unguided interventions (Domhardt, Ebert, et al., 2018; Karyotaki et al., 2015). Even though appropriate statistical methods were used to account for missing data in primary studies and studies with low dropout were pooled separately, the risk of attrition bias may be relevant in some instances. Thirdly, as the current systematic review and meta-analysis already provides an extensive overview and integration of existing literature on IMIs for PTSD, only between-group comparisons were considered. Future studies could extend to within-group comparisons (e.g. Assink & Wibbelink, 2016) in order to reduce heterogeneity. Fourthly, publication bias represents a stable problem in meta-analyses (Dickersin, 1990). Although we aimed to identify unpublished studies and assessed publication bias, all studies identified as eligible for inclusion were written in English. This could restrict the generalizability of results. Moreover, as trials with statistically significant results have been shown to be more likely to be published in English (Egger, Zellweger-Zähner, et al., 1997), effects may have been overestimated (Morrison et al., 2012). The fifth limitation is that both of the quality assessments (i.e. risk of bias and requirements for process research) unveiled important deficits or unclear reporting in the included studies, which may limit the validity of the results. Sixth, the sample of participants included in primary studies may not adequately represent the population affected by PTSD (i.e. overrepresentation of veterans). Seventh, in order to reduce methodological heterogeneity only RCTs with active control groups were included in this review. However, relevant evidence on mechanisms of change may be also found in studies with other designs (e.g. experimental design studies, mobile and passive sensing, computational psychiatry, neurobiological research paradigms; Domhardt, Cuijpers, Ebert, & Baumeister, accepted). For example, results of an uncontrolled laboratory investigation of Benight, Shoji, Yeager, Weisman, and Boult (2018) indicate that changes in trauma coping self-efficacy may be an important mechanism of change in PTSD symptom reduction. What is more, to our knowledge longitudinal assessments of the interplay between symptomatology and possible mediators in IMIs for PTSD (e.g. coping strategies as proposed in Lorenz et al., 2019) have not been carried out yet. Last but foremost, it must be pointed out that the overall power of this meta-analysis as well as the scope of the qualitative review of components and mediators is limited due to the small number of studies that were eligible for inclusion. Despite employing a thorough search strategy, additional relevant studies may have been missed. As the body of research is rapidly growing in this field, this systematic review should be updated in the future.

3.2. Implications

Despite these limitations, the current systematic review and meta-analysis has several strengths, including the adherence to a pre-registered published study protocol (Steubl et al., 2019), a conclusive summary of evidence on efficacy compared to (bonafide and non-bonafide) active comparisons, as well as a comprehensive consideration of intervention components and mediators. More precisely, the results of the present review contribute to the understanding of how IMIs function which in turn may help develop more effective interventions, and assist in the verification and advancement of psychotherapeutic theories (Kazdin, 2007; Lemmens et al., 2016). Moreover, the findings and discussed limitations indicate important gaps in the existing literature. In this regard, they may lead the way to much needed high quality RCTs in the assessment of the efficacy of IMIs for PTSD compared to first line face-to-face treatments or active (bonafide) control conditions. Additionally, CBT-based IMIs have resulted in the highest SMD in our subgroup analyses. Therefore, future IMIs should aim to include key commonalities of existing first line face-to-face treatments for PTSD in adults (i.e. psychoeducation, emotion regulation, coping skills, imaginal exposure, cognitive processing, restructuring, and/or meaning making; Schnyder et al., 2015). In particular, more research aiming for a better understanding of active components and mechanisms of change is worthwhile. This is relevant not only to improve IMIs for PTSD and other disorders, but also to give new ideas to the evolving debate on psychotherapy processes and the accountability of common factors or specific techniques for the efficacy of psychotherapeutic interventions (Kazantzis et al., 2018; Lorenzo-Luaces & DeRubeis, 2018). Additive and dismantling studies investigating the incremental effects of different treatment components in an online format as well as mediation studies using well-established approaches to mediation analysis, will further improve our knowledge on mechanisms of change of IMIs for PTSD in adults.

4. Conclusion

In conclusion, the results of this systematic review and meta-analysis provide support for the superiority of IMIs for PTSD compared to active controls. In addition, the results highlight possible mechanisms of change in these rather novel digital interventions and in light of previous meta-analyses (Kuester et al., 2016; Sijbrandij et al., 2016; Simblett et al., 2017), they endorse the future use of IMIs for PTSD in routine care, when first-line face-to-face psychotherapies are unavailable. Moreover, the compiled evidence highlights substantial research gaps. More high-quality studies comparing IMIs with first line face-to-face treatments and active (bonafide) controls are needed. Additionally, future research should focus on replications of the included studies in different settings, assess additional active components besides guidance, and analyse different mediators that have not been investigated so far. Thereby, it is important to adhere to the quality criteria for process research and amend common methodological shortcomings as highlighted in this review.

Contributors

LS, CS, HB, and MD developed the design of the study. LS conducted the systematic literature searches, extracted the data, assessed the risk of bias and methodological quality of the included studies, and wrote the first draft of the manuscript. MD provided support at all stages of the process. All authors (LS, CS, HB, and MD) contributed to the further writing of the manuscript, and have approved the final version for submission.

Ethics statement

Since the present work consists of a systematic review and aggregated meta-analysis, an ethics vote was not required. No individual participant data was used or stored for the current study. Informed consents from the participants of the original studies were not required.

Open scholarship

This article has earned the Center for Open Science badge for Preregistered. The materials are openly accessible at: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42019130314

Acknowledgments

The authors wish to thank Charlotte Dechmann (CD), Amelie Kunow (AK), Chiara Ritter (CR), and Anna-Lena Schröder (AS) for their contributions to the systematic literature search and the assessment of the risk of bias and methodological quality of the included studies.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The data supporting this systematic review and meta-analysis are from previously reported studies, which have been cited.

Supplementary material

Supplemental data for this article can be accessed here.

Additional information

Funding

This work was supported by the German Federal Ministry of Education and Research (BMBF) under Grant number FKZ01KG1802.