The international trauma questionnaire (ITQ) measures reliable and clinically significant treatment-related change in PTSD and complex PTSD

ABSTRACT

Background: The International Trauma Questionnaire (ITQ) is a validated measure that assesses ICD-11 posttraumatic stress disorder (PTSD) and complex PTSD (CPTSD). An important task is to determine whether the ITQ is an appropriate evaluative measure for clinical trials.

Objective: To assess the psychometric properties of the ITQ in the context of treatment and determine if the ITQ measures reliable and clinically significant change over the course of a psychosocial intervention.

Method: Analyses were based on data from an online skills training programme delivered to 254 U.S. Veterans. Reliability and validity of the ITQ scores were assessed at baseline. Changes in symptom scores and probable diagnostic rates were compared at pre-, mid- and post-treatment. A reliable change index (RCI) score was computed to classify participants as improved, unchanged, or worsened. The PCL-5 was used as a comparison measure.

Results: Baseline concurrent and factorial validity was similar to previous studies. Internal consistency at each assessment was excellent and comparable to the PCL-5. Decline in symptoms from pre-to-post-treatment was significant for PTSD and CPTSD symptom profiles. Rate of probable disorder (PTSD or CPTSD) declined significantly from pre-treatment to post-treatment. Pre-to-post treatment declines exceeded the critical RCI values for the ITQ. Clinically significant changes were observed where most participants improved, some stayed the same, and few worsened. The performance of the ITQ was consistent with the PCL-5 regarding sensitivity to change.

Conclusion: This study provides the first demonstration that the ITQ measures reliable and clinically significant treatment-related change of ICD-11 PTSD and CPTSD symptoms.

HIGHLIGHTS

• This study provides the first demonstration that the International Trauma Questionnaire (ITQ) measures reliable and clinically significant treatment-related change in ICD-11 PTSD and Complex PTSD.

Antecedentes: el Cuestionario Internacional de Trauma (ITQ por su sigla en inglés) es una medida validada que evalúa el trastorno por estrés postraumático (TEPT) y el TEPT complejo (TEPT-C) según la CIE-11. Una tarea importante es determinar si la ITQ es una medida de evaluación adecuada para los ensayos clínicos.

Objetivo: Evaluar las propiedades psicométricas de la ITQ en el contexto del tratamiento y determinar si la ITQ mide un cambio confiable y clínicamente significativo durante el transcurso de una intervención psicosocial.

Método: Los análisis se basaron en datos de un programa de entrenamiento de habilidades en línea entregado a 254 veteranos estadounidenses. La confiabilidad y la validez de las puntuaciones de ITQ se evaluaron al inicio del estudio. Se compararon los cambios en las puntuaciones de los síntomas y las tasas de diagnóstico probables antes, en la mitad y después del tratamiento. Se calculó una puntuación de índice de cambio confiable (RCI) para clasificar a los participantes como mejorados, sin cambios o empeorados. El PCL-5 se utilizó como medida de comparación.

Resultados: La validez basal concurrente y factorial fue similar a estudios previos. La consistencia interna en cada evaluación fue excelente y comparable a la del PCL-5. La disminución de los síntomas de antes a después del tratamiento fue significativa para los perfiles de síntomas de TEPT TEPT-C. La tasa de trastorno probable (TEPT o TEPT-C) disminuyó significativamente desde el pretratamiento hasta el postratamiento. Las disminuciones antes-después del tratamiento excedieron los valores críticos de RCI para la ITQ. Se observaron cambios clínicamente significativos en los que la mayoría de los participantes mejoraron, algunos permanecieron igual y pocos empeoraron. El desempeño de la ITQ fue consistente con el PCL-5 con respecto a la sensibilidad al cambio.

Conclusión: Este estudio proporciona la primera demostración de que la ITQ mide cambios confiables y clínicamente significativos relacionados con el tratamiento de los síntomas de TEPT y TEPT-C de la CIE-11.

背景: 国际创伤问卷 (ITQ) 是一个经过验证的评估ICD-11创伤后应激障碍 (PTSD) 和复杂性PTSD (CPTSD) 的测量工具。一项重要的任务是确定ITQ是否适合用于临床试验。

目的: 在治疗的背景下评估ITQ的心理测量特性, 并确定ITQ是否测量出可靠且具有临床意义的社会心理干预过程中的变化。

方法: 分析基于来自一项提供给254名美国退伍军人的在线技能培训计划的数据。在基线时评估了ITQ得分的信度和效度。在治疗前, 中, 后比较症状得分的变化和可能的诊断率。计算了一个可靠的变化指数 (RCI) 分数, 以将参与者分为改善, 不变或恶化。 PCL-5用作对比测量。

结果: 基线的同时和因素效度与前人研究相似。每次评估的内部一致性都非常好, 可与PCL-5相媲美。对于PTSD和CPTSD症状, 治疗前后症状的下降是显著的。从治疗前到治疗后, 可能的发病率 (PTSD或CPTSD) 显著下降。治疗前后下降幅度超过了ITQ的关键RCI值。观察到了临床上显著的变化, 大多数参与者有所改善, 一些参与者保持不变, 很少恶化。 ITQ在对改变的敏感性上的表现与PCL-5一致。

结论: 本研究首次证明了ITQ可测量ICD-11 PTSD和CPTSD症状的可靠且临床显著的治疗相关变化。

KEYWORDS:

• International trauma questionnaire
• ITQ
• RCI
• reliable change in psychotherapy
• clinically significant change in psychotherapy
• ICD-11 PTSD
• ICD-CPTSD

PALABRAS CLAVE:

• Cuestionario Internacional de Trauma
• ITQ
• RCI
• cambio confiable en psicoterapia
• cambio clínicamente significativo en psicoterapia
• TEPT según CIE-11
• TEPT-C según CIE-11

关键词:

• 国际创伤问卷
• ITQ
• RCI
• 心理治疗的可靠变化
• 心理治疗的临床显著变化
• ICD-11 PTSD
• ICD-CPTSD

The 11^th edition of the International Classification of Disorders and Related Health Problems (ICD-11; World Health Organization, 2018) brought significant changes to the formulation of posttraumatic stress disorder (PTSD) and introduced a new disorder, complex PTSD (CPTSD). The diagnosis of PTSD was refined to include symptoms organized into three clusters: re-experiencing of the traumatic event in the here and now, avoidance of traumatic reminders, and a sense of current threat. CPTSD was introduced to describe a broader array of symptoms which include not only PTSD symptoms but also the adverse effects that trauma can have on self-organization, particularly when the traumatic experience is of a prolonged or repeated nature (e.g. childhood abuse, domestic violence). The diagnosis of CPTSD is comprised of six symptom clusters; the three PTSD clusters and three clusters related to disturbances in self-organization (DSO): affect dysregulation, negative self-concept, and disturbances in relationships (Maercker et al., 2013).

Clinical interviews and self-report instruments for assessing ICD-11 PTSD and CPTSD have been developed to align with ICD-11 criteria (e.g. Litvin, Kaminski, & Riggs, 2017; Roberts, Cloitre, Bisson, & Brewin, 2018). One of the more widely used self-report questionnaires is the International Trauma Questionnaire (ITQ: Cloitre et al., 2018). Psychometric evaluations indicate that the PTSD and DSO items of the ITQ produce scores with satisfactory internal consistency, as measured by Cronbach’s alpha (α), across a range of study samples including epidemiological (Ben-Ezra et al., 2018; Cloitre et al., 2019), community (Ho et al., 2020, 2019), and clinical (Hyland, Shevlin, Fyvie, & Karatzias, 2018; Kazlauskas, Gegieckaite, Hyland, Zelviene, & Cloitre, 2018) samples as well as among refugees (Hyland et al., 2018) and military personnel (Letica-Crepulja et al., 2020; Mordeno, Nalipay, & Mordeno, 2019; Murphy et al., 2020).

Evidence of the concurrent and discriminant validity of the ITQ scores has been demonstrated in studies showing that the PTSD and DSO item clusters are differentially related to multiple criterion variables (e.g. Ho et al., 2019; Hyland et al., 2017). For example, Hyland et al. (2017) reported that the PTSD symptoms uniquely predicted panic disorder symptoms and were a stronger predictor of anxiety symptoms relative to the DSO symptoms, whereas DSO symptoms predicted emotion dysregulation, negative beliefs about self, negative beliefs about the world, and depression while the PTSD symptoms did not. Discriminant validity has also been demonstrated in studies using latent profile analyses and latent class analyses of trauma samples. Consistent with the proposed distinction between the diagnosis of PTSD and CPTSD, analyses of study samples have consistently revealed two subgroups distinguished by different patterns of symptom endorsement, one following the PTSD profile and the other following the CPTSD profile (e.g. Karatzias et al., 2017; Kazlauskas et al., 2018; Murphy, Elklit, Dokkedahl, & Shevlin, 2016). A recent systematic review of factor analytic and latent class/profile analyses with the ITQ reported that, across a total of 12 studies, the number of profiles identified varied from two to six; however, all analyses evidenced a PTSD and a CPTSD profile (Redican et al., 2021).

Lastly, the factorial validity of the ITQ has also been demonstrated across different countries and cultures (Redican et al., 2021). Two models have consistently been found to fit the data well (Shevlin et al., 2017). The first is a correlated six-factor model (e.g. Ben-Ezra et al., 2018; Ho et al., 2020; Mordeno et al., 2019), and the second is a two-factor, higher order model in which the three PTSD symptom clusters fall under a PTSD factor and the three DSO clusters fall under a DSO factor (e.g. Hyland et al., 2017; Karatzias et al., 2016; Kazlauskas et al., 2018; Owczarek et al., 2020; Vallières et al., 2018). The review found that the two-factor second-order solution was the best fit in the majority but not all clinical studies while community setting studies were equally split between finding the first order versus the two-factor model as the better fit. Notably, all studies tested both models and all found that both were a good fit to the data (Redican et al., 2021).

An important next step is to determine whether the ITQ is responsive to change and appropriate as an evaluative measure for clinical trials (Guyatt, Kirshner, & Jaeschke, 1992). A measure’s responsiveness to change is sometimes seen as a form of validity (Hays & Hadorn, 1992), and other times as a psychometric characteristic separate from reliability and validity (Guyatt, Walter, & Norman, 1987). Regardless of its psychometric placement, an evaluative measure must be sufficiently sensitive to capture real change over time. In clinical research, reliable and clinically significant change is often used to summarize responsiveness at the individual level within the context of observed reliable changes for the whole group (Evans, Margison, & Barkham, 1998). To date, no study has evaluated the psychometric properties of the ITQ during treatment.

The study had two objectives. The first objective was to replicate findings of the concurrent, discriminant, and factorial validity, as well as the internal consistency, of the ITQ scores in a treatment context. The second and more important goal was to determine the ability of the ITQ to measure reliable and clinically significant change in symptoms scores, and probable diagnostic status, over the course of a psychological intervention. Because the PTSD Checklist for DSM-5 (PCL-5; Weathers et al., 2013) is a frequently used self-report measure in PTSD outcome studies, we included the PCL-5 as an external comparator.

This study utilized data from a web-based psychosocial transdiagnostic intervention (webSTAIR) evaluation programme delivered to U.S. Veterans, one goal of which was to increase engagement into care among trauma-exposed Veterans via the use of technology as well as via the application of a relatively ‘wide net’ inclusion criteria. Accordingly, enrollment into the programme required a positive screen for either PTSD or depression. This enrollment strategy was based on literature indicating that PTSD and depression are highly comorbid (Rytwinski, Scur, Feeny, & Youngstrom, 2013), particularly among Veterans (Chan, Cheadle, Reiber, Unützer, & Chaney, 2009) and that emotion regulation and interpersonal difficulties, key targets in the webSTAIR intervention, are associated with both depression and PTSD (Beck, Grant, Clapp, & Palyo, 2009; Cloitre, Hyland et al., 2019). The use of data from the evaluation programme was appropriate for the goals of this psychometric study as we surmised that the ‘wide net’ inclusion criteria would yield enrollment of individuals with ICD-11 PTSD as well as those with CPTSD.

1. Methods

1.1. Participants and procedure

Participants were 254 male and female United States (U.S.) Veterans enrolled in an online 10-module web-based (i.e. webSTAIR) programme recruited from eight, predominantly rural Veterans Affairs facilities located across the country. Enrolment into the programme was based on referrals by mental health providers. The study combined data from three different evaluation projects that varied by amount of coaching support provided to the Veteran during the programme.

Pre-screening criteria included age of 18 years or older, history of trauma exposure, positive screen for either PTSD symptoms (PC-PTSD-5; Prins et al., 2016) or depression (PHQ-2; Arroll et al., 2010), no changes to medication in the previous 6 weeks, willingness to use an online treatment format and availability for the duration of the study. Exclusion criteria included presence of psychosis and significant cognitive impairment as assessed by the evaluator during the screen. Participants were required to complete the 10 modules over a 15-week period. Assessment data included in this study were collected at pre-treatment, preceding entry into the first module (T1, n = 254), mid-treatment, following the completion of the fifth module (T2, n = 168, follow-up rate = 66.1%), and post-treatment, following completion of the tenth module (T3, n = 130, follow-up rate = 51.2%). Non-completers were significantly younger than completers (M = 42.80 vs M = 46.25, t(254) = 2.34, p = .021). On all other demographic, symptomatic, and diagnostic variables, there were no significant differences between completers and non-completers.

Assessments were completed via phone guided by a Master’s level psychologist. Participants received copies of all measures via mail and used the forms to report their scores. Participants were paid 30.00 USD, for each assessment. This study was approved by the study site’s VA Research and Development Committee. The Institutional Review Board affiliated with the VA approved the study and a waiver of informed consent was provided as the project was considered an evaluation project and not research.

1.2. Intervention

The webSTAIR programme consists of 10 web-based modules adapted from STAIR (Bauer et al., 2021; Cloitre, Cohen, Ortigo, Jackson, & Koenen, 2020). The first five modules review emotional awareness, emotion management, and distress tolerance, while the final five modules raise awareness about relationship patterns and provide interpersonal skills training regarding effective assertiveness, interpersonal flexibility, and compassion for self and others. The primary goal of webSTAIR, identical to STAIR, is to improve functioning via increased emotion regulation and interpersonal skills. Nevertheless, data from recent treatment outcome studies indicates that STAIR is effective in providing significant reductions in DSM-5 PTSD and depression among veterans receiving mental health services in primary care (Jain et al., 2020), PTSD speciality care (Jackson, Weiss, & Cloitre, 2019) and telemental health for military sexual trauma (Weiss, Azevedo, Webb, Gimeno, & Cloitre, 2018) as well as in receipt of the web-based version of STAIR (Bauer et al., 2021).

1.3. Measures

The International Trauma Questionnaire (ITQ: Cloitre et al., 2018) was used to measure PTSD and CPTSD symptoms at the three assessment periods. The ITQ first screens for a respondent’s ‘index’ trauma event and participants are instructed to answer all questions in relation to this event. The ITQ includes six items measuring each PTSD symptom from the three clusters of ‘Re-experiencing in the here and now’, ‘Avoidance’, and ‘Sense of Threat’, and these items are answered in terms of how bothersome that symptom has been in the past month. The ITQ also includes six items measuring each ‘Disturbance in Self-Organization’ (DSO) symptom from the three clusters of ‘Affective Dysregulation’, ‘Negative Self-Concept’, and ‘Disturbed Relationships’. These items are answered in terms of how a respondent typically feels, thinks about oneself, and relates to others. The PTSD and DSO symptoms are accompanied by three items measuring associated functional impairments in the domains of social, occupation, and other important areas of life.

All ITQ items are answered on a 5-point Likert scale ranging from 0 (Not at all) to 4 (Extremely). Thus, PTSD and DSO symptom scores range from 0 to 24 (i.e. the sum of the six items from each subscale), and CPTSD symptom scores range from 0 to 48 (i.e. the sum of the 12 ITQ items). For diagnostic purposes, each symptom and functional impairment indicator was considered ‘present’ based on a score of ≥ 2 (Moderately) on the Likert scale (Cloitre et al., 2018). The requirements for probable PTSD diagnosis are met if a person is trauma exposed, at least one symptom is present from each PTSD cluster, and there is functional impairment associated with these symptoms. The diagnostic requirements for CPTSD are met if the PTSD criteria are satisfied, and at least one symptom is present from each DSO cluster, and there is functional impairment associated with these symptoms. As per the ICD-11 diagnostic rules, a person can may receive a diagnosis of PTSD or CPTSD, but not both. Thus, if a person meets the criteria for CPTSD they do not also receive a diagnosis of PTSD. In this study, we examine changes in (a) probable PTSD diagnostic rates, (b) probable CPTSD diagnostic rates, and (c) probable PTSD or CPTSD diagnostic rates (i.e. the total number of PTSD and CPTSD cases).

The following measures were used to explore differential associations with the ITQ PTSD and DSO subscales. The PCL-5 was also used as a comparator measure of change across assessments. The PCL-5 is a frequently used treatment outcome measure in PTSD trails and is similar to the ITQ in that respondents rate the degree to which they have been ‘bothered’ by symptoms (0 = not at all to 4 = extremely) in the past month.

1.3.1. PTSD Checklist for DSM-5 (PCL-5; Weathers et al., 2013)

The PCL-5 is a 20-item self-report measure of PTSD as assessed in the DSM-5. It is comprised of four symptom clusters: intrusions (5 items), avoidance (2 items), negative alterations in cognitions and mood (7 items) and alterations in arousal and reactivity (6 items). Responses are summed to create a total score, with higher scores indicating more severe PTSD symptoms. The PCL-5 has good psychometric properties in veteran samples and has demonstrated sensitivity to clinical change pre to post treatment (Bovin et al., 2016; Wortmann et al., 2016).

Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) was used to measure emotion regulation. The DERS is 36-item measure which asks participants to rate items on a scale of 0 (almost never) to 5 (almost always) identifying how often a person experiences difficulties in emotion regulation. Responses were summed to create a total score with higher scores indicating greater emotion regulation difficulties.

Inventory of Interpersonal Problems 32 item Version (IIP-32, Barkham, Hardy, & Startup, 1996) was used to measure interpersonal difficulties. The IIP-32 is a 32-item self-report measure which asks participants to rate difficulties with someone important in their life on a scale of 0 (not at all) to 4 (extremely). Responses were averaged to create a mean score, with higher scores indicating greater interpersonal difficulties.

1.4. Data analytic plan and predictions

1.4.1. Phase 1: examination of concurrent, discriminant, factorial validity and internal consistency of the ITQ scores

Baseline zero-order correlations were computed for all measures to identify differential correlations between the ITQ subscales and the various other measures outlined above. It was expected that observation of the pattern of correlations would find that the largest correlations for the ITQ PTSD subscale would be with the PCL-5 intrusions, avoidance, and arousal clusters while the correlations with the DERS and IIP-32 would be smaller. We also expected that the largest correlations for the ITQ DSO cluster would be with the DERS and IIP-32 while those with the PCL-5 intrusion, avoidance, and arousal clusters would be smaller. The webSTAIR intervention study did not include a measure of self-concept and so assessment of the concurrent validity of the ITQ in regard to this construct could not be evaluated.

Factorial validity was tested via confirmatory factor analysis on the data at baseline. The two models supported in the literature (Redican et al., 2021) – the correlated six-factor model and the two-factor second-order model – were assessed. The models were tested using Weighted Least-Squares Mean and Variance (WLSMV) adjusted estimation, and model fit was judge in relation to standard recommendations (Hu & Bentler, 1999). It was predicted that both models would fit the data well but the two-factor second-order model might be a closer fit of the data (see Redican et al., 2021).

Cronbach’s alpha was computed for the ITQ, as well as the PCL-5, DERS and IIP-32, for purposes of comparison at each assessment point (pre, mid, and post treatment). It was expected that the internal consistency of the ITQ would high, stable across time, and comparable to those of more established measures.

1.4.2. Phase 2: testing for change

The second phase involved testing for change in (a) symptom scores (i.e. PTSD symptoms, DSO symptoms, and CPTSD symptoms), and (b) probable diagnostic rates (i.e. PTSD, CPTSD, and PTSD or CPTSD) across the three assessments. A repeated measures ANOVA would be unsatisfactory to test for such changes as it makes strong assumptions about the data such as sphericity (Hancock & Mueller, 2013), and relies on listwise deletion to handle missing observations. The repeated measures ANOVA can, however, be specified as a more general model within a structural equation modelling framework where no assumptions are made about the variance–covariance structure of the observations (Hoffman, 2015) and missing data are handled efficiently using full information maximum likelihood estimation (MLR: Schafer & Graham, 2002).

This analytic process involves several steps. Initially, for each variable (e.g. symptoms of PTSD, DSO, and CPTSD and diagnostic status), a null or ‘constrained’ model was specified where the three means were constrained equal. Next, an alternative or ‘unconstrained’ model was specified with the three means freely estimated. The constrained and unconstrained models differed by two degrees of freedom so improvement in model fit can be tested using the loglikelihood difference test, which is distributed as a chi-square (χ²). A significant χ² result indicates that the unconstrained model is better than the constrained model, meaning that the null hypothesis of equal means can be rejected (Hoffman, 2015). Importantly, the use of MLR estimation means that all available information at T1 (n = 240) was used to estimate the means, variances, and covariances at T2 and T3, thus avoiding the deleterious effects of listwise deletion.

These models were specified and estimated using Mplus Version 8.2 (Muthén & Muthén, 2018). Mplus also incorporates the ‘model test’ feature that allows constraints to be tested using the Wald χ² test, and this was used to test pairwise comparisons (T1 vs T2, T2 vs T3, and T1 vs T3). The magnitude of the change in symptoms scores was then calculated using Cohen’s d with a correction for repeated measures designs (d_rm). Values < 0.40 reflect ‘small’ effects, values from 0.40 to 0.79 reflect ‘medium’ effects, and values ≥ 0.80 reflect ‘large’ effects (Fritz, Morris, & Richler, 2011). This approach was also used to test for the equality of proportions (i.e. changes probable diagnostic rates of PTSD, CPTSD, and PTSD or CPTSD) across the three assessment periods.

1.4.3. Phase three: testing for reliable and ‘clinically significant’ change

The third phase of the analyses focused on testing if the ITQ scores captured reliable and ‘clinically significant’ change over time. To do so, the approach proposed by Jacobson and Truax (1991) was followed and a Reliable Change Index (RCI) value was calculated for PTSD symptoms, DSO symptoms, and CPTSD symptoms as well as for the PCL-5. The RCI reflects the change in symptoms from pre-treatment (T1) to post-treatment (T3) that cannot be attributed to measurement error. In other words, the RCI reflects the absolute change in scores on the ITQ that must occur to be confident that the observed change was not due to random fluctuations over time (at a probability level of < 5%). The RCI is calculated by dividing the difference between the pre-treatment and post-treatment scores by the standard error of the difference between these two scores. The standard error of the difference in the scores is determined using the standard deviation and internal reliability estimates of the scale scores from the pre-treatment assessment period (Ferguson, Robinson, & Splaine, 2002). We used a freely available online calculator to calculate the RCI values for the ITQ and the PCL-5 (https://www.psyctc.org/stats/rcsc1.htm).

To determine if the observed change in symptom scores were ‘clinically significant’, Jacobson and Truax (1991) recommend comparing the RCI from a ‘treatment’ group to a ‘normative’ population group. As there are no population norms available for the ITQ (it is only used with trauma-exposed persons), we followed the approach suggested by Shiner, Watts, Pomerantz, Young‐Xu, and Schnurr (2011) for measures of trauma-related symptoms. We used the critical RCI values determined in the previous step to classify participants as ‘improved’ (i.e. a decrease in symptom scores for PTSD, DSO, and CPTSD that exceed the relevant RCI values), ‘worsened’ (i.e. an increase in symptom scores for PTSD, DSO, and CPTSD that exceed the relevant RCI values), or ‘unchanged’ (i.e. a change in symptom scores for PTSD, DSO, and CPTSD that does not exceed the relevant RCI values in either direction). These analyses were repeated for the PCL-5 scores as a comparator measure of change across assessments.

2. Results

2.1. Participants

The sample had a mean age of 44.20 (Mdn = 43.00, SD = 11.94, range: 22–77); 61.8% identified as female, 37.6% identified as male, and 0.6% as other (e.g. transgender). In terms of racial/ethnic background, 63.9% identified as White/Caucasian, 17.5% identified as Black/African American, 5.6% identified as Hispanic/Latino(a)/Mexican American, 1.9% identified as American Indian/Alaskan Native, 1.5% as Asian, and 4.5% identified as Other. In terms of education, 53.2% had completed some college/2 year degree, 23.6% had completed a 4-year college degree, 10.2% had completed a postgraduate degree, 12.1% earned a high school degree, and 1.0% had completed some high school. Most participants stated that they were employed full or part time (42.2%), 34.8% were unemployed, and 23.0% were retired. Regarding relationship status, 55.1% reported being married/partnered, 20.9% as single, 23.3% as divorced, and 1% as widowed. Participants served in a variety of conflict eras, with 11.1% serving during the Vietnam war, 25.8% during the Persian Gulf War, 35.9% during the Iraq and Afghanistan conflicts, and 26.2% in some other conflict. More than half of the sample (51.7%) reported being deployed during service.

The sample had experienced multiple and chronic forms of trauma exposure. The average number of traumatic life events reported was 7.80 (SD = 3.05). The three most frequently endorsed events were (1) sudden unexpected death of a loved one (81.4%), (2) physical assault (75.5%), and (3) transportation accident (75.9%). A total of 53% reported combat experience, 34% reported childhood sexual abuse and 47% reported childhood physical abuse.

2.2. Baseline concurrent validity

Table 1 provides the baseline zero order correlations among the ITQ and its subscales, the PCL-5 and its subscales, the DERS, and the IIP-32. Consistent with expectations, a review of the pattern of associations indicates that the largest correlations for the ITQ PTSD subscale were with PCL-5 intrusions (r = .86), avoidance (r = .75) and arousal (r = .75), while those related to the DERS (r = .49) and the IIP-32 (r = .47) were smaller. Conversely, the largest correlations for the ITQ DSO subscales were the DERS (r = .71) and the IIP-32 (r = .63) with smaller correlations associated with PCL-5 intrusion (r = .58), avoidance (r = .50), and arousal (r = .59).

Table 1. Correlations among variables at baseline

	1	2	3	4	5	6	7	8	9	10
1. Total ITQ	1
2. ITQ-PTSD	.84***	1
3. ITQ-DSO	.85***	.56**	1
4. Total PCL-5	.89***	.87***	.72***	1
5. PCL5-Intrusions	.79***	.86***	.58***	.87***	1
6. PCL5-Avoidance	.69***	.75***	.50***	.77***	.69***	1
7. PCL5-NACM	.81***	.65***	.76***	.87***	.64***	.61***	1
8. PCL5-Arousal	.76***	.75***	.59***	.85***	.67***	.56***	.63***	1
9. Total DERS	.66***	.49***	.71***	.59***	.48***	.39***	.58***	.54***	1
10. Total IIP-32	.64***	.47***	.63***	.58***	.46***	.39***	.57***	.51***	.62***	1

Statistical significance: *p < .05; **p < .01; ***p < .001.

ITQ = International Trauma Questionnaire; PTSD = Posttraumatic Stress Disorder; DSO = Disturbances in Self-Organization; PCL-5 = Posttraumatic Stress Disorder Checklist for DSM-5; NAMC = Negative Alterations in Cognitions and Mood; DERS = Difficulties in Emotion Regulation Scale; IIP-32 = Inventory of Interpersonal Problems- 32 Item Version.

2.3. Baseline factorial validity and internal reliability

The two-factor second-order model (χ² (47) = 80.57, p = .002; CFI = .98; TLI = .97; RMSEA = .05 (.03, .07); SRMR = .04) and the correlated six-factor model (χ² (39) = 67.12, p = .003; CFI = .98; TLI = .97; RMSEA = .05 (.031, .074); SRMR = .03) were both good fitting models and were statistically indistinguishable from one another. The second-order model may be preferred based on parsimony.

Table 2 provides the Cronbach’s alpha for all measures as well as the associated mean and standard deviations at pre-, mid- and post-treatment. Alphas for the ITQ were high ranging from .89 to .94 and stable over time. Alphas for the ITQ were comparable to those for the PCL-5 which ranged from .92 to .96 as well as to those for the DERS and IIP-32 which ranged from .85 to .95.

Table 2. Descriptive statistics and cronbach’s alpha for study measures at pre-, mid-, and post-treatment

	Pre-Treatment		Mid-Treatment		Post-Treatment
Variables	Mean (SD)	Cronbach’s Alpha	Mean (SD)	Cronbach’s Alpha	Mean (SD)	Cronbach’s Alpha
Total ITQ	31.00 (9.75)	.89	26.20 (10.66)	.92	23.12 (11.92)	.94
Total PCL-5	50.59 (15.62)	.92	43.49 (17.72)	.95	37.93 (19.16)	.96
Total DERS	110.11 (25.36)	.94	98.76 (23.52)	.94	90.89 (25.88)	.95
Total IIP-32	58.89 (16.77)	.85	55.26 (18.27)	.89	49.86 (19.59)	.92

ITQ = International Trauma Questionnaire; PCL-5 = Posttraumatic Stress Disorder Checklist for DSM-5; DERS = Difficulties in Emotion Regulation Scale; IIP-32 = Inventory of Interpersonal Problems- 32 Item Version.

2.4. Change in symptoms over time

The means and standard deviations for the PTSD, DSO, and CPTSD symptom scores at each assessment, along with the overall and pairwise χ² test results, are presented in Table 3. In every case, the ‘unconstrained’ model was significantly (ps < .001) different from the ‘constrained’ model indicating that the mean scores for PTSD, DSO, and CPTSD significantly changed across the three assessments. All pairwise comparisons were statistically significant (ps < .001), and inspection of the means showed that PTSD, DSO, and CPTSD symptoms significantly declined through each assessment. The magnitude of the decline in PTSD symptoms from pre-treatment to post-treatment was ‘moderate’ (Cohen’s d_rm = 0.71), and the decline in DSO symptoms (Cohen’s d_rm = 0.86) and CPTSD symptoms (Cohen’s d_rm = 0.94) from pre-treatment to post-treatment were ‘large’. Similar results were obtained for the PCL-5 where symptoms significantly declined through each assessment, and the magnitude of change was large (Cohen’s d_rm = 0.96).

Table 3. Tests of differences in mean CPTSD, PTSD, and DSO symptom scores (N = 263)

	PTSD symptoms Mean (SD)	DSO symptoms Mean (SD)	CPTSD symptoms Mean (SD)	PCL-5 Mean (SD)
Time 1: Pre-treatment	15.84 (5.58)	15.15 (5.46)	31.00 (9.75)	50.59 (15.62)
Time 2: Mid-treatment	14.12 (5.77)	12.12 (5.87)	26.20 (10.66)	43.49 (17.72)
Time 3: Post-treatment	12.64 (6.24)	10.51 (6.23)	23.12 (11.92)	37.93 (19.16)
Overall test of significance (χ^2)^	52.82 (p < .001)	83.64 (p < .001)	83.98 (p < .001)	78.35 (p < .001)
Pairwise tests of significance (Wald χ^2)^^
Time 1 vs. Time 2, Cohen’s drm	34.96 (p < .001), 0.45	84.45 (p < .001), 0.72	83.39 (p < .001), 0.73	62.21 (p < .001), 0.76
Time 2 vs. Time 3, Cohen’s drm	23.14 (p < .001), 0.47	21.54 (p < .001), 0.42	27.53 (p < .001), 0.51	32.03 (p < .001), 0.48
Time 1 vs. Time 3, Cohen’s drm	62.36 (p < .001), 0.71	100.29 (p < .001), 0.86	101.33 (p < .001), 0.94	100.84 (p < .001), 0.96

^ tests have 2 degrees of freedom; ^^ tests have 1 degree of freedom; Cohen’s d_rm = Cohen’s d value corrected for repeated measures.

PTSD = Posttraumatic Stress Disorder; DSO = Disturbances in Self-Organization; CPTSD = Complex PTSD; PCL-5 = Posttraumatic Stress Disorder Checklist for DSM-5.

2.5. Change in probable diagnostic rates over time

The probable diagnostic rates for PTSD, CPTSD, and PTSD or CPTSD at each assessment, along with the overall and pairwise χ² test results, are presented in Table 4. At pre-treatment (T1), 70.9% of participants met requirements for a probable diagnosis of PTSD or CPTSD, and there were more people who met requirements for CPTSD (52.8%) than PTSD (18.1%).

Table 4. Tests of differences in probable diagnostic rates for PTSD, CPTSD, and PTSD or CPTSD (N = 254)

	PTSD	CPTSD	PTSD or CPTSD
Time 1: Pre-treatment	18.1%	52.8%	70.9%
Time 2: Mid-treatment	13.8%	22.0%	35.8%
Time 3: Post-treatment	11.0%	12.6%	23.6%
Overall test of significance (χ^2)^	6.50 (p = .039)	122.48 (p < .001)	145.634 (p < .001)
Pairwise tests of significance (Wald χ^2)^^
Time 1 vs. Time 2	2.30 (p = .129)	94.99 (p < .001)	107.31 (p < .001)
Time 2 vs. Time 3	1.70 (p = .192)	14.50 (p < .001)	16.80 (p < .001)
Time 1 vs. Time 3	6.39 (p = .012)	155.19 (p < .001)	191.22 (p < .001)

^ tests have 2 degrees of freedom; ^^ tests have 1 degree of freedom.

Note: PTSD = Posttraumatic Stress Disorder; CPTSD = Complex PTSD.

In all cases the unconstrained models of change in probable diagnostic status were statistically different from the constrained models indicating that rates of PTSD (p < .05), CPTSD (p < .001), and PTSD or CPTSD (p < .001) significantly declined across the three assessments. The pairwise comparisons for PTSD showed no significant change from pre-treatment to mid-treatment, or from mid-treatment to post-treatment; however, the decline from pre-treatment to post-treatment was statistically significant. In the case of CPTSD, and PTSD or CPTSD, all pairwise comparisons were statistically significant indicating that these rates declined through each assessment period.

2.6. Reliable and clinically significant change in symptoms over time

Based on the means, standard deviations, and internal consistency values for the ITQ at baseline, critical RCI values were calculated for the ITQ and the PTSD and DSO subscales (see the fourth column of Table 5). The decline in CPTSD symptoms scores from pre-treatment to post-treatment (7.88) exceeded the critical RCI value of 5.41 for the ITQ. Moreover, the pre-treatment to post-treatment decline in DSO symptoms (4.64) also exceeded the relevant critical RCI values for these subscales. However, the pre-treatment to post-treatment decline in PTSD symptoms (3.20) did not. Thus, the ITQ measured reliable changes in DSO and CPTSD symptoms across the three assessment periods of this intervention. The decline in PCL-5 scores from pre- to post-treatment (12.66) exceeded the critical RCI of 9.68 indicating reliable changes in PCL-5 scores.

Table 5. Reliable change index (RCI) results for the ITQ (n = 254) and clinically significant change results (n = 123)

					Reliable change?	Clinically significant change (changes exceeding critical RCI values)
	Pre-Treatment mean score	Post-Treatment mean score	Mean change from pre- to post-treatment	RCI value	Y/N	% Improved	% Worsened	% Unchanged
PTSD	15.84	12.64	3.20	3.79	N	43.1%	6.5%	50.4%
DSO	15.15	10.51	4.64	4.54	Y	54.0%	2.4%	43.5%
CPTSD	31.00	23.12	7.88	5.41	Y	56.1%	4.9%	39.0%
PCL-5	50.59	37.93	12.66	9.68	Y	54.8%	2.4%	42.7%

PTSD = Posttraumatic Stress Disorder; DSO = Disturbances in Self-Organization; CPTSD = Complex PTSD; PCL-5 = Posttraumatic Stress Disorder Checklist for DSM-5.

Table 5 also includes information about the proportion of patients who experienced ‘clinically significant’ changes in PTSD, DSO, and CPTSD symptoms from pre-treatment to post-treatment. For PTSD symptoms, 43.1% of patients ‘improved’, 6.5% ‘worsened’, and 50.4% were ‘unchanged’. For DSO symptoms, 54.0% ‘improved’, 2.4% ‘worsened’, and 43.5% were ‘unchanged’. Finally, for CPTSD symptoms, 56.1% ‘improved’, 4.9% ‘worsened’, and 39.0% were ‘unchanged’. These changes were similar to those observed for the PCL-5 where 54.8% of patients ‘improved’ 2.4% ‘worsened’ and 42.7% unchanged.

3. Discussion

The study was performed to provide a comprehensive assessment of the psychometric properties of the ITQ scores in the context of a treatment study. Our results provided support for the concurrent, discriminant, and factorial validity of the ITQ scores, similar to previous studies with clinical and non-clinical samples (Redican et al., 2021). The internal consistency of the ITQ scores across pre-, mid- and post-treatment was high and stable across time and comparable to the PCL-5, DERS and IIP-32. More importantly, the study demonstrated that the ITQ captures reliable and clinically significant change during treatment. Means and standard deviations of the PTSD, DSO and CPTSD scores significantly declined across assessments, as did the probable diagnoses of PTSD and CPTSD. Critical RCI values for PTSD symptoms, DSO symptoms, and total score (CPTSD) symptoms were calculated and change patterns based on the RCI were similar to the PCL-5. Overall, the performance of the ITQ was consistent with, and similar to that found for the PCL-5 in terms of magnitude, reliability, and clinical significance of change.

There were significant reductions in the rates of ICD-11 PTSD diagnoses from pre to post treatment (from 18.1% to 11.0%) and significant and substantial reductions in CPTSD (from 52.8% to 12.6%). The decline in symptoms from pre-to-post-treatment for the ITQ (Cohen’s d_rm = .94) and PCL-5 (Cohen’s d_rm = 0.94) total scores were large and equivalent. The decline in symptoms for the PTSD and DSO subscales differed slightly where the changes in symptoms from pre-to-post-treatment were moderate for the PTSD scores (Cohen’s d_rm = 0.71) and large for DSO (Cohen’s d_rm = 0.86). Critical RCIs were used to identify ‘clinically significant change’ where status as ‘improved’ was defined as a decrease in scores that exceeded the relevant RCI values, status as ‘worsened’ was defined as an increase in scores that exceeded the relevant RCI values, and status as ‘unchanged’ was defined as a change not exceeding the relevant RCI values in either direction (Shiner et al., 2011). Participants were distributed across all three categories within each of the symptom scores where a substantial proportion fell into the improved group for PTSD symptoms (43.1%), DSO symptoms (54.0%) and CPTSD symptoms (56.1%). This result was similar to that for the PCL-5 where the greatest proportion of participants also fell into the ‘improved’ group (54.8%).

Reduction in scores surpassed the reliable change index identified for the PCL-5 and for the ITQ total score representing CPTSD. Reliable change was also observed for the ITQ DSO symptoms. Of note, the change observed for the ITQ PTSD symptoms was close to but did not exceed the RCI. While previous studies of STAIR and webSTAIR have been shown to provide significant reductions in DSM-5 PTSD, the reliability of symptom reduction was not evidenced in this analysis as it related to the ‘core’ symptoms represented in ICD-11 PTSD (re-experiencing, avoidance, sense of threat). This result is likely a reflection of the treatment rather than the measure. It is quite possible that reliable reduction in these core PTSD symptoms is best obtained by interventions that have a trauma-focused component and the intervention that is part of this assessment did not.

WebSTAIR provides interventions that focus on problems in emotion regulation and social functioning and to a lesser extent, negative beliefs about oneself and others. All of these interventions directly address DSO symptoms and may be viewed to indirectly address PTSD symptoms. The greater improvements noted for the DSO and CPTSD scales are consistent with the proposed treatment targets of this intervention. It is expected that STAIR or webSTAIR provided in conjunction with a trauma focused component, or indeed, other trauma-focused CBTS would provide larger and more reliable reductions in ICD-11 PTSD symptoms. The differences in the reliability of change observed between the DSO and PTSD subscales speaks to the potential value of the ITQ and ICD-11 diagnoses in assessing differential effectiveness of treatments regarding specific symptom clusters.

Tracking change with the ITQ may be useful in determining the relative advantages of a particular treatment for those with a PTSD versus a CPTSD diagnosis. It may also help refine and personalize treatments for those with CPTSD. The relative importance of the PTSD and DSO symptom clusters as well as their rate of change and relationship to each other across the course of treatment may differ depending on the CPTSD population (e.g. individuals who have experienced childhood maltreatment, interpersonal violence, torture, or life-threatening illnesses). Information regarding persistent versus rapidly changing symptoms or relative degree of co-occurring change can be used to flexibly adapt treatments, add, or eliminate treatment components, and thereby contribute to the efficiency and effectiveness of treatment for CPTSD.

This study has some limitations that should be noted. The participants were U.S. Veterans therefore these findings may not generalize to other clinical populations. There was considerable sample attrition – a common occurrence in clinical trials (Little et al., 2012) – and while the most robust methods of managing missing data were used, bias due to attrition cannot be entirely ruled out. The investigation lacked a control group and changes observed might have been due to factors other than the intervention. It is well established that chronic PTSD does not resolve over time (Kessler, 2000) and thus the passage of time is unlikely to have accounted for the observed change. However, a control group with a limited number of assessments would provide an evaluation of the effect of time as well as of repeated assessments.

Lastly, it should be noted that RCI values are relative to a specific treatment and study population and those obtained in this study may not generalize to other clinical samples and interventions. As noted by Jacobson and Truax (1991), it is recommended that clinically significant cut scores be standardized by aggregating results from various studies. Research is needed to identify RCI values for refugees, survivors of domestic violence, and childhood abuse survivors as well as for treatments which differ by interventions and targeted outcome (e.g. trauma-focused treatments, mindfulness interventions).

In conclusion, this study is the first to offer evidence that the psychometric properties of the ITQ hold within a treatment context and that the ITQ captures reliable and clinically significant change related to treatment. The results of this study establish initial support for use of the ITQ in clinical trials and highlight the value of additional studies investigating reliable change in other trauma populations and treatments.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Under U.S. Department of Veterans Affairs security and confidentiality guidelines, Veterans’ data cannot be shared or made freely available to the general public.

Additional information

Funding

The study was funded by the United States Department of Veterans Affairs Office of Rural Health.