https://orcid.org/0000-0002-9353-7936
ABSTRACT
Background:
The study examined the psychometric properties of the Child and Adolescent Trauma Screen 2 (CATS-2) as a measure of posttraumatic stress disorder (PTSD) according to DSM-5 and (Complex) PTSD following the ICD-11 criteria in children and adolescents (7–17 years).
Methods:
Psychometric properties were investigated in an international sample of traumatized children and adolescents (N = 283) and their caregivers (N = 255). We examined the internal consistency (α), convergent and discriminant validity, the factor structure of the CATS-2 total scores, latent classes of PTSD/Complex PTSD (CPTSD) discrimination, as well as the diagnostic utility using ROC-curves.
Results:
The DSM-5 total score (self: α = .89; caregiver: α = .91), the ICD-11 PTSD total score (self: α = .67; caregiver: α = .79) and the ICD-11 CPTSD total score (self: α = .83; caregiver: α = .87) have proven acceptable to excellent reliability. The latent structure of the 12-item ICD-11 PTSD/CPTSD construct was consistent with prior findings. Latent profile analyses revealed that ICD-11 CPTSD was empirically distinguishable from ICD-11 PTSD using the CATS-2. ROC-analysis using the CAPS-CA-5 as outcome revealed that CATS-2 DSM-5 PTSD scores of ≥21 (screening) to ≥25 (diagnostic) were optimally efficient for detecting probable DSM-5 PTSD diagnosis. For the ICD-11 PTSD scale scores of ≥7 (screening) to ≥9 (diagnostic) were optimally efficient for detecting probable DSM-5 PTSD diagnosis.
Conclusions:
The CATS-2 is a brief, reliable and valid measure of DSM-5 PTSD, ICD-11 PTSD and CPTSD symptomatology in traumatized children and adolescents, allowing crosswalk between diagnostic systems using one measure.
HIGHLIGHTS
The CATS-2 screens for potentially traumatic events (PTEs) and PTSD symptoms.
The CATS-2 captures DSM-5 and ICD-11 criteria for PTSD and CPTSD and enables clinicians and researchers to crosswalk between both diagnostic systems.
International validation has proven good psychometric properties and presents cut-off scores
The CATS-2 is a license-free instrument and is freely accessible.
Antecedentes:
El estudio examinó las propiedades psicométricas del tamizaje de trauma para niños y adolescentes 2(CATS-2, por sus siglas en inglés) como una medida del trastorno de estrés postraumático (TEPT) según el DSM-5 y el TEPT complejo siguiendo los criterios de la CIE-11 en niños y adolescentes (7–17 años).
Método:
Se investigaron las propiedades psicométricas en una muestra internacional de niños y adolescentes traumatizados (N = 283) y sus cuidadores (N = 255). Examinamos la consistencia interna (α), la validez convergente y discriminante, la estructura factorial de las puntuaciones totales del CATS-2, las clases latentes de discriminación por TEPT/TEPT complejo (TEPTC), así como la utilidad diagnóstica utilizando curvas ROC.
Resultados:
La puntuación total del TEPT del DSM-5 (muestra: α = .89; cuidadores: α = .91), la puntuación total de TEPT de la CIE-11 (muestra: α = .67; cuidadores: α = .79) y la puntuación total del TEPTc de la CIE-11 (muestra: α = 0,83; cuidadores: α = 0,87) ha demostrado una fiabilidad de aceptable a excelente. La estructura latente del constructo CIE-11 TEPT/TEPTC de 12 ítems fue consistente con hallazgos previos. Los análisis de perfil latente revelaron que TEPTC DE LA cie-11 era empíricamente diferente del TEPT de la CIE-11utilizando CATS-2. El análisis ROC utilizando el CAPS-CA-5 como resultado reveló que las puntuaciones de TEPT según el DSM-5 del CATS-2 de ≥21 (detección) a ≥25 (diagnóstico) fueron óptimamente eficientes para detectar el diagnóstico probable de TEPT según el DSM-5. Para la escala de TEPT de la CIE-11, las puntuaciones de ≥7 (cribado) a ≥9 (diagnóstico) fueron óptimamente eficientes para detectar el diagnóstico probable de TEPT del DSM-5.
Conclusiones:
El CATS-2 es un instrumento breve, fiable y válido para la sintomatología del TEPT del DSM-5, el TEPT de la CIE-11 y el TEPTC en niños y adolescentes traumatizados, que permite el cruce entre sistemas de diagnóstico utilizando un único instrumento.
背景：
本研究考查了儿童和青少年创伤筛查 2 (CATS-2)作为一项根据 DSM-5 和（复杂性）创伤后应激障碍 (PTSD) 测量儿童和青少年（7–17 岁）PTSD的心理测量学特性。
方法：
在一个遭受创伤的儿童和青少年 (N = 283) 及其照顾者 (N = 255) 的国际样本中考查心理测量特性。我们考查了内部一致性 (α)、收敛和区分效度、CATS-2 总分的因子结构、PTSD/复杂性 PTSD (CPTSD) 区分的潜在类别，以及使用 ROC 曲线的诊断效能。
结果：
DSM-5 总分（自我：α = .89；照顾者：α = .91）、ICD-11 PTSD 总分（自我：α = .67；照顾者：α = .79）和 ICD- 11 CPTSD 总分（自我：α = .83；看护者：α = .87）已被证明具有出色的可靠性。 12 条目 ICD-11 PTSD/CPTSD 构念的潜在结构与先前的发现一致。潜在剖面分析显示使用 CATS-2可在经验上将 ICD-11 CPTSD与ICD-11 PTSD 区分开来。使用 CAPS-CA-5 作为结果的 ROC 分析显示，CATS-2 DSM-5 PTSD 评分≥21（筛查）至≥25（诊断）对于检测可能的 DSM-5 PTSD 诊断是最有效的。对于 ICD-11 PTSD 量表评分≥7（筛查）至≥9（诊断）对于检测可能的 DSM-5 PTSD 诊断是最有效的。
结论：
CATS-2 是对遭受创伤儿童和青少年的 DSM-5 PTSD、ICD-11 PTSD 和 CPTSD 症状的一项简短、可靠和有效的测量，使得用一种测量穿行于诊断系统之间成为可能。
PALABRAS LLAVE:
1. Background
With the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, Citation2013) and the 11th revision of the International Classification of Diseases (ICD-11; World Health Organization) the field of psychotraumatology is facing two classification systems with diverging diagnostic criteria for Posttraumatic Stress Disorder (PTSD). Whereas the DSM-5 defined 4 symptom clusters (re-experiencing, avoidance, negative alterations in cognitions and mood, alterations in arousal) with 20 symptoms, the ICD-11 has conceptualized PTSD within only 3 clusters (re-experiencing, avoidance, sense of current threat) with 6 core symptoms. Additionally, the ICD-11 includes a sibling diagnosis of Complex PTSD (CPTSD), which comprises the ‘core’ symptoms of PTSD plus additional symptoms in the domain called disturbances in self-organization (DSO) (affect dysregulation, negative self-concept, and disturbances in relationships). The revisions to diagnostic criteria in DSM-5 and ICD-11 take opposing positions on the conceptualization of post-traumatic presentations that include affect dysregulation, negative self-concept and disturbances in self-organization. Whereas the DSM-5 expanded PTSD to encompass symptoms in these domains, ICD-11 defines a separate category of CPTSD (Lehrner & Yehuda, Citation2020). In comparison with the DSM-5, which includes a list of specific symptoms and diagnostic algorithms for a diagnosis, the ICD-11 focusses on clinical utility and facilitates the identification and treatment by frontline health workers providing only broad formulations and narrative descriptions of disorders, which include the essential features/symptoms of disorders (Cloitre et al., Citation2018; First, Reed, Hyman, & Saxena, Citation2015). Consistent with these beforementioned principles, the ICD-11 workgroup for disorders specifically associated with stress developed a set of 12 symptoms and simple diagnostic rules to measure PTSD and CPTSD and maximize ease of use in clinical and research settings (Cloitre et al., Citation2018).
Recent research on the consequences of these differences in diagnostic criteria provide preliminary evidence that the ICD-11 PTSD formulation results in lower diagnostic rates of PTSD in treatment-seeking children and adolescents compared with ICD-10 (Sachser & Goldbeck, Citation2016), DSM-IV (Eilers et al., Citation2020) and DSM-5 (Bruckmann, Haselgruber, Sölva, & Lueger-Schuster, Citation2020; Sachser et al., Citation2018). Evidence accumulates that CPTSD is empirically distinguishable from PTSD following the ICD-11 approach in different samples of traumatized children and adolescents (Haselgruber, Sölva, & Lueger-Schuster, Citation2020a; Hébert & Amédée, Citation2020; Kazlauskas et al., Citation2020; Sachser, Keller, & Goldbeck, Citation2017). Therefore, it seems important for an instrument measuring posttraumatic stress in children and adolescents to be able to differentiate PTSD from CPTSD. Child and adolescent self-report measures have been developed, adapted and validated to measure PTSD according to DSM-5 (Foa, Asnaani, Zang, Capaldi, & Yeh, Citation2018; Kaplow et al., Citation2020; Sachser et al., Citation2017a) and ICD-11 (Haselgruber et al., Citation2020a, Citation2020b; Kazlauskas et al., Citation2020); however, there is no validated measure which allows clinicians and researchers to crosswalk between diagnostic systems. This is important for clinicians who may need information on both algorithms depending on the clinical context in which they work (e.g. applicability and legal force of the two diagnostic systems). Further, this will help researchers investigate PTSD in children by allowing for an empirical comparison of validity and utility of the diagnostic systems, which can contribute to advances in the field (First et al., Citation2021).
Based on the original Child and Adolescent Trauma Screen (CATS, Sachser et al., Citation2017a), which closely followed DSM-5 criteria, we developed the Child and Adolescent Trauma Screen 2 (CATS-2), which assesses both DSM-5 and ICD-11 diagnostic criteria for PTSD and CPTSD in children and adolescents. The original CATS is an open access instrument, which has proven its usefulness both in research and clinical practice in over 41 empirical studies in various populations (different trauma types, age groups, languages and ethnical/racial populations) (Dowdy-Hazlett, Killian, & Woods, Citation2021). Therefore, the present psychometric study aimed (1) to test the reliability of the CATS-2 scales (DSM−5PTSD; ICD−11PTSD and ICD−11CPTSD); (2) to investigate the convergent-divergent validity pattern of the CATS-2 scales; (3) to test the factorial validity/symptom structure of the CATS-2 scales; (4) to demonstrate the ability of the CATS-2 to differentiate PTSD and CPTSD according to ICD-11; and (5) to establish empirically derived clinical cutoff scores for a probable PTSD diagnosis according to DSM-5 and ICD-11, using the clinical interview Clinician-Administered PTSD Scale for DSM-5 (CAPS-CA-5) as a criterion in a clinical sample of traumatized children and adolescents.
2. Method
2.1. Study design
In 2017 our international workgroup from Germany, Norway and U.S.A. began revising the CATS to measure ICD-11 PTSD and CPTSD in addition to capturing the DSM-5 PTSD criteria. Based on the DSM-5 and ICD-11 criteria and our experiences in using the CATS in clinical practice and research with traumatized children, we carefully reviewed the existing DSM-based CATS measure and current alternative measures for children and adolescents. First, we adapted the event checklist of the CATS refining some items (e.g. sexual abuse) based on feedback we received from therapists, children and caregivers. We then included severe bullying and cyberbullying in the event checklist as studies have revealed strong associations between bullying and PTSD symptoms in children (Idsoe et al., Citation2021). Second, we adapted the symptom list to measure ICD-11PTSD and ICD-11CPTSD while having only minor changes to the DSM-5 formulations (e.g. split up the negative cognitions about the self or world into 4 different items). We then pilot-tested the CATS-2 with traumatized children and their caregivers in clinical practice and reviewed wordings several times to ensure that items are covering the important features and essential meaning of the diagnostic symptoms, while at the same time using developmentally appropriate and simplified language for children. The goal was to achieve a reading level that was commensurate with the lowest age for self-report (7–8 years). After the pilot testing of the English version had proven good feasibility and acceptance, we performed translations into the respective target languages (German, Norwegian) following the WHO criteria: Forward translation, independent expert panel back-translation, resolving discrepancies, pre-testing and cognitive interviewing (think aloud method) (WHO, Citation2018).
Before conducting the study, all staff who conducted the assessments in the three countries had to undergo a formal training before implementing the measures with study participants and their caregivers. This four-hour standardized workshop included general information on traumatic experiences, diagnostic criteria for DSM-5 PTSD, ICD-11 PTSD and CPTSD and frequent comorbid disorders; and training in implementing, scoring and interpreting the measures, as well as the clinical interview CAPS-CA-5. The trained interviewers then implemented the study after two supervised training cases. All participants were patients at the participating sites and had to fulfil the following inclusion criteria: (1) age between 7 and 17 years, (2) sufficient language skills in the target language, and (3) report at least one traumatic event which was measured via the event checklist of the CATS-2. An a priori sample size calculation using the software easyROC with a minimal expected AUC at .70 and an allocation ratio of 2.5 (40% positive cases) revealed that a sample of 60 participants is sufficient to obtain statistical power at the recommended .80 level (α = 0.05). The study protocol was approved by the ethics committees at the relevant sites prior to data collection (Ulm: #95/18, approved 5/17/18; Oklahoma: #10019, approved 5/9/19; Oslo: #61092, approved 12/17/18). Informed consent from legal guardians and informed assent of children and adolescents were obtained prior to the assessment at all sites.
2.2. Participants and sample description
A total of N = 283 children and adolescents completed the questionnaires with additional N = 255 caregiver-reports. The mean age of the sample was 12.2 years (SD = 3.16) with 59% being female. The first 121 participants included in the study sites additionally underwent the diagnostic interview. Fifty-four percent of the participants fulfilled DSM-5 criteria for PTSD and 30% fulfilled ICD-11 criteria for PTSD. A detailed description of the total sample and sample characteristics by study site can be seen in .
Table 1. Descriptive characteristics of the study samples.
2.3. Measures
Child and Adolescent Trauma Screen 2 (CATS-2). The CATS-2 measures potentially traumatic events (PTEs), posttraumatic stress symptoms (PTSS) and impairment according to DSM-5 and ICD-11 criteria in children and adolescent from 7 to 17 years. In addition to a self-report measure for children and adolescents, the CATS-2 includes a parallel caregiver version. First, PTEs are assessed using a 15-item structured PTE checklist. The PTE checklist follows the definitions of traumatic events in the DSM-5 and ICD-11 and includes items assessing natural disasters, serious accidents, experiencing or seeing violence at home or in the community, sexual abuse (off- and online), bullying, cyberbullying, traumatic loss, medical procedures and war. Participants can indicate whether they had experienced the event by checking ‘yes’ or ‘no’. Those respondents with at least one PTE, posttraumatic stress symptoms in the last four weeks are then assessed using 25 items rated on a 4-point Likert scale with the following anchors: 0 = ‘Never’, 1 = ‘Sometimes’, 2 = ‘Often’, 3 = ‘Almost Always’. Based on the fact that most other child and adolescent symptom measures and the previous version of the CATS are measuring symptom frequency rather than intensity or impairment, we made the decision to use a frequency scale for rating the symptoms. The 25 items map directly onto the diagnostic criteria for PTSD in DSM-5 and ICD-11 PTSD and CPTSD. The DSM-5 PTSD severity score is the sum of items 1–20 (range 0–60), including only the highest score of items 9, 10, 15. The ICD-11 PTSD severity score (range 0–18) is the sum of items 2, 3, 6, 7, 17, 18. The ICD-11 CPTSD score (range 0–36) is the sum of ICD-11 PTSD severity score (2, 3, 6, 7, 17, 18) plus the sum of the ICD-11 DSO severity score (9b, 9d, 10a, 13, 14, 15a) (for more information on the items please see OS1). In addition, or as an alternative to this dimensional scoring approach, the clinician can apply a more categorical item-mapping approach to follow the diagnostic algorithms of the DSM-5 and ICD-11 with a symptom being rated as present for values of 2 = ‘Often’ or 3 = ‘Almost Always’. Impairment in psychosocial functioning is assessed via five ‘yes’ or ‘no’ items that ask whether the previously rated posttraumatic stress symptoms interfere with five key areas of functioning (getting along with others, school/work, hobbies, family relationships, and general happiness).
Short Mood and Feelings Questionnaire (SMFQ). The SMFQ is a short measure of symptoms of depression in children and adolescents and is validated for use in children aged 6 years and up (Angold and Costello, Citation1987). The self-report version of the SMFQ was used for this study, which consists of 13 items measured on a 3-point Likert scale with 0 = ‘Not true’, 1 = ‘Sometimes’ and 2 = ‘True’ (range 0–26). Scoring a 12 or higher on the short version indicates the presence of depression. The SMFQ has demonstrated excellent reliability in the total study sample (α = .91).
Pediatric Symptom Checklist (PSC-17). The PSC-17 is a brief questionnaire that assesses emotional or behavioural problems in the domains of attention, internalizing problems and externalizing problems using 17 items rated as 0 = ‘Never’, 1 = ‘Sometimes’ and 2 = ‘Often’ (Gardner et al., Citation1999). The caregiver version was used for this study. Scores ≥ 15 on the total scale (range 0–34), ≥ 7 on the attention scale (range 0–10), ≥ 5 on the internalizing scale (range 0–10) and ≥ 7 on the externalizing scale (range 0–14) indicate an increased likelihood of a behavioural health disorder being present. Internal consistency was good for the total scale and all three subscales (total scale: α = .81, attention problems: α = .79, externalizing problems: α = .81, internalizing problems: α = .81) in the current sample.
The Clinician-Administered PTSD Scale for DSM-5 – Child and Adolescent Version (CAPS-CA-5). The CAPS-CA-5 is a semi-structured 30-item clinician administered interview to assess PTSD based on DSM-5 criteria for children and adolescents of age 7 years and older (Pynoos et al., Citation2015). The assessor combines information about frequency and intensity of a symptom into a single severity rating (0 = ‘absent’, 1 = ‘mild/subthreshold’, 2 = ‘moderate/threshold’, 3 = ‘severe/markedly elevated’, 4 = extreme/incapacitating‘).The scoring follows the DSM-5 criteria for PTSD with a symptom rating of ≥ 2 being indicative of the presence of a symptom. The internal consistency was excellent for the 20-item DSM-5 scale (range 0–80) with α = .91. For the study we also approximated ICD-11 PTSD diagnosis based on the six ICD-11 PTSD symptoms (2,3,6,7,17,18). The internal consistency was adequate for the 6-item ICD-11 scale (range 0–24) with α = 74.
2.4. Statistical analyses
First, sample and item descriptives were calculated. Next, internal consistency (Cronbach's α) was analyzed for the DSM-5 PTSD scale, the ICD-11 PTSD scale and the ICD-11 CPTSD scale for the self- and the caregiver-report. The construct validity (convergent-divergent validity pattern) for the self- and the caregiver-report scales was investigated using bivariate correlations between the CATS-2 scales, the MFQ and the PSC-17 subscales of internalizing, externalizing and attention.
Overall, there was very little missing data present in the CATS-2 (0.2%), MFQ (8.7%), PSC-17 (0.3%) and CAPS-CA-5 (0%). Missing data were handled using the full information maximum likelihood method using Mplus.
Factor structure – Confirmatory Factor Analyses (CFA). The latent structure of the CATS-2 scales of ICD-11PTSD and ICD-11CPTSD was investigated via confirmatory factor analyses (CFA). For the CFA of the DSM-5PTSD scale of the CATS see Sachser and colleagues (Citation2018). For the ICD-11CPTSD scale we specified 4 alternative models of the symptom structure based on the available evidence of competing models in the literature (Haselgruber, Sölva, & Lueger-Schuster, Citation2020b): Model 1: Unidimensional-PTSD Model (12 items loading on a single factor); Model 2: Six-Factor First-Order-Model (six correlated first-order factors: re-experiencing, avoidance, threat, affective dysregulation, negative self-concept, disturbances in relationships); Model 3: Single-Factor-Second-Order-Model (six first-order factors and one second-order factor); Model 4: Two-Factor-Second-Order-Model (six first-order factors and two correlated second-order factors) (see ). Each model was estimated in Mplus (version 7.31) using robust maximum likelihood estimation (MLR; Yuan & Bentler, Citation2000). As the standardized factor loading of the affect dysregulation factor on the disturbances in self-organization (DSO) factor in model 3 and model 4 was ≥ 1, we fixed this loading to one. All other model parameters (estimates, standard errors and test statistics) were correct in all other models. Model fit was then evaluated using chi-square goodness of fit statistic (χ2), comparative fit index (CFI), Tucker-Lewis index (TLI), standardized root mean square residual (SRMR); root mean square error of approximation (RMSEA 90% CI). Model fit was evaluated following the recommendations of Kline (Citation2015): CFI and TLI values of >0.90 indicate acceptable model fit, whereas values of >0.95 indicate excellent fit. RMSEA and SRMR values of <0.10 reflect acceptable fit, whereas values of <0.05 indicate excellent model fit. For comparison of non-nested models, the Bayesian information criterion (BIC) was used, which penalizes less parsimonious models. A BIC-value of 10 points smaller indicates a better fitting model (Raftery, Citation1995).
Figure 1. Alternative models of ICD-11 CPTSD using the CATS-2: Loading patterns of confirmatory factor analyses.
Note. PTSD: Posttraumatic Stress Disorder; CPTSD: complex PTSD; RE: Re-experiencing; Av: Avoidance; Th: Sense of Threat; AD: Affect Dysregulation; NSC: Negative Self-concept; DR: Disturbances in Relationships.
Latent profile analyses (LPA). A series of latent profile analyses (LPAs, Mplus 7.31) using the twelve ICD-11PTSD/CPTSD items of the CATS-2 was performed to explore the ability of the CATS-2 to empirically distinguish ICD-11PTSD and ICD-11CPTSD. The optimal number of extracted profiles was based on theoretical assumptions, meaningfulness, class size, parsimony and fit indices: Lo-Mendell-Rubin adjusted likelihood ratio test (LMRA-LRT; Lo, Mendell, & Rubin, Citation2001), bootstrap likelihood ratio test (BLRT), the Bayesian information criterion (BIC), and the Akaike information criterion (AIC). As the AIC and BIC declined with addition of every new class and BLRT was highly significant up to 5 classes we followed the recommendations of Asparouhov and Muthén (Citation2015) and calculated multivariate normal mixture models, where we allowed the continuous variables to correlate within class.
Sensitivity Analyses. The diagnostic accuracy of the DSM-5PTSD scale and ICD-11PTSD scale were explored using receiver operating characteristics (ROC) analyses in the self-report sample. The CAPS-CA-5 was used as the gold standard for PTSD diagnosis against which the CATS-2 DSM-5PTSD and the ICD-11PTSD scale was assessed. To determine the cut-offs of the CATS-2 DSM-5PTSD scale and the CATS-2 ICD-11PTSD scale we inspected the sensitivity, specificity, positive predictive value, negative predictive value and Youden's J statistic (Youden, Citation1950).
3. Results
Sample and item descriptives. Item descriptions, item mappings, item means, standard deviations, skew, kurtosis, range and the discrimination index for the self- and caregiver reports can be found in the online supplement (OS1, OS2, OS7, OS8). Descriptive characteristics of the study sample (N = 283) can be obtained in .
Internal consistency. The self-report DSM-5PTSD score (α = .89), the ICD-11PTSD score (α = .67) and the ICD-11CPTSD score (α = .83) of the CATS-2 proved adequate to excellent reliability in the total sample. Regarding the caregiver-report, the DSM-5PTSD score (α = .91), the ICD-11PTSD score (α = .79) and the ICD-11CPTSD score (α = .87) of the CATS-2 proved good to excellent reliability in the total sample. All reliability scores can be seen in .
Table 2. Convergent-divergent validity pattern.
Construct validity. For the self-reports the total score of the CATS-2 DSM-5PTSD score, the ICD-11PTSD score and the ICD-11CPTSD score showed strong correlations with the CAPS-CA-5 total scale. The convergent-divergent validity pattern of the self- and caregiver-reported questionnaires is depicted in . Regarding the self-reports, the CATS-2 DSM-5PTSD score was strongly correlated with the ICD-11PTSD score, the ICD-11DSO score and the ICD-11CPTSD score. The same pattern was found for the caregiver-reported CATS-2 scales. Between the self-reported and caregiver-reported CATS-2 scales we found weak to moderate correlations. Regarding the divergent validity, self-reported depression (SMFQ) showed strong correlations with the self-reports of DSM-5PTSD, and ICD-11 CPTSD and moderate correlations with ICD-11PTSD and the ICD-11DSO cluster. Caregiver-reported internalizing symptoms showed a moderate correlation with the caregiver-reported CATS-2 scales and low correlation with the self-reported CATS-2 scales. Caregiver-reported attention problems and externalizing symptoms showed no significant associations with the self-reported CATS-2 scales. Weak correlations were found between the caregiver-reported CATS-2 DSO scale and attention and externalizing symptoms.
Factor structure. The results of the CFA for the CATS-2 ICD-11CPTSD scale showed that the six-factor first-order model (model 2), the single-factor second-order model (model 3) and the two-factor second-order model (model 4) were excellent representations of the factor structure in the self-report and the caregiver-report sample. After a detailed comparison of fit indices (see ), the two-factor second-order model (model 4) was selected as the best fitting model given the highest CFI and TLI and the lowest RMSEA and BIC for the caregiver-report sample. In the self-report model 2, model 3 and model 4 revealed almost the same fit indices, with the lowest BIC for model 3 and model 4. To align measurement between self- and caregiver report we selected the two-factor second-order model (model 4) as the best fitting model for the self-report. For the two-factor second-order model (model 4) all items loaded significantly onto the factor of the respective symptom cluster in both the self- and the caregiver-report. Also, the first-order factors loaded significantly onto the respective second-order factors (see OS3 and OS4).
Table 3. Goodness of fit indices for the ICD-11 PTSD / CPTSD models.
Latent Profile Analyses (LPA). The fit indices of the latent profile analyses with the 12 ICD-11PTSD/CPTSD items are shown in . As AIC and BIC did not reach a minimum within 1–5 classes and the BLRT was still significant with addition of every class we decided to calculate multivariate normal mixture models (MNMM), in which we allowed the continuous variables to correlate within class. Based on the fit indices, class size, theoretical assumptions, parsimony and meaningfulness, the 2-class MNMM model was selected as the best fitting model. The plot of the 2-class model can be seen in . Class 1 (63%) was labelled ‘PTSD’ as indicated by medium scores on the ICD-11PTSD core symptoms and low to medium scores on the ICD-11DSO symptoms. Class 2 (37%) showed medium to high scores on the ICD-11PTSD core symptoms and medium to high scores in the ICD-11DSO symptoms and was, therefore, labelled as ‘CPTSD’ class.
Table 4. Latent profile analyses and multivariate normal mixture models with fit indices for the ICD-11 CPTSD items.
Figure 2. Plot of the 2-class multivariate normal mixture model.
Note. 2-class multivariate normal mixture model based on the CATS-2 self-report (n = 283); PTSD: Posttraumatic Stress Disorder; CPTSD: complex PTSD; RE: Re-experiencing; Av: Avoidance; Th: Sense of Threat; AD: Affect Dysregulation; NSC: Negative Self-concept; DR: Disturbances in Relationships.
Sensitivity analyses. The ROC-curve analysis of the CATS-2 DSM-5PTSD self-report scale validated against the CAPS-CA-5 diagnosis has proven a satisfactory accuracy (AUC = .88; n = 121; see OS5). Inspecting sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and Youden's J we found a cut-off of ≥ 21 points on the CATS-2 DSM-5PTSD scale to be a sensitive screening cut-off for probable PTSD and a cut-off of ≥ 25 points to be a more specific cut-off for PTSD diagnosis (see ). The ROC-curve analysis of the CATS-2 ICD-11PTSD self-report scale validated against the probable CAPS-CA-5 ICD-11PTSD diagnosis has proven a satisfactory accuracy (AUC = .84; n = 121; see OS6). Inspecting sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and Youden's J we found a cut-off of ≥ 7 points on the CATS-2 ICD-11PTSD scale to be a sensitive screening cut-off for probable PTSD and a cut-off of ≥ 9 points to be a more specific cut-off of for PTSD diagnosis (see ).
Table 5. Sensitivity, specificity, positive predictive value and negative predictive value for different CATS-2 cut-off values (n = 121).
4. Discussion
The current study investigated the psychometric properties of the CATS-2 as a measure of PTSD according to DSM-5 and ICD-11 PTSD and CPTSD. All CATS-2 scales have demonstrated adequate to excellent internal consistency in the self- and the caregiver-report version. CATS-2 scales demonstrated conceptually meaningful patterns of convergent-divergent validity with measures of depression, internalizing, externalizing and attention symptoms. Similar to other studies using the International Trauma Questionnaire (ITQ) in child and adolescent samples (Haselgruber et al., Citation2020a, Citation2020b), we found that the symptom structure of ICD-11CPTSD was best reflected by the two-factor second-order model, with PTSD and DSO as correlated second-order factors, with three first-order factors each, for the self- and caregiver-report. One other study, however, found the six-factor first-order model best fitting in adolescents from the general population (Kazlauskas et al., Citation2020). In line with these differences, in studies with adults, evidence accumulates that the six-factor first-order model is best fitting in population or convenience samples (Ben-Ezra et al., Citation2018; Ho et al., Citation2019) and the two-factor second-order model best fitting in clinical samples (Cloitre et al., Citation2018; Hyland et al., Citation2017). Using the approach of a latent profile analyses, the CATS-2 was able to differentiate a PTSD and a CPTSD class according to ICD-11 in the current sample. However, the distinction of the profile was not as clear as in previous studies, which might be the result of not having PTSD and CPTSD items separated and ordered as in the ITQ. In line with other studies (Haselgruber et al., Citation2020; Sachser et al., Citation2017b), the hyperactivation item of affect dysregulation poorly discriminated between PTSD and CPTSD. Sensitivity analyses revealed that a CATS-2 DSM-5PTSD score of ≥ 21 was a sensitive cut-off optimal for screening purposes, while a score of ≥ 25 demonstrated higher specificity for probable PTSD diagnoses. For the ICD-11PTSD scale a score of ≥ 7 was a sensitive cut-off optimal for screening purposes, while a score of ≥ 9 demonstrated higher specificity for probable PTSD diagnoses.
Altogether our findings indicate that the CATS-2 is a psychometrically sound measure of DSM-5PTSD and ICD-11PTSD/CPTSD and, with the established cut-offs, well suited for screening and assessing the diagnostic status and symptom severity in traumatized children and adolescents. One of the major strengths is the international development and that the study sample comprises a clinical sample of children and adolescents with different age ranges, trauma history and levels of posttraumatic stress symptoms, resulting in good external validity for the findings. Several other language versions have already been created and will undergo validation soon. The inclusion of bullying and cyberbullying will stimulate further research on the applicability of these events as DSM-5 criterion A stressors for children and adolescents, as recent research in child and adolescent psychotraumatology and bullying points towards the direction that this might be warranted (Birkeland, Skar, & Jensen, Citation2022; Idsoe et al., Citation2021; Ossa, Pietrowsky, Bering, & Kaess, Citation2019).
Although the results on the psychometric properties of the CATS-2 are promising, several limitations should be considered. First, as there were no validated interviews for children and adolescents measuring ICD-11CPTSD when starting the study, the validation of the cut-off for the CATS-2 CPTSD scale is pending. Due to limited resources for the study we were able to perform the CAPS-CA-5 validation interviews only for the first included cases in each respective study site, resulting in 121 interviews. As we have not recorded the interviews and there was only one rater at one measurement time point interrater and intrarater reliability could not be measured. However, the sample size is appropriate for the ROC analyses according to our power calculation. Unfortunately, we have not used an a priori power calculation to determine the sample size needed for our CFA and LCA. While stating this as a clear limitation, the sample sizes of n = 283 and n = 255 seem adequate for those types of analyses (see Wolf, Harrington, Clark, & Miller, Citation2013). With regard to the cultural, ethnical and racial diversity the study is limited by a predominantly white sample and almost no individuals of Asian descent. Race and Ethnicity were not assessed in the Norwegian and German sample. Therefore, validation of the CATS-2 in other specific or more diverse cultural backgrounds is pending.
Additionally, we have not adapted the original CATS DSM-5 pre-school version (3–6 years) to also capture CPTSD criteria, as there was no evidence that the CPTSD construct following ICD-11 is applicable for children in this age group. As the CATS-2 and most other child and adolescent trauma screeners use frequency ratings for assessment of symptoms, and the ITQ-CA uses an intensity/distress rating, future research should compare the effect of different rating anchors. Another crucial next step is to investigate possible differences in item functioning or other psychometric properties comparing different language versions (e.g. measurement invariance).
Based on our results clinicians may use our measure for screening and symptom monitoring of their patients as the rate of traumatic events and PTSS is high in clinical samples and routine trauma screening is recommended in several guidelines (Cohen & Issues, Citation2010). Additionally, a prior study using the CATS shows that trauma screening is tolerated well by children and adolescents (Skar, Ormhaug, & Jensen, Citation2019). The CATS-2 provides a useful and license free tool for clinicians and researchers to assess traumatic events and PTSS following the DSM-5 and the ICD-11 conceptualization of PTSD and CPTSD.
Supplemental Material
Download MS Word (61 KB)Disclosure statement
No potential conflict of interest was reported by the author(s).
Acknowledgements
The authors thank all participating children, adolescents and caregivers. We also want to thank Elisabeth Pollio, Beth Cooper and Prof. Dr. Esther Deblinger from the CARES Institute, Rowan University School of Osteopathic Medicine for their help refining the CATS items and drafting the CATS-2 versions. Further thanks are due to Prof. Ulrich Voderholzer for supporting recruitment within the Schön Klinik Roseneck for the German sample.
Data availability statement
Data assessment was conducted in the course of a research project under different legal backgrounds. For legal reasons, primary data cannot be published online, as we had not included this in our informed consent and assent for participants. The dataset generated during and analyzed for the current study is available from the corresponding author after complying with data protection agreements.
Additional information
Funding
References
- American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596
- Angold, A., & Costello, E. J. (1987). Mood and feelings questionnaire: Short version. Duke University.
- Asparouhov, T., & Muthén, B. (2015). Residual associations in latent class and latent transition analysis. Structural Equation Modeling: A Multidisciplinary Journal, 22(2), 169–177. doi: 10.1080/10705511.2014.935844
- Ben-Ezra, M., Karatzias, T., Hyland, P., Brewein, C. R., Cloitre, M., Bisson, J. I., … Shevlin, M. (2018). Posttraumatic stress disorder (PTSD) and complex PTSD (CPTSD) as per ICD-11 proposals: A population study in Israel. Depression and Anxiety, 25(3), 264–274. doi: 10.1002/da.22723
- Birkeland, M. S., Skar, A. M. S., & Jensen, T. K. (2022). Understanding the relationships between trauma type and individual posttraumatic stress symptoms: A cross-sectional study of a clinical sample of children and adolescents. Journal of Child Psychology and Psychiatry. Advance Online Publication. https://acamh.onlinelibrary.wiley.com/doi/full/10.1111/jcpp.13602
- Bruckmann, P., Haselgruber, A., Sölva, K., & Lueger-Schuster, B. (2020). Comparing rates of ICD-11 and DSM-5 posttraumatic stress disorder in Austrian children and adolescents in foster care: Prevalence, comorbidity and predictors. European Journal of Psychotraumatology, 11(1), 1767988. doi: 10.1080/20008198.2020.1767988
- Cloitre, M., Shevlin, M., Brewin, C. R., Bisson, J. I., Roberts, N. P., Maercker, A., … Hyland, P. (2018). The international trauma questionnaire: Development of a self-report measure of ICD-11 PTSD and complex PTSD. Acta Psychiatrica Scandinavica, 138(6), 536–546. doi: 10.1111/acps.12956
- Cohen, J. A., Issues, T. W. G. O. Q., & AACAP work group on quality issues. (2010). Practice parameter for the assessment and treatment of children and adolescents with posttraumatic stress disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 49(4), 414–430.
- Dowdy-Hazlett, T., Killian, M., & Woods, M. (2021). Measurement of traumatic experiences of children within survey and intervention research: A systematic review of the child and adolescent trauma screen. Children and Youth Services Review, 131, 106259.
- Eilers, R., Rimane, E., Vogel, A., Renneberg, B., Steil, R., & Rosner, R. (2020). The impact of the new ICD-11 criteria on abused young people: 30% less PTSD and CPTSD diagnoses compared to DSM-IV. Psychotherapy and Psychosomatics, 89(1), 59–62. doi: 10.1159/000503794
- First, M. B., Gaebel, W., Maj, M., Stein, D. J., Kogan, C. S., Saunders, J. B., … Reed, G. M. (2021). An organization-and category-level comparison of diagnostic requirements for mental disorders in ICD-11 and DSM-5. World Psychiatry, 20(1), 34–51. doi: 10.1002/wps.20825
- First, M. B., Reed, G. M., Hyman, S. E., & Saxena, S. (2015). The development of the ICD-11 clinical descriptions and diagnostic guidelines for mental and behavioural disorders. World Psychiatry, 14(1), 82–90. doi: 10.1002/wps.20189
- Foa, E. B., Asnaani, A., Zang, Y., Capaldi, S., & Yeh, R. (2018). Psychometrics of the child PTSD symptom scale for DSM-5 for trauma-exposed children and adolescents. Journal of Clinical Child & Adolescent Psychology, 47(1), 38–46. doi: 10.1080/15374416.2017.1350962
- Gardner, W. P., Murphy, M., Childs, G., Kelleher, K. J., Pagano, M., Jellinek, M., … Chiappetta, L. (1999). The PSC-17: A brief pediatric symptom checklist with psychosocial problem subscales. A report from PROS and ASPN. Ambulatory Child Health, 5, 225–236.
- Haselgruber, A., Sölva, K., & Lueger-Schuster, B. (2020b). Symptom structure of ICD-11 complex posttraumatic stress disorder (CPTSD) in trauma-exposed foster children: Examining the international trauma questionnaire–child and adolescent version (ITQ-CA). European Journal of Psychotraumatology, 11(1), 1818974. doi: 10.1080/20008198.2020.1818974
- Haselgruber, A., Sölva, K., & Lueger-Schuster, B. (2020a). Validation of ICD-11 PTSD and complex PTSD in foster children using the international trauma questionnaire. Acta Psychiatrica Scandinavica, 141(1), 60–73. doi: 10.1111/acps.13100
- Hébert, M., & Amédée, L. M. (2020). Latent class analysis of post-traumatic stress symptoms and complex PTSD in child victims of sexual abuse and their response to trauma-focused cognitive behavioural therapy. European Journal of Psychotraumatology, 11(1), 1807171. doi: 10.1080/20008198.2020.1807171
- Ho, G. W., Karatzias, T., Cloitre, M., Chan, A. C., Bressington, D., Chien, W. T., … Shevlin, M. (2019). Translation and validation of the Chinese ICD-11 international trauma questionnaire (ITQ) for the assessment of posttraumatic stress disorder (PTSD) and complex PTSD (CPTSD). European Journal of Psychotraumatology, 10(1), 1608718. doi: 10.1080/20008198.2019.1608718
- Hyland, P., Shevlin, M., Brewin, C. R., Cloitre, M., Downer, A. J., Jumbe, S., … Roberts, N. P. (2017). Validation of posttraumatic stress disorder (PTSD) and complex PTSD using the international trauma questionnaire. Acta Psychiatrica Scandinavica, 136(3), 313–322. doi: 10.1111/acps.12771
- Idsoe, T., Vaillancourt, T., Dyregov, A., Hagen, K. A., Ogden, T., & Naerde, A. (2021). Bullying victimization and trauma. Frontiers in Psychiatry, 11, 1602. doi: 10.3389/fpsyt.2020.480353
- Kaplow, J. B., Rolon-Arroyo, B., Layne, C. M., Rooney, E., Oosterhoff, B., Hill, R., … Pynoos, R. S. (2020). Validation of the UCLA PTSD reaction index for DSM-5: A developmentally informed assessment tool for youth. Journal of the American Academy of Child & Adolescent Psychiatry, 59(1), 186–194. doi: 10.1016/j.jaac.2018.10.019
- Kazlauskas, E., Zelviene, P., Daniunaite, I., Hyland, P., Kvedaraite, M., Shevlin, M., & Cloitre, M. (2020). The structure of ICD-11 PTSD and complex PTSD in adolescents exposed to potentially traumatic experiences. Journal of Affective Disorders, 265, 169–174. doi: 10.1016/j.jad.2020.01.061
- Kline, R. B. (2015). Principles and Practice of Structural Equation Modeling. Guilford publications.
- Lehrner, A., & Yehuda, R. (2020). PTSD diagnoses and treatments: Closing the gap between ICD-11 and DSM-5. BJPsych Advances, 26(3), 153–155. doi: 10.1192/bja.2020.10
- Lo, Y., Mendell, N. R., & Rubin, D. B. (2001). Testing the number of components in a normal mixture. Biometrika, 88(3), 767–778. doi: 10.1093/biomet/88.3.767
- Ossa, F. C., Pietrowsky, R., Bering, R., & Kaess, M. (2019). Symptoms of posttraumatic stress disorder among targets of school bullying. Child and Adolescent Psychiatry and Mental Health, 13(1), 1–11. doi: 10.1186/s13034-019-0304-1
- Pynoos, R. S., Weathers, F. W., Steinberg, A. M., Marx, B. P., Layne, C. M., Kaloupek, D. G., … Kriegler, J. A. (2015). Clinician-administered PTSD scale for DSM-5 - Child/adolescent version. National Center for Posttraumatic Stress Disorder and National Center for Child Traumatic Stress. www.ptsd.va.gov.
- Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163. doi: 10.2307/271063
- Sachser, C., Berliner, L., Holt, T., Jensen, T., Jungbluth, N., Risch, E., … Goldbeck, L. (2018). Comparing the dimensional structure and diagnostic algorithms between DSM-5 and ICD-11 PTSD in children and adolescents. European Child & Adolescent Psychiatry, 27(2), 181–190. doi: 10.1007/s00787-017-1032-9
- Sachser, C., Berliner, L., Holt, T., Jensen, T. K., Jungbluth, N., Risch, E., … Goldbeck, L. (2017a). International development and psychometric properties of the child and adolescent trauma screen (CATS). Journal of Affective Disorders, 210, 189–195. doi: 10.1016/j.jad.2016.12.040
- Sachser, C., & Goldbeck, L. (2016). Consequences of the diagnostic criteria proposed for the ICD-11 on the prevalence of PTSD in children and adolescents. Journal of Traumatic Stress, 29(2), 120–123. doi: 10.1002/jts.22080
- Sachser, C., Keller, F., & Goldbeck, L. (2017b). Complex PTSD as proposed for ICD-11: Validation of a new disorder in children and adolescents and their response to trauma-focused cognitive behavioral therapy. Journal of Child Psychology and Psychiatry, 58(2), 160–168. doi: 10.1111/jcpp.12640
- Skar, A. M. S., Ormhaug, S. M., & Jensen, T. K. (2019). Reported levels of upset in youth after routine trauma screening at mental health clinics. JAMA Network Open, 2(5), e194003–e194003. doi: 10.1001/jamanetworkopen.2019.4003
- Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913–934. doi: 10.1177/0013164413495237
- World Health Organization. (2018, May). WHO Guidelines on Translation and Adaptation of Instruments. WHO. http://www.who.int/substance_abuse/research_tools/translation/en/.
- Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32–35. doi: 10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3
- Yuan, K. H., & Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30(1), 165–200. doi: 10.1111/0081-1750.00078