Psychometric properties of post-traumatic stress disorder (PTSD) checklist for DSM-5 in persons with serious mental illness

ABSTRACT

Background

PCL-5 is a self-report measure consisting of 20 items that are used to assess the symptoms of Post-Traumatic Stress Disorder (PTSD) according to the DSM-5.

Objective

This study evaluated the factor structure of the Post-Traumatic Stress Disorder (PTSD) Checklist for DSM-5 (PCL-5) in people with serious mental illness.

Method

The sample in Study 1 included 536 participants with serious mental illness who were receiving supported employment services through community mental health agencies or supported housing programmes. Confirmatory factor analysis assessed the fit of six different models of PTSD.

Results

Results indicated that Armour’s Hybrid 7-factor model composed of re-experiencing, avoidance, dysphoria, dysphoric arousal, anxious arousal, negative affect, anhedonia, and externalizing behaviours demonstrated the best fit. Study 2 found support for convergent validity for PCL-5 among 132 participants who met criteria for PTSD.

Conclusion

Findings provide support for the psychometric properties of the PCL-5 and the conceptualization of the 7-factor hybrid model and the 4-factor DSM-5 model of PTSD among persons living with serious mental illness.

HIGHLIGHTS

• This study evaluated the factor structure of the Post- Traumatic Stress Disorder (PTSD) Checklist for DSM-5 (PCL-5) in people with serious mental illness (SMI).

• Findings support the psychometric properties of the PCL-5 and the seven- symptom-clusters of PTSD in persons with SMI.

• The PCL-5 was found to be psychometrically sound among persons with psychiatric diagnoses receiving community mental health services, as evidenced by excellent internal consistency and convergent/divergent validity for both the 4-factor DSM-5 model and the 7-factor model of PTSD.

Antecedentes:PCL-5 es una medida de autoinforme que consta de 20 ítems que se utilizan para evaluar los síntomas del TEPT de acuerdo al DSM-5.

Objetivo:Este estudio evaluó la estructura factorial de la Lista de verificación de Trastorno de Estrés Postraumático (TEPT) para DSM-5 (PCL-5) en personas con enfermedades mentales graves.

Método:La muestra del Estudio 1 incluyó a 536 participantes con enfermedad mental grave que estaban recibiendo servicios de empleo subvencionado a través de agencias comunitarias de salud mental o programas de vivienda subvencionados. El análisis factorial confirmatorio evaluó el ajuste de seis modelos diferentes de TEPT.

Resultados:Los resultados indicaron que el modelo híbrido de 7 factores de Armour - compuesto de reexperimentación, evitación, disforia, excitación disfórica, excitación ansiosa, afecto negativo, anhedonia y conductas de externalización - demostró el mejor ajuste. El estudio 2 encontró sustento para la validez convergente de PCL-5 entre 132 participantes que cumplieron con los criterios para TEPT.

Conclusión:Los hallazgos respaldan las propiedades psicométricas del PCL-5 y la conceptualización del modelo híbrido de 7 factores y el modelo DSM-5 de 4 factores de TEPT entre personas que viven con una enfermedad mentales graves.

背景:PCL-5 是一项由 20个条目组成、根据 DSM-5 评估 PTSD 症状的自我报告测量工具。

目的:本研究评估了严重精神障碍患者中 DSM-5 创伤后应激障碍 (PTSD) 检查表(PCL-5)的因素结构。

方法:研究 1 中的样本包括 536 名通过社区心理健康机构或支持性住房计划接受支持性就业服务的严重精神障碍患者的参与者。验证性因素分析评估了六种不同 PTSD 模型的拟合度。

结果:结果表明, 由再体验、回避、精神痛苦、精神痛苦性唤醒、焦虑性唤醒、负性情绪、快感缺失和外化行为组成的Armour 混合 7 因素模型, 表现出最佳拟合。研究 2 在符合 PTSD 标准的 132 名参与者中发现支持 PCL-5 的收敛效度

结论:研究结果为 PCL-5 的心理测量学特性以及严重精神疾病患者 PTSD 的 7 因子混合模型和 4 因子 DSM-5 模型的概念化提供了支持。

KEYWORDS:

• PTSD
• confirmatory factor analysis
• serious mental illness
• factor structure
• PTSD checklist

PALABRAS CLAVE:

• TEPT
• análisis factorial confirmatorio
• enfermedad mental grave
• estructura factorial
• Lista de verificación de TEPT

关键词:

• PTSD
• 验证性因素分析
• 严重精神障碍
• 因素结构
• PTSD检查表

1. Confirmatory factor analysis of PTSD Checklist in persons with serious mental illness

The PTSD Checklist for the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; PCL-5) (Weathers et al., 2013a) is a commonly used scale for screening individuals with PTSD and assessing severity of PTSD symptoms. The PCL-5 contains 20 items corresponding to 20 symptoms of PTSD outlined in the DSM-5. A large number of studies have used confirmatory factor analyses (CFA) to examine the factor structure of the PCL-5 for various models of PTSD (Armour et al., 2015; Ashbaugh, Houle-Johnson, Herbert, El-Hage, & Brunet, 2016; Cheng et al., 2020; Eddinger & McDevitt-Murphy, 2017; Krüger-Gottschalk et al., 2017; Lee et al., 2019; Liu et al., 2014; Van Praag, Fardzadeh, Covic, Maas, & von Steinbüchel, 2020), in a range of diverse populations. Prior research, however, has not evaluated the factor structure of the PCL-5 among individuals with serious mental illness (SMI), which is commonly defined as ‘having (within the past year) a diagnosable mental, behavior, or emotional disorder that causes serious functional impairment that substantially interferes with or limits one or more major life activities.’ (Substance Abuse and Mental Health Services Administration (SAMHSA), 2017). Although clear consensus is lacking in its definition (Martínez-Martínez, Richart-Martínez, & Ramos-Pichardo, 2020), SMI has traditionally been linked with schizophrenia, bipolar disorder, and treatment refractory major depression (Parabiaghi, Bonetto, Ruggeri, Lasalvia, & Leese, 2006; Ruggeri, Leese, Thornicroft, Bisoffi, & Tansella, 2000; Ellison, Russinova, Lyass, & Rogers, 2008; Russinova, Bloch, Wewiorski, Shappell, & Rogers, 2018, Grubaugh, Brown, Wojtalik, Myers, & Eack, 2021) and most states in the U.S. define SMI as any major psychiatric disorder that is accompanied by persistent impairment in functioning.

One reason why an evaluation of the factor structure of the PCL-5 among people with SMI is needed is that adverse childhood experiences (ACEs) are associated with an increased risk of developing SMI (Breslau et al., 1998; Loewy et al., 2019; Lu, Mueser, Rosenberg, & Jankowski, 2008; Rosenberg et al., 2001), and there are substantially elevated rates of co-occurring PTSD among people with SMI compared to the general population (Breslau et al., 1998; Howgego et al., 2005; Kessler, Chiu, Demler, & Walters, 2005; Mueser, Essock, Haines, Wolfe, & Xie, 2004; Mueser et al., 1998). At the same time, there is evidence for an under-detection of PTSD among people with SMI, which may be partly the result of overlap between PTSD and other symptoms related to SMI such as persecutory ideas, depression and suicidality, mania, and neurocognitive deficits (Grubaugh, Elhai, Cusack, Wells, & Frueh, 2007; Mueser et al., 1998; Zammit Lewis et al., 2018). Furthermore, there is evidence that traumatic events such as childhood sexual abuse are related to increased frequency and severity of psychotic symptoms (Muenzenmaier et al., 2015; Shevlin, Dorahy, & Adamson, 2007; Varese et al., 2012). This suggests a need to evaluate whether the factor structure of PTSD symptoms in people with SMI differs from the general population or other populations of trauma survivors.

Past research on the PCL-5 reported different factor structures (Armour et al., 2015; Ashbaugh et al., 2016; Cheng et al., 2020; Eddinger & McDevitt-Murphy, 2017; Elhai et al., 2011; King, Leskin, King, & Weathers, 1998; Krüger-Gottschalk et al., 2017; Lee et al., 2019; Liu et al., 2014; Simms, Watson, & Doebbelling, 2002; Tsai et al., 2015; Van Praag et al., 2020), summarized in Table 1. The DSM-5 4-factor model proposes re-experiencing, avoidance, negative alterations in cognition and mood, and alterations in arousal and reactivity (American Psychiatric Association, 2013), and received support in studies on PCL-5 (Armour et al., 2015; Ashbaugh et al., 2016; Eddinger & McDevitt-Murphy, 2017; Krüger-Gottschalk et al., 2017; Lee et al., 2019; Liu et al., 2014; Tsai et al., 2015; Van Praag et al., 2020). The DSM-5 4-factor model is consistent with King et al.’s 4-factor model (King et al., 1998), which includes re-experiencing, avoidance, numbing, and alterations in arousal and reactivity. Another 4-factor model, Simms’ 4-factor Dysphoria model (Simms et al., 2002), includes re-experiencing, avoidance, dysphoria, and arousal, and has also received support from some studies on the PCL-5 (Cheng et al., 2020; Contractor, Caldas, Dolan, Lagdon, & Armour, 2018; Eddinger & McDevitt-Murphy, 2017; Krüger-Gottschalk et al., 2017; Lee et al., 2019; Liu, Wang, Cao, Qing, & Armour, 2016; Liu et al., 2014; Van Praag et al., 2020).

Table 1. Models of PTSD

			Numbing	Dysphoria	Dysphoric Arousal	Anhedonia	Externalizing Behaviours	Hybrid
	DSM-IV	DSM-5	King, 1998 (Muenzenmaier et al., 2015)	Simms, 2002 (Shevlin et al., 2007)	Elhai, 2011 (Varese et al., 2012)	Liu, 2014 (Liu et al., 2014)	Tsai, 2015 (King et al., 1998)	Armour et al., 2015 (Armour et al., 2015)
	3-factor	4-factor	4-factor	4-factor	5-factor	6-factor	6-factor	7-factor
DSM 5 Factors
B1. Intrusive thoughts	R	R	R	R	R	In	R	R
B2. Nightmares	R	R	R	R	R	In	R	R
B3. Flashbacks	R	R	R	R	R	In	R	R
B4. Emotional cue reactivity	R	R	R	R	R	In	R	R
B5. Physiological cue reactivity	R	R	R	R	R	In	R	R
C1. Avoidance of thoughts	Av	Av	Av	Av	Av	Av	Av	Av
C2. Avoidance of reminders	Av	Av	Av	Av	Av	Av	Av	Av
D1. Trauma-related amnesia	Av	NAMC	NC	Dy	NC	NA	N	NA
D2. Negative beliefs	-	NAMC	NC	Dy	NC	NA	N	NA
D3. Distorted blame	-	NAMC	NC	Dy	NC	NA	N	NA
D4. Pervasive negative emotional state	Av	NAMC	NC	Dy	NC	NA	N	NA
D5. Lack of interest	Av	NAMC	NC	Dy	NC	An	N	An
D6. Feeling detached	Av	NAMC	NC	Dy	NC	An	N	An
D7. Inability to experience positive emotions	Av	NAMC	NC	Dy	NC	An	N	An
E1. Irritability/aggression	Hy	Hy	Hy	Dy	DA	DA	EB	EB
E2. Recklessness	-	Hy	Hy	Hy	DA	DA	EB	EB
E3. Hypervigilance	Hy	Hy	Hy	Hy	AA	AA	AA	AA
E4. Exaggerated startle	Hy	Hy	Hy	Hy	AA	AA	AA	AA
E5. Difficulty concentrating	Hy	Hy	Hy	Dy	DA	DA	DA	DA
E6. Sleep disturbance	Hy	Hy	Hy	Dy	DA	DA	DA	DA

R, Re-Experiencing; Av, avoidance; NAMC, negative alterations in mood and cognitions; NC, negative cognitions; Hy, hyperarousal; Dy, dysphoria; DA, dysphoric arousal; N, Emotional Numbing; In, intrusion; AA, anxious arousal; NA, negative affect; An, anhedonia; EB, externalizing behaviours.

One newer model of the factor structure of PTSD symptoms, include Elhai’s Dysphoric Arousal 5- factor model (Elhai et al., 2011; Wang, Elhai, Dai, & Yao, 2012; Wang, Long, Li, & Armour, 2011a; Wang et al., 2011b). Aside from re-experiencing and avoidance factors, this model also conceptualizes the three arousal symptoms (sleep disturbance, irritability, and difficulty concentrating) as a separate Dysphoric Arousal factor, which is distinct from the Anxious Arousal and Dysphoria factors. Studies have shown support for this model using the PCL-5 (Cheng et al., 2020; Contractor et al., 2018; Eddinger & McDevitt-Murphy, 2017; Lee et al., 2019; Liu et al., 2016, 2014; Tsai et al., 2015; Wang et al., 2017, 2012, 2011a, 2011b). Another recently proposed model is Liu’s 6-factor Anhedonia model consisting of intrusion, avoidance, negative affect, anhedonia, dysphoric arousal, and anxious arousal (Liu et al., 2014). This Liu’s 6-factor model has been supported by PCL-5 studies (Armour et al., 2015; Ashbaugh et al., 2016; Contractor et al., 2018; Liu et al., 2016; Van Praag et al., 2020; Wang et al., 2017). Tsai’s 6-factor Externalizing Behaviours model consisting of re-experiencing, avoidance, emotional numbing, externalizing behaviours, anxious arousal, and dysphoric arousal factors (Tsai et al., 2015) has also been supported by research on the PCL-5 (Armour et al., 2015). Lastly, a 7-factor hybrid model proposed by Armour et al. (2015) consisting of re-experiencing, avoidance, negative affect, anhedonia, externalizing behaviours, anxious and dysphoric arousal factors has been supported in some research on the PCL-5 (Ashbaugh et al., 2016; Cheng et al., 2020; Contractor et al., 2018; Lee et al., 2019; Liu et al., 2016; Van Praag et al., 2020; Wang et al., 2017).

Table 2 summarizes the results of studies examining the latent factor structure of PCL-5 using CFA in different clinical populations (Armour et al., 2015; Ashbaugh et al., 2016; Cheng et al., 2020; Contractor et al., 2018; Eddinger & McDevitt-Murphy, 2017; Krüger-Gottschalk et al., 2017; Lee et al., 2019; Liu et al., 2016, 2014; Tsai et al., 2015; Van Praag et al., 2020; Wang et al., 2017, 2011a), with 9,578 participants in total. As can be seen in Table 2, the average fit indices for Elhai’s 5-factor Dysphoric Arousal model were slightly better than King’s 4-factor Numbing, or Simms’ 4-factor Dysphoria model. Average CFI fit indices for Liu’s, Tsai’s and Armour’s models were all quite good. Even though all five of these models demonstrated adequate fit for the samples studied, the best fitting model appears to be Armour’s 7-factor hybrid model. As seen in Table 2, studies on the PCL-5 varied in findings across different populations. However, no previous CFA studies have evaluated the factor structure of PCL-5 in persons with SMI.

Table 2. Summary of CFA Studies on PCL-5

			DSM-5	Dysphoria	DysphoricArousal	Anhedonia	ExternalizingBehaviours	Hybrid
			4-factor	4-factor	5-factor	6-factor	6-factor	7-factor
			King, 1998	Simms, 2002	Elhai, 2011	Liu, 2014	Tsai, 2015	Armour, 2015
Study	Sample	N	CFI/RMSEA	CFI/RMSEA	CFI/RMSEA	CFI/RMSEA	CFI/RMSEA	CFI/RMSEA
Armour, 2015	Veterans	1484	0.93/0.04	0.93/0.04	0.94/0.04	0.96/0.03	0.94/0.04	0.96/0.03
(Armour et al., 2015)
Armour, 2015	University	497	0.97/0.09	0.96/0.09	0.97/0.08	0.99/0.06	0.98/0.08	0.99/0.06
(Armour et al., 2015)	Students
Tsai, 2015	Veterans	1484	0.92/0.04	-	0.93/0.04	-	0.94 /0.04	-
(King et al., 1998)
Eddinger, 2017	Veteran	129	0.88/0.12	0.92/0.11	0.89/0.12	-	-	-
(Eddinger & McDevitt-Murphy, 2017)
Eddinger, 2017	College Sample	737	0.91/0.09	0.95/0.08	0.92/0.08	-	-	-
(Eddinger & McDevitt-Murphy, 2017)
Krüger-G., 2017	Clinical Sample	352	0.89/0.09	0.89/0.09	-	-	-	-
(Krüger-Gottschalk et al., 2017)
Contractor, 2018	University	191	-	0.93/0.06	0.94/0.06	0.97/0.04	0.94/0.04	0.98/0.04
(Elhai et al., 2011)	Students
Lee, 2019	Veterans	380	0.95/0.05	0.95/0.05	0.96/0.05	-	0.96/0.05	0.97/0.04
(Lee et al., 2019)
Ashbaugh, 2016	Undergraduate	838	0.91/0.08	-	-	0.95/0.06	-	0.96/0.06
(Ashbaugh et al., 2016)	(English)
Ashbaugh, 2016^a,	Undergraduate	262	0.89/0.09	-	-	0.92/0.08	-	0.92/0.08
(Ashbaugh et al., 2016)	(French)
Liu, 2014^b,	Earthquake	1196	0.95/0.04	0.95/0.05	0.96/0.04	0.97/0.04	-	-
(Liu et al., 2014)	Survivors
Liu, 2016^b,	Trauma-Exposed	559	0.95/0.04	0.94/0.05	0.95/0.04	0.96/0.04	0.95/0.04	0.97/0.04
(Tsai et al., 2015)	Adolescents
Wang, 2017^b,	Trauma-Exposed	762	0.97/0.06	-	-	0.97/0.05	0.98/0.05	0.98/0.05
(Wang et al., 2011a)	Adolescents
Cheng, 2020^b,	Healthcare	212	0.80/0.11	0.77/0.12	0.83/0.10	0.92/0.07	0.88/0.09	0.96/0.05
(Cheng et al., 2020)	Workers
Van Praag, 2020^c,	Civilian TBI	495	1.00/0.03	-	-	1.00/0.00	-	1.00/0.00
(Van Praag et al., 2020)	Patients

^a = French version of PCL-5; ^b = Chinese version of PCL-5; ^c = Dutch version of PCL-5

The current study evaluated the psychometric properties of the PCL-5 in a sample of persons living with SMI. We first evaluated the factor structure of PCL-5. Based on our review, we hypothesized that Armour’s 7-factor hybrid model would have the best fit, while the other models (reviewed above) would have adequate fit. We then examined the convergent validity of the PCL-5 and its subscales.

2. Method

2.1. Participants

2.1.1. Study 1

In Study 1, 536 participants were drawn from the screening data for a larger randomized control trial (RCT) which compared a 12-week group cognitive behavioural treatment (CBT) for PTSD programme with treatment as usual (TAU) in 10 supported employment programmes in three Northeastern states, serving people with SMI (Lu, Waynor, Yanos, Parrott, & Gill, 2020) [SMI was defined as mental, behavioural, or emotional disorders that result in serious functional impairment, i.e. that affect an individual’s ability to perform major life activities such as working, maintaining social relationships, or taking care of oneself (the most common SMIs include, but are not limited to, schizophrenia, schizoaffective disorder, bipolar disorder, and major depression)] (Substance Abuse and Mental Health Services Administration (SAMHSA), 2017). The study sites were located in urban, suburban and rural communities. Additionally, the supported employment programmes were all part of larger community mental health agencies serving SMI clients, which provided an array of public mental health services including: supported housing, partial hospitalization, medication management, substance abuse counselling, peer support, assertive community treatment, and other case management programmes. Trauma history and PTSD screening were implemented at these sites. The study was approved by the university’s Institutional Review Board.

As can be seen in Table 3, participants were closely split by gender, and were most typically African American and in their late 40s. Diagnoses of SMI for this study were based on self-report, and only 194 (36.2%) participants reported their psychiatric diagnoses on the Eligibility Checklist. The most common self-reported diagnoses were depressive disorders, bipolar disorder, and schizophrenia or schizoaffective disorder. Only 6.2% of participants reported having a diagnosis of PTSD.

Table 3. Demographic/clinical characteristics

	Study 1 (N = 536)		Study 2 (N = 132)
	N	%	N	%
Gender
Male	288	53.7	51	38.6
Female	248	46.3	81	61.4
Race/ Ethnicity
African American	248	46.3	56	42.4
White (non-Hispanic)	187	34.9	58	43.9
Hispanic	38	7.1	8	6.1
Other	26	4.9	10	7.6
Missing	37	6.9	0.0	0.0
Primary Psychiatric Diagnosis
Schizophrenia/ Schizoaffective	92	27.5	28	21.2
Depressive Disorders	114	34.1	55	41.7
Bipolar Disorders	92	27.5	38	28.8
Other	36	10.8	11	8.3
Current Psychotropic Medication
Antipsychotic			66	50.0
Mood Stabilizer			30	22.7
Antidepressant			72	54.5
Anxiolytic/Sedative			44	33.3
No Medication			15	11.4
Disability Benefits			55	41.7
Medicare/Medicaid Insurance			81	81.6^1
Currently Working			36	27.9^1
	M	SD	M	SD
Age	47.23	12.91	45.97	11.94
Age at 1^st Hospitalization			25.43	13.27
Total # of Hospitalizations			8.50	13.59
Total Months Hospitalized			14.23	47.03
# of Months Worked in Past 5 Years			6.10	11.44
Earned Income Past Month			219.33	434.85
BAI			23.44	12.33
BDI-II			27.03	12.11
CAPS-5			37.21	10.25
PCL-5	36.44	21.31	48.84	14.76
PTCI			137.43	40.98
BPRS 24-Item.			47.30	8.53
BPRS 18-Item			35.11	6.66

BDI-II = Beck Depression Inventory-II; BAI = Beck Anxiety Inventory; BPRS = Brief Psychiatric Rating Scale, PANSS = Positive and Negative Syndrome Scale; CAPS = Clinician-Administered PTSD Scale; PTCI = Posttraumatic Cognitions Inventory; ¹ = valid percent. Self-reported diagnoses collected from 334 participants in study 1 upon IRB approval.

2.1.2. Study 2

The participants in Study 2 were a subset of those in Study 1, and included participants who met criteria for PTSD and were enrolled in the CBT for PTSD study. In Study 2, 132 participants completed the baseline interview consisting of a series of psychological measures (see Table 3). Participants were typically in their late 40s, mostly female, and were nearly evenly split between African-American and White racial groups.

Presence of SMI in Study 2 was established for all participants following criteria used by Russinova et al. (2018), which included self-report of a psychiatric diagnosis and receipt of Social Security disability benefits due to mental illness or a history of at least 1 psychiatric hospitalization. 94.4% of the sample met criteria for SMI. 91.7% of participants reported a lifetime history of at least one psychiatric hospitalization (M = 8, SD = 14), with 55.1% reporting at least three psychiatric hospitalizations, and 40.3% reporting at least one within 2 years prior to enrolling in the study. Additionally, 50.5% reported receiving disability benefits at the time of study.

Primary psychiatric diagnosis was obtained from self-report in Studies 1 and 2. However, in Study 2, where more information was collected, consistent with (Ellison et al., 2008; Russinova et al., 2018) we made an effort to validate based on either reported use of psychotropic medications or diagnosis-specific symptoms, in the following ways: 1) we confirmed a diagnosis of a schizophrenia spectrum disorder if at least one antipsychotic medication was reported being used; 2) We confirmed a bipolar diagnosis if a mood stabilizer was reported as being used, additionally if an individual self-reported a depressive disorder but reported using a mood stabilizer, we coded this as a bipolar diagnosis; 3) a depressive disorder diagnosis was confirmed if the individual reported using antidepressants; 4) For individuals who did not report using psychotropic medications, we confirmed their self reported diagnosis only if difficulties with diagnosis-specific symptoms were reported on relevant Brief Psychiatric Rating Scale-Expanded (BPRS) (Lukoff, Liberman, & Nuechterlein, 1986) items. For example, a schizophrenia spectrum diagnosis was confirmed if they scored positively on Thought Disturbance Subscale of BPRS, which included grandiosity, suspiciousness, hallucinations, and unusual thought content. A diagnosis of a mood disorder was confirmed if the person scored positively in Anergia subscale and/or Affect subscale. 5) We did not validate the diagnosis for individuals who self-reported other diagnostic categories. There were 117 out of 132 cases (88.6%) that had their diagnosis validated based on psychotropic medication and based on the endorsement of relevant BPRS (Lukoff et al., 1986) items. In terms of current psychotropic medications, 50.0% were on antipsychotics, 22.7% on mood stabilizers, 54.5% on antidepressants, 33.3% on anxiolytics/sedatives, and 11.4% were not on medications.

2.2. Procedure

In study 1, supported employment programme and study staff were trained to conduct PTSD screening and choose dates for PTSD screenings at their respective programmes. Supported employment staff then notified the programme clients of the opportunity to be screened for PTSD, and the dates the study staff would conduct the screenings at the agency. Agency staff posted flyers in the office and also made personal calls to clients informing them of the day of the screening (it should be noted that the invitation to participate was open to all clients at the recruitment sites). Individuals who were interested came to the supported employment programme on the day of the screening and met with study personnel who explained the screening process. If the individual agreed, study personnel and supported employment staff conducted a comprehensive screening of trauma exposure and PTSD symptoms. The following script was used to introduce the screening to clients: “It is very common for people to have experienced some very stressful and upsetting events. Even if these events happened a long time ago, they can still affect how a person thinks and feels, and how a person reacts to other people and situations many years later. People who have experienced a traumatic event, repeated traumatic events, or certain kinds of stress over a long period of time often have different mental health treatment needs than people who have not experienced trauma or chronic stress. Because of this, it can be helpful to you if your treatment providers are aware of your past experiences of trauma and chronic stress, and the way in which these may be still affecting you now. We would like you to try to answer the following questions. We want to see if any of these things, problems or complaints has happened to you. If you are not sure of an answer to a question, please make your best guess. If you have any questions, I would be happy to talk with you about them.” Participants consented to being screened and to providing screening data, and were paid $10 for completing the screening.

Upon the completion of the trauma screening in Study 1, if participants scored positive on PTSD Checklist (PCL-5) (Weathers et al., 2013a) (PCL-5 ≥ 30), they were invited for possible participation in the study on CBT for PTSD. For Study 2, inclusion criteria were the following: 1) age ≥ 18; 2) currently receiving supported employment services within the past 24 months; 3) history of treatment for mental illness; 4) current diagnosis of PTSD as determined by Clinician Administered PTSD Scale for DSM-5 (CAPS-5 (Eddinger & McDevitt-Murphy, 2017; Weathers et al., 2013b)) no current diagnosis of alcohol or drug dependence as described in chart; 6) no hospitalization or suicide attempt in the past 2 months; and 7) willingness to provide informed consent to participate in the study. Potentially eligible and interested clients were contacted by a team member, who described the study and obtained informed consent. Once consent was obtained, the completion of a baseline interview confirmed the eligibility of participation. Participants were paid $30 for the completion of the baseline interview. Participants were then randomized into treatment as usual or treatment condition. All clients were followed up on a monthly basis and provided their PCL-5 and BDI data in addition to their employment status. Participants were paid $10 for these monthly interviews. A subset of participants (n = 36) in the treatment as usual condition whose PCL-5 was administered one month apart was used to calculate their test-retest reliability.

2.2.1. Measures study 1

2.2.1.1. Traumatic life events questionnaire

In Study 1, an abbreviated 16-item version of the Traumatic Life Events Questionnaire (TLEQ) (Kubany et al., 2000) was used to screen lifetime trauma history for all participants. For each event on the scale, the participant indicated whether they had ever experienced it over their lifetime in a binary (yes/no) format. The TLEQ asks about the experience of traumatic events using wording that corresponds with the DSM-IV criterion A for PTSD. This version of the TLEQ was used to screen for trauma exposure in previous studies with persons with SMI (Mueser et al., 2008).

2.2.1.2. PTSD checklist-5

The PTSD Checklist (PCL-5) (Weathers et al., 2013a) is a 20-item self-report measure that assesses the 20 DSM-5 symptoms of PTSD. This assessment can be used to screen individuals for PTSD and to make a provisional PTSD diagnosis. A total symptom severity score (range: 0–80) can be obtained by summing the scores for each of the 20 items. A provisional PTSD diagnosis can be made by treating each item rated as 2 = ‘Moderately’ or higher as a symptom endorsed, then following the DSM-5 diagnostic rule which requires at least: 1 B item (intrusion questions 1–5), 1 C item (avoidance questions 6–7), 2 D items (negative cognitions/affect questions 8–14), 2 E items (hyperarousal questions 15–20). Preliminary work suggests that a PCL-5 cut-off of 33 indicates probable PTSD. Strong convergent validity has been found with other clinician administered measures of PTSD (Wortmann et al., 2016). The Cronbach’s $\alpha$ of PCL-5 in Study 1 was 0.96.

2.2.2. Study 2

In Study 2, 132 participants scored at 33 or above on PCL-5 at screening, met criteria for PTSD as determined by CAPS-5 as well as meeting other aforementioned eligibility criteria for the CBT for PTSD study, and completed the following tests at the baseline interview.

2.2.2.1. Clinician administered PTSD scale for DSM-5

The Clinician Administered PTSD Scale for DSM-5 (CAPS-5) (Weathers et al., 2013b) is a 30-item structured interview which a clinician interviews a client with exposure to at least one traumatic event and assess for PTSD symptom severity over the previous 30 days. Scoring of the CAPS-5 involves the clinician rating both frequency and intensity to determine a client’s severity score for a particular item, ranging from 0 to 4 (absent, mild/subthreshold, moderate/threshold, severe/markedly elevated, and extreme/incapacitating). The CAPS-5 total symptom severity score is then calculated by adding the severity scores for the PTSD symptom items in the assessment. The CAPS-5 also demonstrated good test–retest reliability (α = 0.78), and strong interrater reliability (α = 0.91) and convergent validity of r = 0.83 (Weathers et al., 2018). Regular reliability checks were conducted using audio-taped interviews among three trained research assistants who conducted the CAPS-5 interviews with excellent agreement among raters achieved.

2.2.2.2. Posttraumatic cognitions inventory

Trauma-related cognitions were evaluated with the Posttraumatic Cognitions Inventory (PTCI) (Foa, Cashman, Jaycox, & Perry, 1997), a self-report measure pertaining to common negative thoughts and beliefs about self, other people, and the world. The PTCI consists of 36 items ranging from 1 (totally disagree) to 7 (totally agree). It has good test–retest reliability and has been shown to be particularly effective at discriminating between traumatized individuals with PTSD and those without (Foa et al., 1997). Both subscale scores and total scores are based on the original 33 items (Foa et al., 1997). Subscale scores are determined by summing each item in the subscale to calculate a raw subscale score and then dividing by the number of items in the subscale, which results in a mean subscale score. The PTCI total score is the sum of the three subscales. In the current investigation, Cronbach’s alpha for the 36-item PTCI total score was .96 in this study.

2.2.2.3. Brief psychiatric rating scale-expanded

The Brief Psychiatric Rating Scale-Expanded (BPRS) (Lukoff et al., 1986) is a widely used 24-item instrument for measuring severity of psychiatric symptoms, with excellent psychometric properties and an established factor structure in the SMI population (Mueser, Curran, & McHugo, 1997). The Expanded version of the measure includes 24 items, all rated on a 7-point Likert scale, with 1 = not present, 2 = very mild, 3 = mild, 4 = moderate, 5 = moderately severe, 6 = severe, 7 = extremely severe. A total symptom severity score (range 24–168) can be obtained by summing the scores for each of the 24 items. A BPRS score of 31, based on the original 18 items of BPRS (Leucht et al., 2005), is considered as ‘mildly ill,’ a score of 41 is ‘moderately ill’, and 53 is ‘markedly ill’ (Leucht et al., 2005). Regular reliability checks were conducted using audio-taped interviews among trained research assistants used the BPRS with excellent agreement between raters achieved.

2.2.2.4. Beck depression inventory

Beck Depression Inventory (BDI-II) (Beck, Steer, & Brown, 1996) was used to measure depression changes. It contains 21 items, each rated on a 4-point Likert scale (0–3) of increasing severity. A total score is obtained by summing the scores for each of the 21 items. Scores of 1 to 10 are considered in the normal range, scores of 11 to 16 are considered ‘mild’ depression, scores of 17 to 30 are considered ‘moderate’ depression, and scores of 31 and higher are considered ‘severe’ depression (Trent & Weiss, 2000). The BDI has high validity in differentiating between depressed and non-depressed individuals and good internal consistency (ranging from .73 to .92 with a mean of .86) (Beck, Steer, & Garbin, 1988; Richter, Werner, Heerlein, Kraus, & Sauer, 1998).

2.2.2.5. Beck anxiety inventory

Beck Anxiety Inventory (BAI) (Beck & Steer, 1993) is a 21-item self-report scale for anxiety. It consists of descriptive statements of anxiety symptoms rated on a 4-point scale (0–3) of increasing severity. Each of the 21 items are summed to obtain a total score. Total scores of 0 to 7 reflect ‘Minimal level of anxiety’; scores of 8 to 15 indicate ‘Mild anxiety’; scores of 16 to 25 reflect ‘Moderate anxiety’; and scores of 26 to 63 indicate ‘Severe anxiety’. The BAI possesses high reliability and validity (Beck et al., 1996).

2.3. Data analysis

Data was entered and cleaned using SPSS 26. For PCL-5, only two participants had completed less than half of the PCL-5 items, and 473 (88.6%) had complete data on the PCL-5.

We first used CFA to evaluate the degree to which the screening sample fit the six models using the Study 1 sample because this dataset included the broad range of PTSD symptoms and was not pre-selected for probable PTSD. Missing data for PCL-5 was handled by listwise deletion in CFA analysis. Those models with the best fit were subsequently evaluated for convergent and divergent validity using the second dataset from Study 2. Correlational analyses were conducted using SPSS 26 to establish convergent and divergent validity. For correlation analysis, to calculate PCL scale scores, missing data was handled by mean imputation.

CFAs were conducted with Amos 26.0. Model of fit was evaluated using several indices, including the model χ2 test, the Tucker-Lewis index (TLI), the comparative fit index (CFI), Akaike Information Criterion (AIC), and the root mean square error of approximation (RMSEA). Indices used in model fit evaluation included the root-mean square error of approximation (RMSEA; values of .06 or less indicate excellent fit), the comparative fit index (CFI; values of .95 or greater indicate excellent fit), and the Tuck Lewis index (TLI; values of .95 or greater indicate excellent fit (Hu & Bentler, 1998, 1999)). The model χ2 test compares the proposed factor structure to the null model with significant p values indicating inadequate model fit. However, the model χ2 test is strongly influenced by sample size, and because of the relatively large sample size of Study 1 we deemphasized the importance of this index relative to the other four indices in evaluating the adequacy of model fit. We considered good indicators of fit to be TLI and CFI > .95, and RMSEA < .06 (Brown, 2006; Hu & Bentler, 1995). Akaike Information Criterion (AIC) (Schwarz, 1978) is used when comparing non-nested competing models, with lower values suggesting better fit. Superior model fit was indicated by an AIC score difference of 10 or greater.

3. Results

Study 1 included 536 participants who were screened for PTSD. The vast majority of participants (92.4%) had been exposed to at least one traumatic event, with large percentage of participants experiencing multiple types of trauma, including physical and sexual assault. Frequently reported types of traumatic events included having one’s life threatened (53.2%), witnessing domestic violence during childhood (49.6%), domestic violence (47.1%), assault by stranger (41.5%), childhood physical abuse (34.3%), childhood sexual abuse by adult (35.1%), childhood sexual abuse by peer (28.6%), and adult sexual abuse (26.1%). Participants reported experiencing an average of 5.87 different types of traumatic life events. Participants in reported an average PCL-5 sum score of 36.44 (SD = 21.31). About 57.2% of the sample met or exceeded a cut-off score of 33 for a provisional PTSD diagnosis.

3.1. Confirmatory factor analysis

Confirmatory Factor Analysis was first used to evaluate the degree to which the Study 1 sample fit the six models proposed and supported by the previous literature. The goodness of fit indices for each of the models, including the King’s (DSM-5) Numbing 4-factor model, Simms’ Dysphoria 4-factor model, Elhai’s Dysphoric Arousal 5-factor model, Liu’s Anhedonia 6-factor model, Tsai’s Externalizing Behaviours 6-factor model and Armour’s (2015), and Hybrid 7-factor model are presented in Table 4. Of the six models evaluated, all models had excellent fit (Table 4; CFIs ≥ 0.95, TLIs > 0.93, RMSEA ranged from 0.04 to 0.05). However, smaller AIC indicated better model fit in non-nested models, so the best fitting model was the Armour’s (2015) Hybrid 7-factor model with the lowest AIC value, RMSEA, and highest CFI and TLI (AIC = 472.79, CFI = 0.98, TLI = 0.97, RMSEA = 0.05).

Table 4. Fit indices for six models of PTSD among the PTSD screening sample (Study 1; N = 536)

Model	χ2	df	χ2/DF	AIC	TLI	CFI	RMSEA	RMSEA 90% CI
King’s (1998) Numbing 4-factor model	513.65	164	3.13	645.65	0.94	0.95	0.06	0.06–0.07
Simms’ (2002) Dysphoria 4-factor model	548.34	164	3.34	680.34	0.93	0.95	0.07	0.06–0.07
Elhai.’s (2011) Dysphoric Arousal 5-factor model	486.20	160	3.04	626.20	0.94	0.96	0.06	0.06–0.07
Liu’s (2014) Anhedonia 6-factor model	370.32	155	2.39	520.32	0.96	0.97	0.05	0.04–0.06
Tsai’s (2015) Externalizing Behaviours 6-factor model	434.42	155	2.80	584.42	0.95	0.96	0.06	0.05–0.07
Armour’s (2015) hybrid 7-factor model	310.79	149	2.09	472.79	0.97	0.98	0.05	0.04–0.05

Reliability was calculated using Study 1 data (N = 536). With the DSM-5 4-Factor Model the internal consistency α = .96 for the total scale and α = .83–.91 for the subscales. Inter-item correlations were computed as another measure for internal consistency and ranged from .28 to .73, which can be regarded as acceptable (Clark & Watson, 1995) (M = .53; re-experiencing items: .53–.58, avoidance items: .53–.56, negative alterations in cognitions and mood items: .34–.60, and alterations in arousal and reactivity items: .44–.56).

With regards to the 7-factor model, the internal consistency for the PCL-5 total score was high, with α = .96 for the total scale and α = .75–.91 for the subscales in the 7-factor model (see Table 5). Inter-correlations among all factors ranged from 0.63 to 0.90. Inter-item correlations for this model were also acceptable and were computed as another measure for internal consistency, ranging from .21 to .73 (Clark & Watson, 1995) (re-experiencing items: .53–.58, avoidance items: .53–.56; negative Affect items .34–.60, anhedonia items: .52–.56; externalizing behaviours items: .44–.49; anxious arousal items .49–.53, and dysphoric arousal items .48–.54.).

Table 5. Factor pattern matrix and inter-factor correlation of Armour’s 7-factor hybrid model among the PTSD screening sample (N = 536)

	Current Sample (N = 536)
	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6	Factor 7
Armour’s 7-factor Hybrid Model	RE	AV	NA	AN	EB	AA	DA
B1.	0.84
B2.	0.78
B3.	0.81
B4.	0.85
B5.	0.80
C1.		0.83
C2.		0.87
D1.			0.54
D2.			0.81
D3.			0.76
D4.			0.87
D5.				0.78
D6.				0.87
D7.				0.81
E1.					0.81
E2.					0.73
E3.						0.76
E4.						0.83
E5.							0.78
E6.							0.69
Intercorrelation
Factor 1	1	0.85	0.85	0.72	0.71	0.79	0.85
Factor 2		1	0.86	0.72	0.63	0.75	0.77
Factor 3			1	0.89	0.77	0.85	0.90
Factor 4				1	0.76	0.77	0.88
Factor 5					1	0.77	0.80
Factor 6						1	0.87
Factor 7							1
Cronbach’s Alpha
	0.91	0.83	0.83	0.86	0.75	0.77	0.70

RE, Re-Experiencing; AV, Avoidance; NA, Negative Affect; AN, Anhedonia; AA, Anxious Arousal; EB Externalizing Behaviours; AA, Anxious Arousal; DA, Dysphoric Arousal.

3.2. Test–retest reliability

Test–retest reliability was calculated using 36 participants who did not receive PTSD intervention and whose PCL-5 was administered 1 month apart. The test–retest reliabilities of the PCL symptom subscales in DSM-5 4-factor model had the following intraclass correlation coefficients (ICCs) ranging between .43 and .66: intrusion: .74 (95% CI = .48–.87, P < .001), avoidance: .38 (95% CI = −.23–.68, P = .09), negative alterations in mood and cognition: .63 (95% CI = .27–.81, P = .002), and hyperarousal: .75 (95% CI = .52–.86, P < .001). The test–retest reliability of the total PCL scale was significant, with an ICC of 0.73 (95% CI = 0.48–0.86, P < .001). The test–retest reliabilities of PCL subscales in the 7-factor model had the following ICCs: re-experiencing: .74 (95% CI = .48–.87, P < .001), avoidance: .38 (95% CI = −.23–.68, P = .085), negative affect: .68 (95% CI = .36–.83, P < .001), anhedonia: .48 (95% CI = −.18–.74, P = .028), externalizing behaviours: .70 (95% CI = .41–.85, P < .001), anxious arousal: .47 (95% CI = −.45–.73, P = .033), dysphoric arousal: .81 (95% CI = .63–.90, P < .001). Test–retest reliability was found for PCL-5 total scale and most of its subscales in both models, with the exception of the avoidance subscale.

3.3. Convergent validity

To evaluate convergent validity, PCL total score and its factor scores were correlated with depression (BDI-II), anxiety (BAI), trauma-related cognitions (PTCI), and PTSD symptoms (CAPS-5) using Study 2 data (Table 6). As hypothesized, the factor scores in DSM-5 4-factor model and Armour’s Hybrid 7-factor model were significantly and positively correlated with depression (BDI-II), anxiety (BAI), trauma-related cognitions (PTCI), and PTSD symptoms (CAPS-5).

Table 6. Divergent and convergent validity of PCL-5 among participants meeting criteria for PTSD based on CAPS-5 (N = 132)

PCL Factor	CAPS-5	PTCI	BDI-II	BAI	BPRS	BPRS Thought Disturb.	BPRS Anergia	BPRS Affect	BPRS Disorg.
DSM-5 4-factor Model
Intrusion	0.58***	0.43***	0.34***	0.33***	0.14	0.08	−0.02	0.13	−0.04
Avoidance	0.36***	0.31***	0.26**	0.21*	0.02	0.03	0.00	0.09	−0.12
Negative Alterations in mood/ cognition	0.51***	0.59***	0.58***	0.39***	0.23**	0.14	−0.01	.26**	−.18*
Hyperarousal	0.54***	0.50***	0.45***	0.51***	0.29**	0.09	0.11	0.15	−0.09
Armour’s 7-factor Hybrid Model
Re-Experiencing	0.58***	0.43***	0.34***	0.33***	0.14	0.08	−0.02	0.13	−0.04
Avoidance	0.36***	0.31***	0.26**	0.21*	0.02	0.03	0.00	0.09	−0.12
Negative Affect	0.50***	0.52***	0.46***	0.38***	0.20*	.19*	−0.01	.26**	−.19*
Anhedonia	0.41***	0.51***	0.59***	0.32***	0.23*	0.06	−0.01	.21*	−0.12
Anxious Arousal	0.44***	0.38***	0.34***	0.46***	0.29**	.18*	0.13	011	0.01
Dysphoric Arousal	0.49***	0.38***	0.43***	0.41***	0.19*	0.11	0.13	.21*	−0.11
Externalizing Behaviours	0.24**	0.36***	0.23**	0.25**	0.15	−0.05	0.02	0.03	−0.13

CAPS-5 = Clinician Administered PTSD Scale for DSM-5; PTCI = Posttraumatic Cognitions Inventory; BDI-II = Beck Depression Inventory-II; BAI = Beck Anxiety Inventory; BPRS = Brief Psychiatric Rating Scale; *p ≤ .05, **p ≤ .01, ***p ≤ .001

3.4. Divergent validity

Divergent validity was established by correlating PCL total score and its factor scores with BPRS subscales: Thought Disturbance, Anergia, Affect, and Disorganization (Lukoff et al., 1986), using Study 2 data (Table 6). As hypothesized, the factor scores in both DSM-5 4-factor model and Armour’s Hybrid 7-factor model were primarily not correlated with the subscales of BPRS, with the subscale of Thought Disturbance, which measured grandiosity, suspiciousness, hallucinations, and unusual thought content; Anergia, which measured blunted affect, emotional withdrawal, motor retardation, and uncooperativeness; Affect, which measured somatic concern, anxiety, depression, guilt, and hostility; and Disorganization, which measured conceptual disorganization, tension, suicidality, and mannerisms and posturing (most ps >0.05).

4. Discussion

This study is the first to examine the psychometric properties of the PCL-5 in a sample of people in employment diagnosed with SMI. Our study used CFA to assess fit of different models proposed by previous literature, and, as hypothesized, found that Armour’s hybrid 7-factor model showed the best fit. Nevertheless, other models studied (Elhai’s 5-factor Dysphoric Arousal model, Liu’s 6-factor Anhedonia model, King’s 4-factor Numbing model, Tsai’s 6-factor Externalizing Behaviour model and Simm’s 4-factor Dysphoria model) were also found to have adequate fit by the commonly accepted standard of a RMSEA value of .08 and a CFI of .90 (Bentler, 1990; Browne & Cudeck, 1992; MacCallum, Browne, & Sugawara, 1996).

Some concerns with supporting a 7-factor model should be noted. Experts have noted that including many factors can be problematic when each factor only has about two items, since the composite score may not be as reliable (Kline, 2015). Even though specific factors in the 7-factor model have been linked to suicidality (Chou, Ito, & Horikoshi, 2020), the superiority of the 7-factor model over the DSM-5 model has not been found (Blevins, Weathers, Davis, Witte, & Domino, 2015). While our study offers some support for the 7-factor model, support is also found for DSM-5 4-factor model in this sample and contributes to its credibility for continued use.

When examining its reliability and validity, the PCL-5 proved to be psychometrically sound in this special population, with its excellent internal consistency, and convergent/divergent validity for both the DSM-5 model and the Armour’s hybrid 7-factor model. Test–retest reliability was found for both models, with the exception of the avoidance subscale. This study supports the integrity of using the PCL-5 among a population of individuals with SMI. Further, the examination of validity data suggests that similar correlational relations of subscales in these two models with other psychopathology measures suggest that the DSM-5 4-factor model warrants continued use.

Some of the limitations of this study include the use of data from persons with SMI receiving vocational services at community mental health agencies. A 2014 survey of mental health facilities serving clients with SMI reported that vocational services were offered at 20% of community mental health facilities in the US (Sherman, Lynch, Teich, & Hudock, 2017). The findings, therefore, may not be generalizable to the broader SMI population such as, such as in private sector long-stay psychiatric hospitals or not in treatment. A further limitation of the study concerns the use of self-report for psychiatric diagnosis. Even though validation steps were taken to ensure accuracy of the data and diagnosis, there could be misrepresentations of diagnoses or discrepancies of clinical diagnosis.

As suggested by previous reviews (Armour, Müllerová, & Elhai, 2016), implications of this study include informing diagnostic algorithms of PTSD, assessment of persons with PTSD symptoms, and intervention development. Our findings may have implications for assessment of PTSD among persons with comorbid diagnoses and severe functional impairment. Implications of this study include informing diagnostic algorithms of PTSD in SMI clients. The presence of both PTSD and SMI may lead to worse clinical and functional outcomes, such as substance use, suicide ideation, more severe delusions, increased psychosis, and lower quality of life than either disorder alone (Grubaugh et al., 2021). Untreated PTSD can also lead to worsening of the primary symptoms of SMI including severity of delusions and positive symptoms of psychosis (Seow et al., 2016). Unfortunately, clients with SMI frequently do not receive evidence-based treatments for PTSD (Grubaugh et al., 2021). Assessment of PTSD using PCL-5 may lead to improved detection of PTSD among SMI populations, thereby facilitating PTSD treatment.

Ethics statement

This study was approved by Rutgers University Newark Health Sciences Institutional Review Board (Study ID: Pro20140000913). Informed consent was required to participate in the study.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The funding agency NIDILRR did not require grantees to share data prior to 2018, so participants of this study did not agree for their data to be shared publicly. Supporting data is not available.

Additional information

Funding

This research was supported by the NIDILRR Field Initiated Grant # HHS-Administration for Community Living number [90IF0074].