Psychometric properties of the PTSD Checklist for DSM-5 in a sample of trauma exposed mental health service users

ABSTRACT

Background: PTSD self-report measures are frequently used in mental health services but very few have been evaluated in clinical samples that include civilians. The PCL-5 was developed to assess for DSM-5 PTSD.

Objective: The aim of this study was to evaluate the psychometric properties of the PCL-5 in a sample of trauma-exposed mental health service users who were evidencing symptoms of PTSD.

Method: Reliability and validity of the PCL-5 were investigated in a sample of 273 participants who reported past diagnosis for PTSD or who had screened positively for traumatic stress symptoms. Diagnostic utility was evaluated in comparison to the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5).

Results: The PCL-5 demonstrated high internal consistency, good convergent and divergent validity, acceptable stability and good diagnostic utility. However, operating characteristics differed from those found in other samples. Scores of 43–44 provided optimal efficiency for diagnosing PTSD. A post hoc regression analysis showed that depression explained more of the variance in PCL-5 total score than the CAPS-5.

Conclusion: Whilst the PCL-5 is psychometrically sound it appears to have difficulty differentiating self-reported depression and anxiety symptoms from PTSD in trauma-exposed mental health service users and clinicians should take care to assess full symptomatology when individuals screen positively on the PCL-5. Clinicians and researchers should also take care not to assume that operating characteristics of self-report PTSD measures are valid for mental health service users, when these have been established in other populations.

HIGHLIGHTS

• The PCL-5 demonstrated good psychometric properties in this sample of trauma-exposed mental health service users.

• An optimal cut off score of 43-44 was higher than reported in other studies evaluating the PCL-5.

• A post-hoc analysis suggested that the PCL-5 has difficulty differentiating self-reported depression and anxiety symptoms from PTSD.

• Clinicians should undertake further assessment to establish differential diagnosis when PCL-5 scores are raised.

 

Antecedentes: Las mediciones auto reportadas para el trastorno de estrés postraumático (TEPT) se emplean con frecuencia en los servicios de salud mental pero muy pocas han sido evaluadas en muestras clínicas que incluyan a civiles. Se desarrolló la lista de chequeo de los síntomas del trastorno de estrés postraumático (PCL-5, por sus siglas en inglés) para la evaluación de los síntomas del TEPT según el DSM-5.

Objetivo: El objetivo del estudio fue evaluar las propiedades psicométricas de la PCL-5 en una muestra de usuarios de servicios de salud mental expuestos a trauma y que mostraban síntomas del TEPT.

Métodos: Se investigaron la confiabilidad y la validez de la PCL-5 en una muestra de 273 participantes que reportaron un diagnóstico previo de TEPT o que fueron tamizados como positivos para síntomas de estrés traumático. La utilidad diagnóstica se evaluó mediante la comparación con la escala para el TEPT administrada por un clínico según el DSM-5 (CAPS-5, por sus siglas en inglés).

Resultados: La PCL-5 mostró alta consistencia interna, buena convergencia y validez divergente, estabilidad aceptable y buena utilidad diagnóstica. Sin embargo, las características operativas eran distintas de aquellas encontradas en otras muestras. Un puntaje entre 43 y 44 puntos tenía una eficiencia óptima para el diagnóstico del TEPT. Un análisis post hoc mostró que la depresión explicaba una mayor proporción de la varianza del puntaje total de la PCL-5 en comparación con la CAPS-5.

Conclusión: Aunque la PCL-5 es psicométricamente sólida, impresiona presentar dificultad para discriminar los síntomas auto reportados de depresión y ansiedad con los síntomas del TEPT en usuarios expuestos a trauma en servicios de salud mental. En las personas que son tamizadas como positivas con la PCL-5, los clínicos deberían evaluar la totalidad los síntomas para el TEPT con atención. Los clínicos y los investigadores también deberían estar atentos a no asumir que las características operativas de las mediciones auto reportadas para el TEPT son válidas para usuarios de servicios de salud mental cuando estas han sido desarrolladas en otras poblaciones.

 

背景:PTSD自我报告测量在心理健康服务中频繁使用, 但很少在包括平民的临床样本中进行评估。开发了PCL-5以评估DSM-5 PTSD。

目的:本研究旨在评估有创伤暴露和PTSD症状的心理健康服务使用者样本中PCL-5的心理测量学特性。

方法:在273名报告了过往PTSD诊断或创伤应激症状筛查阳性者的参与者样本中, 考查了PCL-5的可靠性和有效性。诊断效能通过与DSM-5临床用PTSD量表 (CAPS-5) 相比进行评估。

结果:PCL-5具有很高的内部一致性, 良好的收敛和发散效度, 可接受的稳定性和良好的诊断效能。但是, 操作特性不同于其他样本中的发现。43-44的得分具有PTSD最佳诊断效能。事后回归分析表明, 相较于CAPS-5, 抑郁解释了更多PCL-5总分的变异。

结论:虽然PCL-5从心理测量上讲是可靠的, 但在有创伤暴露的心理健康服务使用者中似乎很难对自我报告的抑郁和焦虑与PTSD进行区分, 当其PCL-5筛查为阳性筛查时, 临床医生应注意评估其全面症状。临床医生和研究人员也应注意不要认为自我报告的PTSD测量特性对精神卫生服务使用者有效, 尽管这些特性已在其他人群中验证。

KEYWORDS:

• Post-traumatic stress disorder
• psychometric assessment
• PCL-5
• psychiatric
• self-report measures

PALABRAS CLAVE:

• Trastorno de estrés postraumático
• evaluación psicométrica
• PCL-5
• psiquiátrico
• mediciones auto reportadas

关键词:

• 创伤后应激障碍,心理测评,PCL-5,精神病学,自我报告测量。

1. Introduction

Post-Traumatic Stress Disorder (PTSD) is a common and well-recognised psychiatric disorder which can develop as a result of exposure to highly threatening or catastrophic events. The condition is characterised by four clusters of symptoms: recurrent involuntary intrusive memories, avoidance, negative alterations in cognitions and mood and alterations in arousal and reactivity (American Psychiatric Association, 2013). PTSD is frequently comorbid with other conditions such as depression, anxiety disorders and substance misuse (Roberts, Back, Mueser, & Murray, 2020). Very few studies have evaluated PTSD self-report measures psychometrically in psychiatric samples outside of specialist military and veterans’ services (Brewin, 2005; Grubaugh, Elhai, Cusack, Wells, & Frueh, 2007). However, these measures are frequently used in mental health settings for screening purposes, to support clinical decision-making and to evaluate therapeutic progress in mental health services (e.g. IAPT, 2011; NHS Commissioning Board, 2013). The PTSD Checklist (PCL) (Weathers, Litz, Herman, Huska, & Keane, 1993) was one of the mostly widely used screening measures for PTSD (Elhai, Gray, Kashdan, & Franklin, 2005; McDonald & Calhoun, 2010). As a result of the publication of the Diagnostic and Statistical Manual of Mental Disorders-version 5 (DSM-5) the PCL was updated to the PTSD Checklist for DSM-5 (PCL-5) (Weathers et al., 2013a). The PCL-5 is one of few PTSD self-report instruments that have been updated to reflect DSM-5 changes to PTSD diagnostic criteria.

The psychometric properties of the PCL-5 have been evaluated across a number of populations, including: serving military and veterans samples (Blevins, Weathers, Davis, Witte, & Domino, 2015; Bovin et al., 2016; Hoge, Riviere, Wilk, Herrell, & Weathers, 2014; Konecky, Meyer, Kimbrel, & Morrisette, 2015; Murphy, Ross, Ashwick, Armour, & Busuttil, 2017; Wortmann et al., 2016), mortuary workers (Makhubela, 2018), trauma exposed treatment seekers (Krüger-Gottschalk et al., 2017), firefighters (Carvalho, da Motta, & Pinto-Gouveia, 2020), parents of children with burns injuries (Sveen, Bondjers, & Willebrand, 2016), refugees (Ibrahim, Ertl, Catani, Ismail, & Neuner, 2018), migrant workers (Hall et al., 2019), a primary care sample with a high HIV prevalence (Verhey, Chibanda, Gibson, Brakarsh, & Seedat, 2018), as well as student samples (Ashbaugh, Houle-Johnson, Herbert, El-Hage, & Brunet, 2016; Blevins et al., 2015). The scale has been shown to have satisfactory reliability and validity in all of these studies and it has also been found to perform with equivalence to the Post-Traumatic Checklist-specific version (PCL-S) for DSM-IV in a US infantry cohort (Hoge et al., 2014) and a college student cohort (Blevins et al., 2015). To our knowledge there has only been one evaluation of the psychometric properties of the PCL-5 for trauma-exposed general mental health service users (Pereira-Lima, Loureiro, Bolsoni, Apolinario da Silva, & Osório, 2019), in a small study conducted in a Brazilian psychiatric outpatient clinic. This study reported that the PCL-5 showed good internal consistency and test–retest reliability.

The PCL-5 has been compared against a structured clinical interview as a reference standard in eight recent studies, in a number of different populations (Bovin et al., 2015; Hall et al., 2019, Ho, Schlenger, Kulka & Marmar, 2017; Krüger-Gottschalk et al., 2017; Murphy et al., 2017; Pereira-Lima et al., 2019; Verhey et al., 2018; Wortmann et al., 2016). Table 1 provides a summary of study features. Whilst the majority of these studies report optimal cut-off scores between 31 and 37, scores ranged from 25 to 42. It is well established that the operating characteristics of self-report measures vary across populations and settings (McDonald & Calhoun, 2010). Given that self-report measures such as the PCL-5 are frequently used for screening and assessment purposes in mental health settings it is important to establish psychometric performance in such settings and to try to identify criterion validity. As noted by Hall et al. (2019) and Murphy et al. (2017), the establishment of such cut-offs are critical to the accurate estimation of PTSD prevalence in specific populations.

Table 1. Studies of the PCL-5 diagnostic accuracy by country, population sample and reference standard

Study	Country	Population sample	N	Internal consistency PCL-5 total Cronbach’s alpha	Test-retest reliability	Summary of convergent and discriminant validity findings	Reference standard	Optimal cut-off
Bovin et al, 2015	USA	Treatment seeking military veterans	140	.96	.84 at 22–48 days	Associated correlations were consistent with predicted observations for a wide battery of measures.	CAPS-5	31–33
Hall et al., 2019	Macao	Filipino migrant workers	131	.95	.58 at 10 days	The PCL-5 correlated moderately with the scores measures of depression and generalised anxiety and more weakly with measures of pain and social support	PTSD Module Neuro-psychiatric Interview (MINI) for DSM-5	25
Ho et al, 2016	USA	Military veterans	390	.97	Not reported	Convergent and discriminant validity were not investigated	CAPS-5	37
Krüger-Gottschalk et al., 2017	Germany	A trauma exposed treatment seeking sample	352	.95	.91 at 3 weeks	Correlation with the CAPS-5 was.77. Investigation of discriminant validity was not reported	CAPS-5	33
Murphy et al., 2017	UK	Treatment seeking military veterans	242	Not reported	Not reported	Convergent and discriminant validity were not investigated	CAPS-5	34
Pereira-Lima et al., 2019	Brazil	Psychiatric outpatients	85	.96	.87 at 10–30 days	Convergent and discriminant validity were not investigated	SCID interview for DSM-5	36
Verhey et al., 2018	Zimbabwe	Primary care sample with high rates of HIV	204	.92	Not reported	Convergent and discriminant validity were not investigated	CAPS-5	33
Wortmann et al., 2016*	USA	Treatment seeking military personnel	912	.91	Not reported	Observed correlations were consistent with predicted observations for a wide battery of measures.	PSS-I	42

*This study reported cut off scores for a number of different scoring methods for the PSS-I. This cut-off was based on a “stringent scoring rule”, which conformed to DSM-5 criteria.

1.1. Aims of the study

The aim of this study was to undertake a psychometric evaluation of the PCL-5 in a UK sample of trauma-exposed mental health service users. We aimed to investigate internal and test–retest reliability, convergent and discriminant validity and to compare it against the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) as the reference standard in order to establish diagnostic accuracy and operating characteristics. Based on previous evaluations (e.g. Blevins et al., 2015; Bovin et al, 2015; Hall et al., 2019; Pereira-Lima et al., 2019), we hypothesised that the PCL-5 would show high levels of internal consistency and acceptable to good test–retest reliability. From previous studies (e.g. Bovin et al, 2015; Hall et al., 2019), we anticipated that the PCL-5 would correlate strongly with measures of depression, panic, and generalised anxiety disorder (GAD) and a weaker association with measures of somatisation, alcohol abuse, interpersonal functioning and general functioning (Blevins et al., 2015; Bovin et al, 2015; Hall et al., 2019; Krüger-Gottschalk et al., 2017; Wortmann). Previous studies have also shown a strong association between negative cognitions of self and PTSD, a moderate association with negative beliefs about the world and a weaker relationship with self-blame (Daie‐Gabai, Aderka, Allon‐Schindel, Foa, & Gilboa‐Schechtman, 2011; Muller et al., 2010) as measured by the Posttraumatic Cognitions Inventory (PTCI) (Foa, Ehlers, Clark, Tolin, & Orsillo, 1999). We anticipated similar associations for the PCL-5.

2. Method

2.1. Measures

Prior trauma history was assessed using an adapted version of the Life Events Checklist for DSM-IV (Gray, Litz, Hsu, & Lombardo, 2004) which included two additional items that assessed specifically for experience of childhood physical and sexual abuse. An item was considered to be endorsed as a likely DSM-5 fulfiling experience if either ‘happened to me’ or ‘witnessed it’ were endorsed, with the addition of ‘learned about it’ for ‘sudden violent death’. When more than one event was endorsed, additional questions identified the worst event, along with the age at which the event occurred or started and how long ago it ended. The Life Events Checklist is a widely used screening tool in research studies and has been found to show good test–retest reliability and strong convergence with measures of pathology (Gray et al., 2004).

The PTSD Checklist for DSM-5 (Weathers et al., 2013a) includes 20 self-report items based on the DSM-5 symptoms of PTSD. Respondents report how much they were bothered by a symptom over the past month using a 5-point Likert scale (0 = ‘Not at all’, 1 = ‘A little bit’, 2 = ‘Moderately’, 3 = ‘Quite a bit’, 4 = ‘Extremely’). Total score can range from 0 to 80. The psychometric properties of the PCL-5 are described above. Participants were asked to complete the PCL-5 in relation to the traumatic experience that troubled them most.

We used the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) (Weathers et al., 2013b) as the reference standard for evaluating DSM-5 related PTSD symptomatology and diagnostic status. The previous version of the Clinician-Administered PTSD Scale (CAPS) for DSM-IV (Blake et al., 1995) was widely considered to be the gold standard for assessment of PTSD. CAPS-5 items are designed to correspond with DSM-5 criteria (Weathers et al., 2018). PTSD diagnosis is established by description of exposure to an event involving actual or threatened death, serious injury, or sexual violence, one item each of reliving and avoidance, and two items each of negative changes in cognitions and mood and hyperarousal, functional impairment and the presence of symptoms for at least a month. Initial questions aim to establish the nature of a worst traumatic event in order to ensure that this index event fulfils DSM-5 criteria for a traumatic event. This index event then provides the reference point for subsequent items exploring for current-associated symptoms, which are assessed through 20 items. Items are scored for symptom severity on a 5-point scale (0 = ‘Absent’, 1 = ‘Mild/subthreshold’, 2 = ‘Moderate/threshold’, 3 = ‘Severe/markedly elevated’, 4 = ‘Extreme/incapacitating’). Total score can range from 0 to 80. Functional impairment is evaluated using the same scale for subjective distress, impairment in social function and impairment in occupational functioning. The CAPS-5 can be used to establish symptoms over various time points. We investigated presence of symptoms over the past month. Additional items establish onset and duration of symptoms, the nature of any functional impairment, global validity and overall severity of symptoms. The CAPS-5 has been shown to be a psychometrically sound measure of DSM-5 PTSD (Weathers et al., 2018). We used the CAPS-5 to make a diagnosis of PTSD based on the participant satisfying all DSM-5 criteria.

The Posttraumatic Cognitions Inventory (PTCI) (Foa et al., 1999) is a 33-item measure designed to assess dysfunctional trauma-related cognitions. Items are scored on a 7-point Likert scale (1 ‘totally disagree’ to 7 ‘totally agree’). Previous factor analytic studies (e.g., Hyland et al., 2015) have consistently suggested the presence of three major factors: (1) negative cognitions of the self (Self), (2) negative cognitions of the world and others (World), and (3) self-blame (Blame). The Self scale has been found to have the strongest association with PTSD in several studies (e.g. Daie‐Gabai et al., 2011; Muller et al., 2010). The Self scale includes 21 items, the World scale 7 items and the Blame scale 5 items. Higher scores represent elevated levels of negative belief. All three scales were included in this study.

We assessed for common comorbidity using the depression, panic and somatoform modules of the Patient Health Questionnaire (PHQ) (Spitzer, Kroenke, & Williams, 1999). The PHQ has shown good psychometric properties in a large number of populations (e.g. Kocalevent, Hinz, & Brähler, 2013; Kroenke, Spitzer, Williams, & Löwe, 2010; Lowe et al., 2003; Williams, Pignone, Ramirez, Perez, & Stellato, 2002; Wittkampf, Baas, van Weert, Lucassen, & Schene, 2011).

We used the GAD-7 (Spitzer, Kroenke, Williams, & Lowe, 2006) to assess for generalized anxiety disorder. The GAD-7 has been evaluated in a number of populations and has shown good psychometric properties in several studies (Kroenke et al., 2010; Lowe et al., 2008).

The Inventory of Interpersonal Problems (IIP-32) (Barkham, Hardy, & Startup, 1994) is a short 32-item version of the 127-item Inventory of Interpersonal Problems (IIP) (Horowitz, Rosenberg, Baer, Ureño, & Villaseñor, 1988). The IIP-32 can be rated in terms of an overall score or in terms of 8 subscales: domineering/controlling; vindictive/self-centred; cold/distant; socially inhibited; non-assertive; overly accommodating; self-sacrificing and intrusive/needy. The IIP-32 has been found to have high reliability (α =.90) and confirmatory analysis of the instrument replicated the IIP structure well (Barkham et al., 1994). We used total score to investigate discriminant validity.

The EQ-5D (EuroQoL.Group, 1990) is a very widely used measure of subjective health status based on the dimensions of mobility; self-care; usual activities; pain and discomfort; and anxiety and depression. The EQ-5D has been evaluated with a wide variety of health conditions and has been found to show acceptable psychometric properties in mental health service users (Pitkänen et al., 2012). We used total score to investigate discriminant validity.

The Alcohol Use Disorders Identification Test (AUDIT) is a 10-item questionnaire for assessing the quantity and frequency of alcohol consumption, drinking behaviour and alcohol-related problems or reactions (Saunders, Aasland, Babor, DE LA Fuente, & GRANT, 1993). The AUDIT has demonstrated excellent psychometric properties in a number of studies in English-speaking populations (Reinert & Allen, 2002).

2.2. Procedure

This study received ethical approval from the UK’s National Research Ethic Service and complied with the Code of Ethics of the World Medical Association. Participants were recruited to the All Wales PTSD Registry via the National Centre for Mental Health (NCMH) (http://ncmh.info/), a research centre investigating a number of mental health conditions. Recruitment to NCMH occurred through various means including primary, secondary and specialist mental health services, and social media. NCMH participants were eligible to be recruited to the PTSD Registry if they were over 18 and had previously received a diagnosis of PTSD or reported exposure to a DSM-5 qualifying traumatic event and screened positively for PTSD on the Trauma Screening Questionnaire (Brewin et al., 2002). NCMH participants were informed about the PTSD Registry after taking part in an initial research interview; those that agreed to take part received a diagnostic interview based on the DSM-5 formulation for PTSD alongside a number of other interview and self-report measures. Some participants had received prior psychological and/or pharmacological treatment, although this was not necessarily always for PTSD. Other participants were waiting for or receiving various mental health treatments. In order to maximise completion of the self-report measures and reduce participant burden the self-report questionnaires were mailed to the participant a week before their scheduled interview and collected at the end of the interview. Participants were able to return questionnaires by post if they were not completed prior to interview. Interviewers administered the CAPS-5 without reference to any self-report measure, remained blind to participant responses on the PCL-5 and were not informed of the study objectives. We attempted to contact all eligible participants. Participants were recruited between March 2013 and April 2019. The interview team comprised of a clinical psychologist, a GP with a special interest in the treatment of trauma-related disorders, a specialist mental health nurse, two psychiatrists and several researcher assistants with extensive previous experience of conducting research interviews with mental health service users. Interviewers received extensive training in administration of measures from the first author and met regularly with him for supervision and to discuss administration and scoring issues. The intraclass correlation coefficient (ICC) based on independent judgements of an audio recording of a training case was .87.

2.3. Participants

Five hundred and fifty-one individuals were contacted by the study team and 355 agreed to take part. Of these, 50 (14.1%) were originally recruited from primary care mental health services and 67(18.9%) from a specialist veterans mental health service. (Two hundred and eighty-four of those taking part completed and returned a PCL-5. Eleven individuals were excluded from analysis on the basis that their ‘worst event’ did not fulfil the DSM-5 A criteria for exposure to a traumatic event and interviewers were unable to identify a significant distressing alternative event to focus the interview on. This resulted in a final sample of 273. Four individuals fulfilled DSM-5 A criteria and completed the self-report questionnaires but did not complete full CAPS-5 interviews and another 19 individuals returned their completed questionnaires more than 1 month after completing the CAPS-5. We included the data for these individuals in all analyses that did not involve the CAPS-5. In order to evaluate test–retest reliability, a sub-group of 60 individuals completed the PCL-5 on a second occasion. Respondents were asked to complete retest questionnaires at home 2 weeks after their interview and return their responses by post. In line with Bovin et al., (2015), we only included responses from participants who completed the re-test version within 1 month of the original version. Nine responses were made after this time point and these were excluded from analysis.

Full descriptive characteristics of the sample are provided in Table 2. The mean age of the final sample was 47.5 years (range 18–77). Gender was evenly split between females and males. The majority of participants were Caucasian; around half were married or co-habiting; only a third of participants were working at the time of recruitment.

Table 2. Descriptive characteristics of sample

Characteristic	Sample = 273
Age (mean, SD) Range	47.5 (12.7) 18–77
Gender Female Male	134 (49.1%) 139 (50.9%)
Ethnic background Caucasian Mixed Race Asian Other	258 (94.5%) 10 (3.7%) 2 (0.7%) 3 (1.1%)
Marital status Married or cohabiting Single Divorced/separated Widower Unknown	137 (50.2%) 70 (25.6%) 56 (20.5%) 8 (2.9%) 2 (0.7%)
Educational attainment Left school without qualification Left school with secondary school qualifications Vocational or other qualifications Completed a university degree or higher education Missing	23 (8.4%) 100 (36.6%) 83 (30.4%) 66 (24.2%) 1 (0.4%)
Employment status In employment Not working or retired	93 (34.1%) 180 (65.9%)
Participant self-identified worst trauma Natural disaster Fire or explosion Transportation accident Serious accident Exposure to toxic substances Childhood physical abuse Physical assault Assault with a weapon Childhood sexual abuse Sexual assault Other unwanted or uncomfortable sexual experiences Combat or exposure to war Held in captivity Life threatening illness or injury Severe human suffering Sudden violent death Sudden unexpected death of someone close Serious harm of death you caused Other	2 (0.7%) 12 (4.4%) 26 (9.5%) 4 (1.4%) 0 (0%) 15 (5.5%) 20 (7.3%) 16 (5.9%) 46 (16.8%) 19 (7.0%) 1 (0.4%) 42 (15.4%) 7 (2.6%) 17 (6.2%) 5 (1.8%) 15 (5.5%) 16 (5.9%) 4 (1.4%) 6 (2.2%)

Participants reported direct exposure to or witnessing a mean of 6.7 (SD = 3.5) types of traumatic events and a large proportion reported directly experiencing childhood physical abuse (98; 35.9%) or childhood sexual abuse (99; 36.2%), with 124 (45.4%) experiencing at least one type of childhood abuse. Seventy-eight participants (30.5%) reported exposure to military combat. Self-identified worst traumas are presented in Table 2. The most common worst reported event was childhood sexual abuse, followed by combat or exposure to war and transportation accident.

2.4. Data analysis

Data were analysed using SPSS version 25 (IBM SPSS). We assessed internal reliability of the PCL-5 by computing Cronbach’s α for the 20 PCL-5 items and separately for B, C, D and E criteria items. Cronbach’s α ≥ 0.70 was interpreted as acceptable, ≥ 0.80 as good and ≥ 0.90 as excellent (George & Mallery, 2003). We then calculated test-retest reliability of the PCL-5. Convergent validity was calculated by computing Pearson correlations between the PCL-5 total score and the total score from the CAPS-5 and the Negative Self, World and Blame subscales from the PTCI, depression and panic scales of the PHQ and generalised anxiety using the GAD-7. All correlational data were treated as continuous.

There were no missing data for the CAPS-5. The level of missing data from self-report measures was low. When 10% or more of responses were missing for a measure it was excluded from analysis, except when the measure had less than 10 items, in which case the measure was excluded if more than one item was missing. We imputed missing values from the mean of other responses on the measure. This resulted in n = 273 for the PCL-5, n = 266 for the PTCI Self scale, n = 268 for the PTCI World scale, n = 268 for the PTCI Self Blame scale, n = 270 for the PHQ-9, n = 266 for the GAD-7, n = 267 for the PHQ Panic scale, n = 265 for the PHQ Somatoform scale, n = 266 for IIP32 total score, n = 265 for EQ-5D and n = 265 for the AUDIT. Correlations of 0.3 to 0.5 were interpreted as low, 0.5 to 0.7 as moderate and 0.7 to 0.9 as high (Mukaka, 2012).

In line with previous studies by Bovin et al. (2016) and Murphy et al. (2017) signal detection analysis (Kraemer, 1992) was used to calculate the diagnostic utility of the PCL-5 relative to the CAPS-5 PTSD diagnosis based on full DSM-5 criteria. In evaluating diagnostic accuracy, we sought to minimize risk of bias by following the principles of the Quality Assessment of Diagnostic Accuracy Studies guidelines, version two (QUADAS-2) (Whiting et al., 2012). We calculated specificity, sensitivity, positive predictive value, negative predictive value and diagnostic efficiency for each score of the PCL-5. We then calculated kappa coefficients as measures of test quality for quality of specificity [κ(0)], quality of sensitivity [κ (1)] and quality of efficiency [κ (.5)]. Quality of efficiency [κ (.5)] was used as the key indicator of diagnostic utility (Kraemer, 1992). Kappa values between 0.21 to 0.40 were taken to indicate fair agreement, 0.41 to 0.60 to indicate moderate agreement and 0.61 to 0.80 to indicate substantial agreement (Kraemer, Periyakoil, & Noda, 2002). In concordance with the procedure undertaken by Bovin et al. (2016) and Ho, Schlenger, Kulka, and Marmar (2017) we also undertook signal detection analysis for the PCL-5 following DSM-5 diagnostic rules for B to E criteria, based on the rule of a score of 2 or more indicating symptom presence for each item.

3. Results

One hundred and seventy-two participants (N = 269; 63.0%) met diagnosis for PTSD based on the CAPS-5; 204 (N = 270; 75.6%) screened positive for a probable diagnosis of moderate to severe depression; 111 (N = 266; 41.7%) screened positive for a probable diagnosis of GAD and 175 (N = 267; 65.5%) screened positive for a probable diagnosis of panic disorder. Eighty-two (N = 264; 31.1%) participants indicated problematic alcohol consumption based on a score of 8 or above on the AUDIT.

3.1. Descriptive characteristics of PCL-5

The mean score on the PCL-5 was 49.12 (SD = 18.25, range = 1–80). There was little difference between mean scores by gender (male mean = 48.99, SD = 17.85; female mean 49.24, SD = 18.70), age (< 50, N = 124, mean = 49.27, SD = 19.01; > 50, N = 108, mean = 48.94, SD = 17.43) or by combat exposure (combat exposed mean = 47.9, SD = 17.6; no combat exposure mean = 18.9, SD = 19.0). Differences for exposure to childhood physical or sexual abuse were significant (childhood trauma mean = 51.8, SD = 17.2; no childhood trauma mean = 45.7, SD = 19.4; t(267) = 2.7, p < .0 01). Cronbach’s α for the all PCL-5 items was .94. Subscale values were .91 for intrusions, .81 for avoidance, .84 negative cognitions and mood and .79 alterations in arousal and reactivity.

3.2. Test-retest reliability

Test–retest analyses were conducted from a subgroup of 51 individuals who also completed the PCL-5 on a second occasion within 1 month of (mean of 19.6 days) initial completion. The test–retest correlation was acceptable at r = .84 (p < .0001).

3.3. Convergent and discriminant validity

Table 3 shows correlations between the PCL-5 and other measures used. As expected, the PCL-5 demonstrated fairly strong correlations with the CAPS-5 and the PTCI. Correlations with PHQ-depression and GAD-7 were also strong, with weaker correlations with other measures that were broadly in line with expectations.

Table 3. Convergent and discriminant validity correlations for the PCL-5 and other measures

Measure	1	2	3	4	5	6	7	8	9	10	11	12
1. PCL-5	1.00
2. CAPS-5	.73**	1.00
3. PTCI – negative self-cognitions	.74**	.67**	1.00
4. PTCI – negative world	.64**	.53**	.71**	1.00
5. PTCI – self blame	.36**	.41**	.46**	.36**	1.00
6. PHQ – depression	.74**	.65**	.74**	.54**	.35**	1.00
7. GAD-7	.69**	.54**	.60**	.51**	.25**	.74**	1.00
8. PHQ – panic	.57**	.51**	.44**	.37**	.21**	.51**	.53**	1.00
9. IIP – interpersonal problems	.55**	.50**	.61**	.51**	.30**	.54**	.42**	.26**	1.00
10. Somatoform	.52**	.41**	.43**	.35**	.21**	.55**	.56**	.52**	.31**	1.00
11. EQ-5D – general health	.59**	.44**	.50**	.39**	.17**	.60**	.52**	.40**	.34**	.56**	1.00
12. AUDIT – alcohol	.11	.03	.14*	.12*	.08	.10	.17**	.13*	.14*	.06	−.04	1.00

** Correlation is significant at the 0.01 level (2-tailed).

* Correlation is significant at the 0.05 level (2-tailed).

3.4. Signal detection analysis

Data were available from 216 participants who completed the PCL-5 within one month of their CAPS-5 interview. The Receiver Operating Curve for the PCL-5 compared to PTSD diagnosis as assessed though the CAPS-5 is presented in Figure 1. The area under the curve (AUC) represents the overall accuracy of the PCL–5 in predicting PTSD diagnosis. The AUC was .86 (SE = .028) indicating a good level of diagnostic accuracy. Table 4 shows data on the relative efficiency of the PCL-5 at predicting diagnosis depending on scoring rule. A score ≥ 42.5–43.5 had the highest quality of efficiency for predicting a CAPS-5 diagnosis based on DSM–5 PTSD, with a sensitivity of .88-.89, specificity of.71-.72, efficiency of .82 and κ (.5) of .61, suggesting an optimal cut-off score of 43–44 in this sample. Diagnostic accuracy was lower when the DSM-5 diagnostic rule was applied to the PCL-5 with an AUC of .75 and κ (.5) of .53.

Table 4. Diagnostic utility of the PCL-5 by cut off score at diagnosing PTSD diagnosis based on the CAPS 5

Cut off	Sensitivity	Specificity	PPV	NPV	Efficiency	κ (0)	κ (.5)	κ (1)
25.5	.98	.39	.75	.91	.77	.28	.42	.86
26.5	.98	.41	.76	.91	.78	.30	.45	.87
27.5	.97	.41	.76	.89	.78	.30	.44	.83
28.5	.96	.43	.76	.84	.77	.30	.43	.76
29.5	.96	.47	.77	.85	.79	.34	.47	.77
30.5	.95	.47	.77	.83	.78	.34	.46	.74
32.0	.95	.51	.78	.84	.80	.38	.50	.76
33.5	.95	.52	.79	.85	.80	.39	.52	.77
34.5	.95	.53	.79	.85	.81	.40	.53	.77
35.5	.94	.53	.79	.83	.80	.40	.52	.74
36.5	.92	.56	.80	.79	.80	.42	.52	.68
37.5	.92	.60	.81	.80	.81	.46	.55	.70
38.5	.92	.61	.82	.79	.81	.47	.56	.68
39.5	.92	.64	.83	.80	.82	.50	.58	.69
40.5	.91	.65	.83	.79	.82	.51	.58	.68
41.5	.89	.68	.84	.77	.82	.54	.59	.65
42.5	.89	.71	.85	.77	.82	.57	.61	.65
43.5	.88	.72	.86	.76	.82	.58	.61	.63
44.1	.87	.73	.86	.74	.82	.59	.60	.61
44.6	.86	.73	.86	.73	.81	.59	.59	.59
45.5	.84	.73	.85	.71	.80	.58	.56	.55
46.5	.81	.73	.85	.67	.78	.57	.53	.50
47.5	.79	.75	.86	.66	.78	.58	.52	.48
48.5	.79	.77	.87	.67	.79	.62	.55	.49
49.5	.78	.77	.87	.65	.78	.61	.53	.47
PCL-diagnostic variable	.93	.57	.80	.81	.80	.43	.53	.70

PPV = positive predictive value; NPV = negative predictive value; κ (0) = quality of specificity; κ (.5) = quality of efficiency; κ (1) = quality of sensitivity. Level of PTSD = 65.3%.

Figure 1. ROC curve for the PCL-5 compared to the CAPS-5 diagnosis of PTSD (AUC = 0.86; CI 0.80 − 0.91)

3.5. Post-hoc analysis

Given the strength of the associations between the PCL-5 total and PHQ depression, GAD-7, PHQ panic scores, we decided to undertake a post-hoc analysis in order to investigate the importance of each of these variables along with the CAPS-5 in explaining variance in the PCL-5 using linear regression. We removed GAD-7 from this analysis due to evidence of multicollinearity with depression, which was the variable of greater interest to us. Two outliers were removed following investigation of scatterplots. Preliminary analyses indicated no violations of the assumptions of normality, linearity, and homoscedasticity. The correlations between the predictor variables ranged from .51 to .65. The minimum tolerance value was .54 and the maximum VIF value was 1.86 indicating that the assumption of multicollinearity was not violated. The model as a whole explained 69.6% of the variance in PCL-5 total scores (F (3, 238) = 181.94, p < .0001). PCL-5 scores were most strongly associated with depression (β = .43), CAPS-5 severity (β = .38), then panic (β = .16).

4. Discussion

In this study we evaluated the psychometric properties of the PCL-5 in a mixed civilian and military sample of UK trauma-exposed mental health service users. This is the first evaluation of a PTSD self-report measure in such a sample that we are aware of. We found the PCL-5 to be psychometrically sound. It showed high levels of internal consistency and acceptable test–retest reliability over 1-month period. In terms of convergent and discriminant validity, correlations with the CAPS-5 total and PTCI negative cognitions of self were fairly strong and broadly as expected. As expected, the PCL-5 also correlated strongly with PHQ depression and GAD-7 generalized anxiety scale scores, with more moderate correlations for scores on PHQ panic disorder, IIP interpersonal functioning and EQ-5D general health and weak correlations for self-blame and alcohol use.

These findings are consistent with those from several other studies that have undertaken psychometric evaluation of the PCL-5 (e.g. Bovin et al., 2016; Weathers et al 2018; Wortmann et al., 2016), The strength of the relationship between PCL-5 and depression and GAD scores in our study and these earlier studies does raise some questions about how well the PCL-5 is able to distinguish self-reported symptoms of PTSD from self-reported symptoms of anxiety and depression in mental health users and we note that other studies have also reported marginally stronger associations between the PHQ-9 and PCL-5 than the CAPS-5 and PCL-5 (Bovin et al., 2016; Weathers et al., 2018). A post-hoc linear regression in our study showed that depression explained more of the variance in the PCL-5 total severity score than CAPS-5 severity. The association between depression and the PCL-5 total severity score may be partly explained by the strong conceptual overlap between DSM-5 PTSD and depression, particularly in relation to the D (negative alterations in cognitions and mood) and E (alterations in arousal and reactivity) criteria, although these domains are also a feature of the CAPS-5. The work of Watson (2009) described in the quadripartite model has previously suggested that much of the variance found in PTSD and depression comorbidity is explained by general distress/negative affect. Chronic general distress is likely to be a particular feature of this population and may go some way to explaining some of our findings.

There is some evidence of patients with a primary diagnosis of depression scoring at levels equivalent to those of individuals with PTSD on another widely used self-report measure, the Impact of Events Scale (Brewin, Hunter, Carroll, & Tata, 1996). It is also possible that the stronger relationship between depression as measured by the PHQ-9 and the PCL-5 over the CAPS-5 is partly explained by participant response bias, which may again reflect general levels of distress in this population, rather than necessarily solely indicating disorder-specific symptoms (Coyne, Thompson, Palmer, & Kagee, 2000; Marshall et al., 2019; Marshall, Schell, & Miles, 2010; Watson, 2009). As the CAPS-5 is a clinician-administered measure one would anticipate that it might be less vulnerable to participant response bias.

Some recent work has noted that whilst CAPS-5 and PCL-5 total scores generally correlate well in terms of total scores, significant discrepancies have been found at the individual item level (Kramer, Whiteman, Petri, Spitzer, & Weathers, 2019). A number of sources of discrepancy have been identified, including question comprehension, trauma-related attribution errors and time frame uncertainties (Kramer et al., 2019). This work highlights the challenge of investigating PTSD symptoms via self-report, when contrasted against more considered scoring rules of an instrument like the CAPS-5.

Findings from signal detection analyses showed that the PCL-5 demonstrated a reasonable degree of diagnostic accuracy with an optimal cut-off score of 43–44 in this sample, based on quality of efficiency. These scores gave sensitivity, specificity and efficiency coefficients broadly comparable to those reported in other evaluations of the PCL-5 (Bovin et al., 2016; Ho et al., 2017; Wortmann et al., 2016). The level of diagnostic accuracy is encouraging given that on average the sample had a high level of exposure to multiple types of trauma and the fact that a significant proportion of individuals in this sample did not meet DSM-5 CAPS-5 diagnosis for PTSD but still reported significant subthreshold symptoms. Scoring based on DSM-5 diagnostic rules had less diagnostic accuracy. The optimal cut-off score of 43–44 was higher than that reported in other studies, where scores have ranged from 25 to 42. It is not unusual for self-report measures to demonstrate significant differences in operating characteristics across different populations (McDonald & Calhoun, 2010) and there is some evidence of higher optimal cut-offs in general psychiatric cohorts (Björgvinsson, Kertz, Bigda-Peyton, McCoy, & Aderka, 2013; Grubaugh et al., 2007) and treatment-seeking victims of interpersonal violence (Cody, Jones, Woodward, Simmons, & Gayle Beck, 2017), for both PTSD and other self-report measures.

Variance in operating characteristics can be a result of a range of clinical factors and comorbidity, symptom severity and trauma exposure can all contribute to such variance (McDonald & Calhoun, 2010). The sample in the current study had high rates of comorbidity for depression, GAD, panic disorder and problem drinking, all of which have significant overlapping symptoms with PTSD and are likely to reduce specificity (McDonald & Calhoun, 2010). Such comorbidity rates are fairly typical in PTSD clinical samples. A significant minority of the sample would also have been classified as having severe mental illness. Decisions about whether and at what level to set a cut-off and caseness scores will of course depend upon the purpose for using a measure and the clinical context (Cody et al., 2017; Grubaugh et al., 2007).

In primary care contexts and settings such as psychological treatment services, it might make sense to favour higher sensitivity to ensure that most symptom-positive patients receive further assessment and reduce the risk of false negatives (Grubaugh et al., 2007). In this sample a cut-off of 35 gave a sensitivity of .95, although the associated specificity of 0.53 was low. In making decisions about optimal cut-offs services also need to consider positive and negative predictive values to balance the risks of false positives and false negatives. Such decisions are likely to be determined in part by resource availability and anticipated prevalence rates in the target population.

The final sample size of 273 participants included in this study was fairly large and evaluation against the CAPS-5 as a reference standard followed QUADAS-2 (Whiting et al., 2012) guidelines for investigation of diagnostic accuracy. Our sample included individuals from primary and secondary mental health services and specialist traumatic stress and veterans’ mental health services. Participants had been exposed to a wide range of traumas, with nearly half reporting exposure to childhood trauma. We recognise a number of limitations. We attempted to recruit a consecutive sample from all potential participants who were eligible to take part in our study and our sample was heterogeneous in terms of nature of index trauma and symptom severity. However, the sample underwent some preselection in order to ensure that participants met our inclusion criteria and the sample may not be totally representative of typical trauma-exposed mental health service users. We also recognise that due to the way in which this cohort was recruited, participants may have had more severe and complex mental health problems than a typical general psychiatric cohort, with multiple comorbidities. The sample was demographically diverse in terms of age, gender, educational background and marital status but the majority of participants were not in work. The sample was broadly representative of the ethnic make-up of the population of Wales but not of the population of the UK as a whole. A further limitation is that we did not include interview measures to assess for symptoms of anxiety, depression and other disorders and these disorders were only evaluated with self-report measures. In addition, we did not counterbalance the order in which the PCL-5, CAPS-5 and other measures were completed and this might have affected the responses that participants provided. Finally, we used the LEC version for DSM-IV to screen for trauma exposure, as the DSM-5 version was not available when we began the study. We believe that this is likely to have had little impact on the findings we have reported.

Also, we aimed to include all participants who completed both the CAPS-5 and the PCL-5 in evaluation of diagnostic utility. However, 5.7% of participants were excluded because the time period between completion of the PCL-5 and CAPS-5 was beyond one month. We also recognise that screening tests are most accurate when tested in a sample with prevalence of the disorder of around 50% (McDonald & Calhoun, 2010). Sixty-three percent of the sample contributing to ROC analyses met diagnosis for PTSD. Other recent studies evaluating the PCL-5 have also experienced a similar level of imbalance (Bovin et al., 2016; Wortmann et al., 2016). We were not able to undertake any evaluation of sensitivity to change in this study. We have used the PCL-5 to evaluate symptom change in a separate pilot RCT (Lewis et al., 2017) and found it responsive to change in a way that was consistent with the CAPS-5 but we did not investigate sensitivity to change formally.

The results of this study suggest that the PCL-5 is an appropriate measure to use with general mental health service users with a history of trauma exposure to screen for the presence of DSM-5 PTSD. However, clinicians will want to be mindful that an association with self-reported anxiety and depressive symptoms indicates probable issues with this measures ability to discriminate between PTSD and other common mental disorders. This overlap may not be a major issue as long as clinicians recognise the dangers of relying on instruments such as the PCL-5 to make a diagnosis. It is therefore vital that clinicians conduct further assessment to establish differential diagnosis when PCL-5 scores are raised (Cody et al., 2017; Coyne et al., 2000).

The PCL-5 needs to be evaluated further in clinical samples. It may also benefit from refinement to improve its capability to distinguish PTSD from comorbid disorders such as depression and to shorten the number of items. As we recognised above, this difficulty in distinguishing PTSD from depression may result in part from the DSM-5 formulation of PTSD. It will be of interest to see if measures such as the International Trauma Questionnaire (Cloitre et al., 2018), based on the more specific ICD-11 formulation of PTSD are more discriminating. In light of the difference between the operating characteristics of the PCL-5 found in our sample and those reported in other studies using military and veteran populations (Bovin et al., 2016; Ho et al., 2017; Murphy et al., 2017; Wortmann et al., 2016) it seems reasonable to assume that other PTSD self-report measures might also demonstrate operating characteristics for mental health service users which are different from established norms. It is essential that clinicians and service commissioners recognise the importance of using measures in clinical practice that have been shown to have strong psychometric properties in relevant populations. It cannot be assumed that operating characteristics established in one specific population can be generalised to other populations, including generic mental health service users. It is, therefore, important that robust evaluations of the operating characteristics of screening measures are undertaken before they are adopted for routine use in clinical screening, assessment and review.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The data that support the findings of this study are available on reasonable request from the corresponding author. The data are not publicly available due to their containing information that could compromise the privacy of research participants.

Additional information

Funding

This study is based on data collected for the All Wales PTSD Registry. This project was established through a Clinical Research Fellowship awarded to Neil Roberts by the National Institute for Social Care and Health Research (now known as Health and Care Research Wales).