Using the Post-traumatic Stress Disorder (PTSD) Checklist for DSM-5 to Screen for PTSD in the Chinese Context: A Pilot Study in a Psychiatric Sample

ABSTRACT

Purpose: Post-traumatic stress disorder (PTSD) is a common mental health problem but it is often unrecognized in health care and social service settings. Reliable and valid measures are important for practitioners to screen for PTSD in the Chinese context. This study developed and pilot tested a Chinese version of the PTSD Checklist for DSM-5 (PCL-5). Methods: The reliability and validity of the Chinese version of the PCL-5 were evaluated in a sample of N = 56 adult psychiatric patients in Taiwan. Online methods were used to facilitate the research process. Results: The Chinese version of the PCL-5 is internally consistent and is associated with trauma exposure and other mental health constructs. Participants with clinically diagnosed DSM-5 PTSD scored significantly higher on the PCL-5 than those without PTSD. The PCL-5 also had an acceptable discrimination performance in this sample. Discussion: Although the initial findings suggest that the PCL-5 is reliable and valid in this pilot study, caution should be taken when interpreting the results. The Chinese version of the PCL-5 may be a promising screening tool, but further psychometric evaluation is necessary.

KEYWORDS:

• Post-traumatic stress disorder (PTSD)
• the post-traumatic stress disorder checklist for DSM-5 (PCL-5)
• assessment
• validation
• online methods

Post-traumatic stress disorder (PTSD), which was also called “combat neurosis” or “shell shock” historically, is a mental health problem that affects some people who have experienced one or more traumatic events. PTSD has been recognized as an official diagnosis in the Diagnostic and Statistical Manual of Mental Disorders (DSM) since 1980 (American Psychiatric Association, 1980). There are four major PTSD symptom clusters recognized by the DSM-5, which include intrusion symptoms, avoidance, negative alterations in cognitions and mood, and alterations in arousal and reactivity (American Psychiatric Association, 2013). The literature shows that PTSD may affect about 1.9% to 8.8% of the general population and that the prevalence of PTSD is much higher in some specific populations, such as conflict and violence victims (Bisson, Cosgrove, Lewis, & Roberts, 2015). PTSD usually co-occurs with other mental health problems (e.g., depression, substance abuse, borderline personality disorder) (Zlotnick, 1997). Although PTSD can lead to significant distress and impairment, it often remains unrecognized in health care and social service settings (Cusack, Grubaugh, Knapp, & Frueh, 2006; da Silva et al., 2018; Makput, Dami, Piwuna, Haa, & Maton, 2018). Early identification of PTSD is important because timely psychosocial interventions (e.g., social work support and evidence-based treatments such as prolonged exposure therapy and cognitive behavioral therapy) can be offered to those service users with PTSD. Therefore, a number of easily administered screening tools for PTSD have been developed in the field. Among them, the Post-traumatic Stress Disorder Checklist for DSM-5 (PCL-5) is one of the most commonly-used self-report measures of PTSD. It has sound psychometric properties (Blevins, Weathers, Davis, Witte, & Domino, 2015).

In the Chinese context (i.e., in populations in which Chinese is the primary language), PTSD is not uncommon, although the available literature is limited. The Division of Clinical Psychology of The Hong Kong Psychological Society (2015) found that the prevalence of PTSD was 2.5% in a representative general population sample in Hong Kong (N = 4748). Another study indicated that PTSD affected 18.8% of participants in a Chinese sample of earthquake survivors (N = 180) (Wang, Zhao, & Naotaka, 1999). Similarly, Lai, Chang, Connor, Lee, and Davidson (2004) reported that the prevalence rates of full and partial PTSD were 10.3% and 19.0%, respectively, in a Taiwan sample of earthquake survivors (N = 252). PTSD may also affect Chinese people living in Western countries – de Bocanegra and Brickman (2004) investigated the mental health impact of the World Trade Center attack in a sample of immigrant Chinese displaced workers (N = 77) and found that 21% of participants met the diagnostic criteria for PTSD. Although little is known about the prevalence of PTSD in Chinese clinical settings, we believe that PTSD is also common among Chinese mental health service users given the high prevalence of trauma reported in Chinese mental health settings – for example, Chiu et al. (2017) found that childhood physical or sexual maltreatment was reported by 40% of participants in a group of non-dissociative disorder acute psychiatric inpatients in Taiwan (N = 70); in another study in a sample of Hong Kong mental health service users (N = 202), Fung, Ross, Yu, and Lau (2019) found that 38.6% of participants reported either or both childhood physical and sexual abuse.

Because of these considerations, it is important to have reliable and valid assessment tools for health care and social service providers to screen for PTSD in Chinese service users. Currently, the Chinese version of the PCL is available (Li et al., 2010; Wu, Chan, & Yiu, 2008). However, the findings from it were reported almost 10 years ago and studies using the measure have not been conducted in psychiatric settings since then. More importantly, the Chinese version of the PCL was designed for DSM-IV – the PCL cannot reflect the DSM-5 criteria for PTSD and the PCL and the PCL-5 cannot be used interchangeably. To our knowledge, no measure of DSM-5 PTSD is available in the Chinese context. Thus, in order to provide a helpful screening tool for Chinese health care and social service providers and researchers, we developed and pilot tested a Chinese version of the PCL-5 in a psychiatric sample. In particular, we evaluated the reliability (internal consistency), construct validity (its relationships with trauma and mental health measures) and discrimination performance of the Chinese version of the PCL-5.

Methods

Scale development and translation

A collaborative approach was used to translate three trauma measures into Chinese because back translation does not necessarily ensure equivalence in meaning and concepts in each culture (Douglas & Craig, 2007; Khosravani & Dastjerdi, 2013). These included the PCL-5, the Life Events Checklist for DSM-5 (LEC-5) (Weathers et al., 2013) and the Primary Care PTSD Screen for DSM-5 (PC-PTSD-5) (Prins et al., 2016).

Three bilingual researchers (one with a psychology background, one with a social work background with experience in trauma research, and one with a linguistics background) independently translated the PCL-5 and the PC-PTSD into Chinese. For the LCE-5, since the Chinese version of the LEC for DSM-IV is already available (Chu, 2004) and since there are only a few changes between the original LEC and the LEC-5, we developed the Chinese LEC-5 primarily based on the existing Chinese LEC. All translated versions were then compared and discussed in an online research-group committee in order to develop a preliminary Chinese version of each scale. The committee included the three translators in addition to a Taiwan psychiatrist and an experienced Chinese-American counseling psychologist who specializes in working with trauma survivors. Care was taken to focus on the conceptual equivalence rather than the literal equivalence between the English version and the Chinese version of each scale.

Participants and procedures

The study was approved by The Hong Kong Polytechnic University (PolyU) Human Subjects Ethics Sub-committee. In order to pilot test the Chinese version of the PCL-5, during the period from December 2018 to April 2019 we recruited adult psychiatric patients who had been assessed using clinical diagnostic interviews in a Taiwan psychiatric clinic to complete an online survey. All clinical diagnostic interviews were made according to DSM-5 rules and were conducted by a psychiatrist (the third author) who is experienced in treating trauma-related disorders and is well-trained in EMDR. Participants with and without DSM-5 PTSD based on the clinical diagnostic interview results were invited to complete the same set of measures in Google Forms. The URL for the study was included in the recruitment flyer. The psychiatrist who conducted made the clinical interviews was blind to the online survey and the survey results, since the interviews were conducted prior to the survey. Informed consent from each participant was also obtained in the online survey; online methods are increasingly used for research purposes in the fields of health and social sciences (Chan, 2016; Chan, Fung, Choi, & Ross, 2017; Fung, Choi, Chan, & Ross, 2018).

A total of N = 56 patients (17 males, 37 females and 2 transgendered participants) from the clinic gave consent and completed the online survey. Their ages ranged from 18 to 66 (M = 33.2, SD = 11.2). Among all participants, 34 participants completed the Google Form for patients with PTSD – this means that 60.7% of the 56 participants had a clinically diagnosed DSM-5 PTSD.

Measures

The following self-report measures were completed by all participants in the online survey.

The Chinese version of the Post-traumatic Stress Disorder Checklist for DSM-5 (PCL-5)

The PCL-5 has 20 items and can be used to assess the 20 DSM-5 symptoms of PTSD with good reliability and validity (Blevins et al., 2015; Bovin et al., 2016). The PCL-5 uses a 5-point scale (0 = “Not at all” to 4 = “Extremely”). Although the PCL-5 is brief and easy to administer, it assesses all symptom clusters of DSM-5 PTSD occurring in the past month. A total score of the PCL-5 can be calculated by summing the scores of each item. As mentioned, the Chinese version of the PCL-5 was translated for this study.

The Chinese version of the Primary Care PTSD Screen for DSM-5 (PC-PTSD-5)

The PC-PTSD-5 is a very brief screening tool for PTSD with excellent diagnostic accuracy (Prins et al., 2016). Unlike the previous version (PC-PTSD) (Cameron & Gusman, 2003), the PC-PTSD-5 asks whether a person has had any trauma exposure recognized in DSM-5 at the beginning of the measure; after that, there are five dichotomous (Yes/No) items to assess post-traumatic symptoms over the past month. Persons who report no trauma exposure do not need to complete the other five symptom items. A total score of the PCL-5 can be calculated by the number of “Yes” in the five symptom items. As mentioned, the Chinese version of the PC-PTSD-5 was translated for this study.

The Chinese version of the Life Events Checklist for DSM-5 (LEC-5)

The LEC, which was originally a part of the Clinician-Administered PTSD Scale for DSM-IV (CAPS), has 17 items to assess different types of trauma exposure (Weathers et al., 2013). The LEC-5 is slightly different from the original LEC (i.e., one item was changed and one response category was added). Participants can endorse one or more responses for each item (i.e., “happened to me”, “witnessed it”, “learned about it”, “part of my job”, “not sure” and/or “doesn’t apply”). The Chinese version of the LEC-5 was developed primarily based on the original Chinese LEC. In this study, a participant was counted as having experienced a particular traumatic event only if he/she endorsed the response options “witnessed it” and/or “happened to you” for that item.

The 20-item Taiwan version of the Borderline Personality iIventory (BPI-T20)

The BPI is a reliable and valid measure of borderline personality organization and borderline personality disorder (BPD) (Leichsenring, 1999). Based on the original BPI, Lee, Wen, Yeh, Lee, and Chong (2009) developed and validated a Taiwan (Chinese) version of the BPI which only has 20 dichotomous (Yes/No) items. A total score of the BPI-T20 can be calculating by counting the number of “Yes”.

Self-rated mental health

A single-item measure of self-rated mental health (SRMH) was also included in the online survey. Using a 5-point scale, the SRMH asked each participant, “how would you rate your overall mental health” (Ahmad, Jhajj, Stewart, Burghardt, & Bierman, 2014). The SRMH item was translated into Chinese by the researchers.

Data analysis

The reliability of the PCL-5 was examined by calculating its internal consistency (Cronbach’s alpha). To evaluate the construct validity of the PCL-5, we examined (1) the correlation between the PCL-5 and trauma exposure (i.e., the LEC-5), (2) the correlation between the PCL-5 and another PTSD measure (i.e., PC-PTSD-5), (3) the correlation between the PCL-5 and other mental health measures (i.e., SRMH and BPI-T20), and (4) the mean differences in the PCL-5 scores between participants with and without DSM-5 PTSD. We also calculated the sensitivity and specificity of the PCL-5 for detecting DSM-5 PTSD (receiver operating characteristic curve analysis).

Results

Reliability

Regarding internal consistency, the Cronbach’s alpha of the PCL-5 was .951.

According to Nunnally and Bernstein (1994), an acceptable corrected item-total correlation should be above 0.3. In this sample, all PCL-5 items had acceptable corrected item-total correlations. Except for item 8 (r = .370), all other PCL-5 items had corrected item-total correlations of > .52.

Construct validity

The PCL-5 was significantly correlated with another PTSD measure, which is the PC-PTSD-5 (r = .443, p = .001).

The PCL-5 was also significantly correlated with the number of traumatic events as measured with the LEC-5 (r = .415, p = .001), SRMH (r = −.318, p < .05) and the BPI-T20 (r = .675, p < .001).

Participants with DSM-5 PTSD (n = 34) scored significantly higher on the PCL-5 total score than participants without DSM-5 PTSD (n = 22) (M = 54.06, SD = 16.16 vs M = 38.72, SD = 20.90), t = 3.087, p = .003, Cohen’s d = 0.82.

Sensitivity and specificity

A cutoff score of 49 seems to yield an optimal balance between sensitivity and specificity in this sample. However, one may consider a lower cutoff score for screening purposes (see Table 1).

Table 1. Sensitivity and specificity of the Chinese version of the Post-traumatic Disorder Checklist for DSM-5 (PCL-5).

PCL-5 total score	Sensitivity	Specificity	Youden’s Index
42	0.735	0.545	0.28
44	0.706	0.591	0.297
46	0.706	0.682	0.388
49	0.706	0.727	0.433
52	0.676	0.727	0.403
53	0.647	0.773	0.42
54	0.618	0.773	0.391

The area under the curve (AUC) of the PCL-5 was .714. According to Hosmer and Lemeshow (2000), AUC between 0.7 and 0.8 is regarded as having acceptable discrimination performance. It should be noted that the PCL-5 performed better than the PC-PTSD-5 (AUC = .638) in this pilot study. One possible reason for this finding is that many participants with PTSD did not endorse the trauma exposure item on the PC-PTSD-5 and therefore they did not complete the remaining items.

Discussion

This pilot study evaluated the reliability and validity of the PCL-5 in a sample of Chinese-speaking psychiatric patients in Taiwan. We also validated the PCL-5 against a diagnosis of DSM-5 PTSD based on a clinical diagnostic interview to establish its diagnostic accuracy. As expected, the PCL-5 is associated with trauma exposure and other mental health constructs; patients with PTSD scored significantly higher on the PCL-5 than those without PTSD. Our initial findings show that the PCL-5 may be a reliable and valid measure in the Chinese context. It has acceptable discrimination performance as a screening tool for DSM-5 PTSD in this sample. There are several issues that require further discussion.

Needs for further research

Although this study shows that the Chinese version of the PCL-5 may be a promising screening tool in the Chinese context, there are several limitations of our study and further evaluation is needed. For example, the size of the sample was not large enough and participants were recruited from only one clinic. Additionally, a structured diagnostic interview should be used in future validation studies to further establish the sensitivity and specificity of the Chinese version of the PCL-5. The factor structure and test-retest reliability of the PCL-5 should be examined in the future as well. In addition, the interval between the clinical diagnostic interview and the online survey was unknown in this study. Having noted these limitations, this study could facilitate further psychometric evaluation and encourage more PTSD research and assessment in the Chinese context. In particular, future studies should evaluate the Chinese version of the PCL-5 in larger samples and further establish its performance as a screening tool.

The PCL-5 as a promising screening tool in the Chinese context

Given that PTSD is often overlooked in the field, we suggest that Chinese health care and social service providers should carefully screen for PTSD in clinical settings. Since the PCL-5 only has 20 items and is easily administered, it has the potential to become a helpful tool in routine practice. If the diagnostic performance of the Chinese version of the PCL-5 can be further established in future studies, practitioners might consider inviting their service users to complete the PCL-5 in a waiting room or on a Web-based assessment form. In particular, after further evaluation, the PCL-5 may be used for screening purposes when working with people who have mental health problems. For screening purposes, a lower cutoff score should be considered in order to increase its sensitivity. Service users who screen positive for PTSD should be followed up and assessed comprehensively by qualified mental health practitioners.

In addition, a further-evaluated Chinese PCL-5 would be very useful in health care and social service settings for vulnerable and potentially traumatized people. Previous studies suggested that trauma and post-traumatic conditions are common in people who need medical and rehabilitation services (Fung, 2016; Krnjacki, Emerson, Llewellyn, & Kavanagh, 2016; Young, Nosek, Howland, Chanpong, & Rintala, 1997). A reliable and valid PTSD measure can facilitate screening of PTSD and post-traumatic symptoms in Chinese people who have encountered traumatic or stressful events, such as family violence, traffic accidents, medical procedures, sudden illnesses and surgery.

Using the PCL-5 for research purposes in the Chinese context

Given that the available literature regarding PTSD (especially DSM-5 PTSD) among Chinese people is limited, the development of the Chinese version of the PCL-5 could encourage and facilitate more trauma and PTSD studies being conducted in the Chinese context. Our study revealed that psychiatric patients with clinically diagnosed PTSD can be detected using online assessment (web-based PCL-5) with acceptable discrimination performance. After further evaluation, future studies may also use a web-based PCL-5 to investigate post-traumatic reactions in Chinese populations. Given that the PCL-5 is quite brief, it could be used in large surveys together with other measures.

Online methods could facilitate research and collaboration in the academic setting

In this study, online methods facilitated the research process. For example, the collaborative translation was conducted entirely online, which allowed the involvement of scholars from different places. This implies that online methods have a great potential to facilitate social work research as well as cross-disciplinary and cross-regional collaborations. Online methods can facilitate both survey studies and intervention studies (e.g., Chan, 2016; Chan & Holosko, 2019; Fung & Chan, 2019; Fung, Ho, & Ross, 2018).

Concluding remarks

This pilot study evaluated the reliability and validity of the Chinese version of the PCL-5. In particular, we examined whether the measure is useful to screen for DSM-5 PTSD in a psychiatric sample. To our knowledge, the Chinese version of the PCL-5 is the first validated measure for DSM-5 PTSD in the Chinese context. While the initial findings suggest that the Chinese version of the PCL-5 is reliable and valid in our sample, caution should be taken when interpreting the results of this pilot study; further evaluation of the measure is necessary.