Measuring workplace trauma response in Australian paramedics: an investigation into the psychometric properties of the Impact of Event Scale

Abstract

Introduction

Investigation into the psychological effects of violence toward health care workers and its associated trauma is increasing. The Impact of Event Scale (IES) provides a measure of current, subjective, emotional distress symptomatic of a specific traumatic event. However, its validity among paramedics is largely unknown.

Problem

The purpose of this study was to investigate the psychometric properties and factor structure of the IES with a sample of Australian paramedics.

Methods

The study aimed to investigate the psychometric properties and factor structure of the 15-item IES with a sample of Australian paramedics using Exploratory Factor Analysis with model fit statistics as found in confirmatory analysis.

Results

Maximum Likelihood Factor Analysis with Varimax rotation supported the hypothesis that a two-factor solution would provide the best fit of the data. Procrustes rotation provided further support for this hypothesis indicating that the factors, labeled “Intrusion” and “Avoidance”, as well as the individual items of the 12-item final model, were a good fit to an ideal solution.

Conclusion

The revision of the scale has improved its validity for use in the general population of paramedics, improving the potential for its use in trauma-related research.

Keywords:

• impact of event scale
• psychometrics
• paramedics
• occupational violence
• PTSD

Introduction

Investigation into the psychological effects of violence toward health care workers and its associated trauma is intensifying.^1,2 As the terminology used to study workplace violence is varied,^2–4 this paper will seek to use a health care research-based definition, referring to occupational-based violence (OBV), which is defined as “hitting with a body part, slapping, kicking, punching, pinching, scratching, biting, pulling hair, hitting with an object, throwing an object, spitting, beating, shooting, stabbing, squeezing, and twisting”.⁵ Though incidence rates of OBV in health care are not well described,⁶ paramedics have been reported as being at elevated risk^7,8 and thus this investigation will focus on the psychological effects of OBV in this population.

Risk of OBV in paramedics

The reported risk of OBV that paramedics face varies greatly. For example, Corbett et al⁹ have reported 61% of firefighters, paramedics, and emergency medical technicians in Southern California as experiencing violence within the previous year, while Suserud et al¹⁰ reported a rate of 80.3% of paramedics in a Swedish sample and Pozzi¹¹ reported that figure in paramedics to be as high as 90% in a New Mexican sample. The most recent OBV estimate by Boyle et al¹² in Australian paramedics was a rate of 87.5% in the previous year. These off-site workers have been reported to be at an increased risk or vulnerability to OBV due to the highly volatile, unpredictable, and comparatively isolated nature of their work^8,9 as well as the tendency for the patients that they interact with to be drug affected.⁸

Post-traumatic stress disorder screening in victims of OBV

The most commonly utilized tools for screening for post-traumatic stress disorder (PTSD) symptoms include the Post-traumatic Diagnostic Scale¹³ the Mississippi Scale,¹⁴ the Penn Inventory,¹⁵ the Clinician-administered PTSD Scale,¹⁶ the Post-traumatic Stress Symptom Scale,¹⁷ and the PTSD Checklist—civilian version.¹⁸ However, consistency in PTSD instrument use with workers exposed to OBV has not been established. The fact that the diagnostic criteria for PTSD in the Diagnostic and Statistical Manual (DSM)¹⁹ has changed between recent editions²⁰ and that PTSD was not a recognized disorder before 1980²¹ may contribute to an explanation for this inconsistency.

Impact of Event Scale

Despite the changing diagnostic criteria for PTSD and the creation of new instruments used to measure its effects, the Impact of Event Scale (IES)¹ remains in use. For this reason, the IES can be seen as providing a standard source of post-traumatic stress symptom data for almost 35 years. A self-report measure, the IES was developed by Horowitz et al¹ and assesses the “typical symptomatic response to exposure to traumatic life events (and) current subjective distress”. The IES was initially administered with a small sample of 66 participants, who were chosen from a pool of participants seeking psychotherapy as a result of reactions to a variety of serious life events. The majority of life events used as the referent for the scale were bereavement related with the remainder including personal injuries resulting from accidents, violence, illness, and surgery. Informed by literature and psychotherapy interviews, Horowitz^22,23 posited that the experience of a traumatic event presents an individual with information, which is incompatible with their existing understanding of the world. This incompatibility leads to distress and a consequent need for the revision and assimilation of that understanding. During that period of change, the individual experiences phases of psychological intrusion and avoidance of the traumatic event that fluctuates until the person is said to have dealt with the experience. At this point, they reach a relative baseline. Beyond this period, intrusion and avoidance may become pathological. The scale comprises 15 items including two subscales measuring intrusive thoughts and those of avoidance, which give a total subjective stress score. The items were derived from the most frequent statements made by participants of previous psychotherapy interviews^24,25 about episodes of distress that followed traumatic life events. Horowitz adjusted the wording of the statements through further use with psychotherapy patients undergoing stressful events and to allow for their application to any traumatic event.

Use of the IES

Administration of the IES has been broad across studies investigating the experience of trauma. In a study evaluating the use of the IES over a 20-year period, Sundin and Horowitz²⁶ report the measure being used for traumatic events such as episodes of illness and injury; natural and technological disaster; bereavement and loss; violence; sexual abuse; and war exposure. The IES has been revised (IES-R²¹) to account for the revised diagnostic criteria for the DSM-IV.²⁷ However, the DSM has undergone revision again¹⁹ and thus the suitability of the IES-R for use has again been brought into question.²⁰ More recent studies have used the IES to measure psychological stress reactions in the comparison of negative acts²⁸ as well as road accident survivors,²⁹ college students,^30,31 and natural disaster survivors.³²

Validity of the IES

The original authors of this scale¹ as well as many subsequent validity studies of the IES have reported a two-factor structure comprising the subscales Intrusion and Avoidance (Table 1). Zilberg et al³³ were the first to investigate and report on the factor structure since Horowitz et al.¹ They conducted a principal component analysis with Varimax rotation retaining all original items and factors explaining 56.0% of the variance. The ability to generalize the results of this study was limited as it used a different sample group (who were undergoing bereavement), which has been argued to result in different factor structures of the IES.³⁴ Zilberg et al³³ also had fewer participants than the suggested five per variable/item³⁵ and included some items in the final factor solution that had very “weak”³⁶ factor loadings.¹¹

Table 1 Populations yielding a two-factor solution from IES administration

Population	n	Method	Variance explained (%)	Reference
Male combat veterans	382	PCA with Varimax rotation	57.0	Schwarzwald et al^Citation54
Adult survivors of a cruise ship disaster	73	PCA (rotation undefined)	56.0	Joseph et al^Citation55
Adolescent survivors of a cruise ship disaster	334	PCA with Varimax rotation	61.1	Yule et al^Citation56
Female bank staff who were victims of armed raids	228	PCA with Varimax rotation	55.6	Hodgkinson and Joseph^Citation57
War veterans	281	PCA with Varimax rotation	47.7	Robbins and Hunt^Citation58
Italian road accident survivors	74	PCA with Quartimax rotation	44.3	Pietrantonio et al^Citation29
Non-clinical sample of American college students	304	Maximum likelihood factor analyses with Varimax rotation	55.3	Thatcher and Krikorian^Citation30

Abbreviations: PCA, principal component analysis; IES, Impact of Event Scale.

The reported factor structure of the IES has varied widely. There has been a lack of population specificity within which the IES has been validated and the definitions of the underlying psychological characteristics of the instrument itself have been modified over time. The difficulty in obtaining a large enough sample from any one distinct type of trauma has also been noted in previous validation studies of the IES.³⁷ Consequently, considering the rates of OBV in a paramedical population, factor analysis with such a sample would prove advantageous. The current study aimed to investigate the psychometric properties and factor structure of the IES with a sample of Australian paramedics using Exploratory Factor Analysis (EFA). Evaluation of the factor structure was based on analysis of congruence coefficients after a Procrustes targeted rotation. It was hypothesized that EFA with model fit statistics as found in confirmatory factor analysis would provide support for a two-factor solution corresponding to the two subscales of Avoidance and Intrusion.

Methods

Participants

Participants were paramedics employed by the South Australian Ambulance Service and Rural Ambulance Victoria.

Instrumentation

The IES consists of 15 items, 7 of which measure intrusive thoughts and 8 of which measure thoughts of avoidance. Respondents read 15 statements about a single, traumatic life event and indicate, on a four-point Likert scale, how many times during the previous 7 days that they have experienced each: (0) “not at all”, (1) “seldom”, (3) “sometimes”, or (5) “often”.

The scale has been validated in several studies, though results tended to vary between samples and over time.^1,33 Good internal consistency has often been reported (total values of Cronbach’s alpha of 0.91; Intrusion values 0.78–0.92; Avoidance values 0.82–0.91) as has test–retest reliabilities (total Pearson’s r values 0.80–0.87; Intrusion values 0.75–0.89; Avoidance values 0.78–0.79 over 1–8 weeks).^1,33,38,39

Procedure

The study was approved by Monash University Human Ethics Committee. The IES was administered as part of a larger, 98-item questionnaire (available on request) given to a sample of paramedics. The sample of paramedics was recruited through the South Australian Ambulance Service and Rural Ambulance Victoria, whose randomly selected paramedic members were mailed questionnaires with a return envelope. Distribution names were blinded to the study team. The nature of the research, assurances of confidentiality, and counseling contacts were given to participants as part of the survey, which was completed anonymously and sent back to a central location. A total of 255 surveys were returned which included 13 students, who were subsequently removed from the analysis. A further 12 participants were removed as they only provided demographic information. No records were made available to indicate the level of response rate.

Statistical analysis

The 15 items of the IES were subjected to Maximum Likelihood Factor Analysis with Varimax rotation and a two-factor forced solution using Statistical Package for Social Sciences (SPSS), IBM SPSS Statistics for Windows, Version 20.0 (IBM Corporation, Armonk, NY, USA). To evaluate the factor structure of the new sample, Procrustes rotation⁴⁰ using Orthosim version 2.01⁴¹ was used, which provides model fit statistics analogous to those found in confirmatory factor analysis. Using factor tables generated by EFA techniques, Procrustes involves carrying out a targeted rotation of new sample table loadings with an ideal matrix where items either load completely or not at all. The resulting report of congruencies between loadings provides an estimate of fit for the overall structure as well as each variable and factor.⁴² Items with values below 0.80 suggest a less than ideal item fit.⁴³

Results

The mean age of participants was 41 years of age, the majority being male n=172, who had worked on average 45.2 hours a week. A full demographic profile of the 230 participants is presented in Table 2.

Table 2 Sample characteristics

Characteristic	n	M	SD
Age (years)		40.9	9.0
Hours worked per week		45.2	10.7
Years in occupation		14.7	9.2
Hours in direct client contact		25.8	12.9
Practice location
Urban	105
Rural or remote	122
Undefined	3
Sex
Female	55
Male	172
Undefined	3

Abbreviations: M, means; SD, standard deviation.

Missing values

Missing value analysis was conducted and all variables had low levels of missing data (mean rate 0.28%, with items 2 and 15 having the highest proportion of missing data at 0.9%). Imputation of missing values was conducted using the SPSS Expectation-Maximization (EM) imputation algorithm. This algorithm uses a maximum likelihood approach to iteratively generate values using a normal distribution.⁴⁴ No significant difference was obtained between variable means both before and after applying the EM imputation method (Intrusion subscale χ² =18.75, df =18, P=0.41; Avoidance subscale χ² =50.26, df =35, P=0.05), and therefore it was concluded there was no pattern for the missing data.⁴⁴ When data are missing completely at random and only a very small portion of data are missing (eg, <5% overall), imputation using the EM algorithm provides unbiased parameter estimates and improves statistical power of analyses.⁴⁵

Exploratory factor analysis

The correlation matrix was inspected for factorability and evidence of coefficients >0.3 was found, attesting to the strength of the relationship among variables or among items.³⁵ Kaiser-Meyer-Olkin measure of sampling adequacy (0.93^46,47 and Bartlett’s test of sphericity (χ² =2,284.55, df =105, P <0.001) indicated that the sample was appropriate for factor analysis.⁴⁸ The sample size of 230 with a 15-item questionnaire produced a ratio of participants to items of 15:1, which was greater than the recommended 10:1,⁴⁹ and as the number of cases exceeded 150, was considered as “sufficient”.⁴⁸ A review of Catell’s scree plot⁵⁰ and of Parallel Analysis⁵¹ supported only one factor; however, Tabachnick and Fidell⁴⁸ suggest to explore one factor more and one factor less than suggested by the scree test and Parallel Analysis. Also, the IES was theoretically designed to measure two factors. Given these conditions, further analysis was conducted retaining two factors.

In total, the two-factor solution accounted for 56.39% of the total variance in scores and the rotated factor matrix is presented in Table 3. Communality values tended to be relatively high with the exception of the item originally written for the Avoidance subscale “I felt as if it hadn’t happened or wasn’t real”, labeled A6. As this item had the lowest communality of 0.29, it correlated very weakly with other items and loaded weakly and primarily on the Intrusion factor. It was consequently excluded from the next factor analysis. Two subsequent factor analyses conducted before determining the final solution removed items A5 and I3 on varied bases of low communality; counterintuitive primary and cross-loading and incongruous and ambiguous wording.

Table 3 Initial rotated factor matrix for maximum likelihood factor analysis with Varimax rotation of IES items and communalities (N=230)

Item	h^2	F1	F2
I4: Pictures about it popped into my mind	0.70	0.76	0.37
I2: Other things kept making me think about it	0.63	0.72	0.33
I7: I had dreams about it	0.54	0.67	0.32
I5: Any reminder brought back feelings about it	0.62	0.64	0.46
I1: I had waves of strong feelings about it	0.56	0.61	0.43
I6: I had trouble falling asleep or staying asleep because of pictures or thoughts about it that came into my mind	0.58	0.61	0.46
A1: I was aware that I still had a lot of feelings about it, but I did not deal with them	0.60	0.58	0.52
A5: My feelings about it were kind of numb	0.42	0.56	0.32
A6: I felt as if it had not happened or was not real	0.29	0.51	0.16
A3: I tried to remove it from memory	0.72	0.29	0.80
A2: I avoided letting myself get upset when I thought about it or was reminded of it	0.56	0.24	0.71
A8: I tried not to think about it	0.66	0.44	0.68
A7: I stayed away from reminders of it	0.58	0.49	0.58
I3: I thought about it when I did not mean to	0.52	0.45	0.56
A4: I tried not to talk about it	0.47	0.44	0.52

Notes: Factor loadings >0.40 are in boldface. h², Communalities; F₁, Intrusion; F₂, Avoidance. Items in table are presented in order of original presentation in Horowitz et al.¹

Abbreviation: IES, Impact of Event Scale.

The final model that provided the best overall fit is described in Table 4. After the removal of item I3 from the analysis, the solution accounted for 60.37% of the total variance in scores. Factorability of the correlation matrix was again supported with a Kaiser-Meyer-Olkin value of 0.93 and a significant Bartlett’s⁵² test of sphericity. Though, once again, Catell’s⁵⁰ scree test indicated one factor.

Table 4 Final rotated factor matrix for maximum likelihood factor analysis with Varimax rotation of IES items, communalities and congruence coefficients for an ideal target matrix (N=230)

Item	h^2	F1	F2	CC
I4: Pictures about it popped into my mind	0.76	0.80	0.35	0.92
I2: Other things kept making me think about it	0.67	0.75	0.33	0.92
I7: I had dreams about it	0.55	0.66	0.34	0.89
I6: I had trouble falling asleep or staying asleep because of pictures or thoughts about it that came into my mind	0.59	0.63	0.43	0.83
I5: Any reminder brought back feelings about it	0.61	0.58	0.52	0.75
I1: I had waves of strong feelings about it	0.53	0.56	0.46	0.78
A8: I tried not to think about it	0.71	0.40	0.75	0.88
A3: I tried to remove it from memory	0.65	0.32	0.75	0.92
A7: I stayed away from reminders of it	0.61	0.42	0.66	0.84
A2: I avoided letting myself get upset when I thought about it or was reminded of it	0.48	0.28	0.64	0.91
A1: I was aware that I still had a lot of feelings about it, but I did not deal with them	0.61	0.51	0.59	0.75
A4: I tried not to talk about it	0.48	0.39	0.58	0.82
Congruence coefficient (CC)		0.85	0.86	0.85

Notes: Factor loadings >0.40 are in boldface. h², Communalities; F₁, Intrusion; F₂, Avoidance.

Abbreviation: IES, Impact of Event Scale.

Procrustes analysis

Procrustes transformation indicated global solution congruence with an ideal target matrix of 0.85, suggesting a good fit.⁴³ The factors within the global solution also displayed good congruence^42,43 with an ideal matrix (Intrusion 0.85 and Avoidance 0.86) as did most items (range 0.75–0.92). The results are presented in Table 4.

Reliability of the final model

Reliability analysis indicated adequate internal consistency with a Cronbach’s alpha coefficient of 0.92. The subscales demonstrated acceptable reliability,⁵³ with Cronbach’s alpha of 0.89 and 0.90 for the subscales of Avoidance and Intrusion, respectively. Inter-item correlation ranged from 0.45 to 0.69 in the Avoidance subscale, from 0.49 to 0.74 in the Intrusion subscale and from 0.31 to 0.74 in the overall scale, providing further support for the reliability of the scale. See Table 5 for subscale alpha, means, standard deviations, Kurtosis and Skewness coefficient data.

Table 5 Subscale alpha, means, standard deviations, Kurtosis, and Skewness coefficients

Subscale	Item	α	M	SD	S	K
Avoidance		0.89
	A1: I was aware that I still had a lot of feelings about it, but I did not deal with them		0.88	1.45	1.67	1.70
	A2: I avoided letting myself get upset when I thought about it or was reminded of it		1.57	1.88	0.78	−0.92
	A3: I tried to remove it from memory		1.33	1.82	1.00	−0.51
	A4: I tried not to talk about it		0.94	1.60	1.65	1.35
	A5: My feelings about it were kind of numb		0.68	1.35	2.10	3.36
	A6: I felt as if it had not happened or was not real		0.45	1.09	2.90	8.10
	A7: I stayed away from reminders of it		0.76	1.44	1.96	2.68
	A8: I tried not to think about it		1.29	1.75	1.09	−0.25
Intrusion		0.90
	I1: I had waves of strong feelings about it		1.36	1.65	0.94	−0.39
	I2: Other things kept making me think about it		1.05	1.44	1.17	0.20
	I3: I thought about it when I did not mean to		1.31	1.61	0.97	−0.31
	I4: Pictures about it popped into my mind		1.08	1.44	1.22	0.44
	I5: Any reminder brought back feelings about it		1.07	1.58	1.34	0.53
	I6: I had trouble falling asleep or staying asleep because of pictures or thoughts about it that came into my mind		1.09	1.53	1.17	0.11
	I7: I had dreams about it		0.54	1.07	2.30	4.98

Abbreviations: M, mean; SD, standard deviations; S, Skewness; K, Kurtosis.

Discussion

The purpose of the study was to investigate the psychometric properties of the IES in a sample of Australian paramedics. The hypothesis that the best-fitting solution for structure and dimensionality was a two-factor solution was supported. The factor structure of the IES was investigated using Maximum Likelihood Factor Analysis with Varimax rotation, followed by an evaluation of model fit using Procrustes rotation. The overall factor structure corresponded to that originally proposed by Horowitz et al,¹ who interpreted the separation of the two factors as indicating the existence of the two phases of psychological intrusion and avoidance which occur after the experience of a traumatic event. Many subsequent validity studies also supported a two-factor structure;^{29,30,33,37,54–58} however, a 12-item model provided a better fit to the data than the original 15-item model. The items “I felt as if it hadn’t happened or wasn’t real” (labeled A6); “My feelings about it were kind of numb” (labeled A5) and “I thought about it when I didn’t mean to” (labeled I3) were removed from the final model. Factor loadings were high (see Table 4). The final model and subscales also demonstrated internal reliability better than that first published by Horowitz et al.

Factor association

Not only do the IES items retained in previous studies vary, but the two-factor structure, which is reported in the current study, is also contentious. While the scree plot test and the eigenvalues of the current study suggest the extraction of a single factor, the moderate to strong correlations that have been reported between the factors in the previous studies^{1,33,54,58–66} have been used by Solomon et al⁶⁵ in their study of the IES with combat veterans also suggest that the overall scale could comprise one factor. The tendency toward lower discriminant validity between the factors may also be indicative of sampling bias – the greater the proportion of the sample who is not reporting experiencing trauma symptoms, the higher the correlation between the factors.

Furthermore, although Andrews et al⁶⁷ reported the IES as being multidimensional in their confirmatory factor analysis, they argued that a multidimensional factor structure could be time dependent. Specifically, Andrews et al postulated that “the longer the time elapsed since the trauma, the more symptoms (of subscales) differentiate”.⁶⁷ However, the researchers provided little supporting evidence for such an assertion other than the aforementioned observation of the moderate to high correlations in previous studies. Instead, Williams et al⁶⁶ argue the reverse. That is, immediately following trauma, intrusion and avoidance may be dissociated but increasingly become associated over time.

Evidence in support of this increasing factor association can be found by analyzing the conditions under which the scale is administered as well as symptom severity. Specifically, the type of traumatic event that has been experienced, the time period between the experiencing of the traumatic event and the completion of the scale and even a total lack of traumatic experience are argued to affect factor structure. Although the robust validation of the IES includes its administration across these varying conditions, it is important to remember that the IES was originally designed to be administered within 7 days to participants who had experienced a traumatic event. Consequently, such administration conditions need to be distinguished to account for any potential confounding effects.

Limitations

The study has a number of limitations. The effect of low-level scoring is important in the current study as responses of participants who had not experienced traumatic events were included. That is, it is predicted that these particular participants recorded very low responses, which would have affected the correlation of the subscales in the same manner that a delayed administration might. However, another limitation of the current study was that these participants were not distinguished according to traumatic event exposure in the scale’s administration and thus testing for the effect of their inclusion was not possible. The inclusion of all participants also provides an explanation for the positive skew in the item responses.

Other methodological limitations of the current study relate to the participants and the instrument itself. Paramedics have been recognized as being particularly difficult to survey in a randomized fashion as they are highly mobile and often selected on the basis of availability on a range of sites at random times.⁸

Future research

Considering the potential for such population differences in our study and that the definition of the traumatic event used was so broad, future research involving factor structure investigation would benefit from the investigation of the effects of specific types and rates of violence on traumatic symptomatology. More investigation into the IES with these groups would need to be conducted to see whether the structural differences occur as a result of the inherent differences in the events experienced or as a result of variability between the occupational groupings themselves.

Conclusion

This research found that when an appropriate confirmatory approach is used, with the exception of three items, the original two-factor structure of the IES in the Australian paramedical sample is evident. The finding that the two-factor solution was statistically superior to the one-factor solution suggests to researchers and practitioners that the primary responses to trauma are reflected by this two-factor structure. Although the complete instrument might previously have been found to provide an explanation for traumatic symptomatology, the revised instrument is more robust as it includes a large proportion of participants who may not have experienced a traumatic event. The current results provide workplace trauma researchers with evidence that the IES is a potentially useful, valid, and psychometrically robust instrument for the screening of traumatic effects of OBV in Australian paramedics.

Acknowledgments

We would like to thank the Australian paramedics for taking the time to complete the questionnaire.

Disclosure

The authors report no conflicts of interest in this work.