Evidence for validity of the epidemic-pandemic impacts inventory (brief healthcare module): Internal structure and association with other variables

Abstract

The COVID-19 pandemic has subjected healthcare workers to enormous stress. Measuring the impact of this public health emergency is essential to developing strategies that can effectively promote resilience and wellness. The Epidemic-Pandemic Impacts Inventory Supplemental Healthcare Module-Brief Version (EPII-SHMb) was developed to measure impacts among occupational cohorts serving on the front lines of healthcare. While this instrument has been utilized in COVID-19 related studies, little is known about its psychometric properties. This study collects evidence for validity of the EPII-SHMb by evaluating its internal structure and how its scores associate with other variables. Physicians and nursing staff across a large New York health system were cross-sectionally surveyed using an online questionnaire between June and November 2020. Exploratory factor analysis resulted in a 3-factor solution, identifying factors Lack of Workplace Safety (7 items), Death/Dying of Patients (3 items), and Lack of Outside Support (2 items). Internal consistency was high overall and within physician/nursing and gender subgroups (Cronbach’s alpha: 0.70 − 0.81). Median scores on Death/Dying of Patients were higher among those who directly cared for COVID-19 patients or worked in COVID-19 hospital units. These results are promising. Additional studies evaluating other dimensions of validity are necessary.

Keywords:

• COVID-19
• epidemic
• healthcare workers
• measurement
• occupational health
• pandemic
• psychometric properties
• stress
• validity
• wellness

The COVID-19 pandemic has had an adverse psychological impact upon healthcare workers (HCWs). Increased symptoms of stress, burnout, anxiety, depression, insomnia and even Posttraumatic Stress Disorder (PTSD) have been reported, often most prominently among those working in direct COVID-19 patient care roles.^1–8 A recent study found that the number of COVID-19 related occupational exposures, regardless of status regarding direct COVID-19 patient care, predicted increased psychological distress.¹ Disaster research has repeatedly indicated that exposures experienced during such times can have a cumulative effect on mental health outcomes, with more exposures leading to poorer outcomes.^9–11 In a sample of physicians across a large New York health system, probable anxiety, depression and PTSD increased with each additional COVID-19 related exposure.¹² A study of pediatric HCWs showed a similar relationship.¹³

To be effective, disaster response must target the causes and consequences of distress as experienced by HCWs. This is not to say that risks that can be addressed immediately should need to wait for assessment, however, consistent with the European Union’s Nine General Principles of Risk Prevention, risks that are not necessarily avoidable, as was often the case during the early phase of the COVID-19 pandemic, need to be assessed.¹⁴ Theoretical frameworks such as the National Academies of Science, Engineering and Medicine’s Systems Model of Clinician Burnout and Professional Well-being as well as Shanafelt et al.’s Model of Executive Leadership and Physician Well-being, identify complex, multi-level constructs that directly impact and mediate healthcare worker risk for stress, burnout and negative mental health impacts.^15,16 These constructs include system levels (eg, frontline care delivery, health care organization, and external environment) that produce work system factors (eg, excessive job demands and inadequate job resources) that interact with individual vulnerabilities to produce clinician burnout and distress which in-turn has a negative impact on patients and the care they receive. These models are not without controversy. For example, some argue that the construct of ‘burnout’ has a great deal of overlap with depression and should not be used.¹⁷

During the COVID-19 pandemic, researchers and healthcare organizations, as part of their overall response, have sought to measure pandemic-related exposures such as occupational safety factors as well as the factors that may mediate exposures such as social support. To address this gap, Grasso and colleagues developed the Epidemic-Pandemic Impacts Inventory (EPII).¹⁸ The EPII main module consists of 92 items assessing impacts of the pandemic across personal and social life domains. It was developed for use with a general adult population; however, various adaptations have been developed that are specific to different populations (eg, adolescents, geriatrics, healthcare workers, and pregnant women). The EPII main module has since been used in numerous studies,^19–27 and is currently maintained in the National Institutes of Health (NIH) Disaster Research Response (DR2) Repository of COVID-19 Research Tools (https://dr2.nlm.nih.gov).

To examine the evidence for validity of the EPII, investigators conducted a Latent Class Analysis (LCA) among adults residing in the Northeast region of the United States to identify cumulative risk profiles that may predict psychosocial risk.²⁸ Several unique subgroups were identified across sociodemographic domains and the degree of perceived stress, depression, anxiety and PTSD. Specifically, they found that three unique profiles (ie, parents with high exposure/high risk; young adults with high exposure/high risk; older adults with moderate exposure/high risk), representing ∼64% of the sample, were characterized by moderate to high exposure to adverse pandemic-related experiences (eg, “continue to work despite close contact with people who may be infected”) and were more likely to screen positive for depression, anxiety and PTSD.²⁹ This finding highlighted the utility of the EPII in assessing the positive and negative COVID-19 specific experiences that are associated with mental health outcomes.²⁹

While the LCA conducted by Grasso et al.²⁹ demonstrated the ability of the EPII as a metric to identify subgroups with meaningful differences in risk, other types of evidence of validity are necessary to justify the use of this measure. Further, no studies to date have examined the adaptation of the original version of the EPII that is currently being used in studies of healthcare workers; namely, the Epidemic-Pandemic Impacts Inventory Supplemental Healthcare Module- Brief Version (EPII-SHMb). Similar to established taxonomies of psychosocial risk, such as those presented by Leka and Cox,³⁰ the domains of the EPII-SHMb evaluate both stressful work and home experiences and how they impact occupational health and wellbeing. Similarly, Gollac and Volkoff³¹ describe taxonomies of risk including physical, chemical, biological and psychological. The EPII-SHMb maps on to some of these domains given its focus on occupational safety issues concerning virus exposure as well as psychological impacts of events, such as the death and dying of patients despite great efforts. This study addresses gaps in healthcare worker pandemic-related occupational exposure assessment by exploring the internal structure and correlation of the scores with other variables (convergent validity) of the supplemental HCW module (brief version) of the EPII (EPII-SHMb).

Methods

Study design

This is a psychometric study that uses data derived from an ongoing prospective longitudinal cohort registry exploring the mental health and well-being of HCWs to explore the evidence for validity of the EPII-SHMb. Consistent with the work of Downing and Kane and current standards in educational and psychological measurement, we used the unitary model of validity, with multiple sources of evidence, including content, response process, internal structure, and relationship with other variables.^32–34 In this study, we focus on the internal structure and relationship with other variables.

Data and participants

This study utilized completely de-identified data from a larger ongoing prospective longitudinal cohort registry of physicians and nursing staff across a large, 23-hospital New York health system. The registry was created to assess healthcare workers personal and occupational well-being as it relates to the COVID-19 pandemic. The registry consists of a baseline assessment and ongoing prospective data collections to evaluate changes over time. However, for the purposes of this study, we only analyzed data from baseline assessments. The assessments utilized in the registry were developed through an iterative process that was expert reviewed by a multidisciplinary team of clinicians and researchers. This study was approved by our institution’s Institutional Review Board under an umbrella approval that was granted for the preexisting registry (Northwell Health IRB #20-0510). All participants were provided written informed consent to participate in the registry. The registry includes participants’ responses to questions about demographic characteristics, pandemic exposures (including the EPII-SHMb), depression, anxiety, PTSD, burnout, individual resilience, organizational support, use of organizational well-being resources, and redeployment- and trainee-status. Details on the governance, recruitment, and data collection have been described previously.¹²

Physicians and nursing staff (i.e., nurses and non-RN nursing staff such as Nursing Assistants) were invited to participate. Participants were eligible for study inclusion if they were a physician or nursing staff member who was actively employed by or affiliated with the health system, and able to electronically consent for participation. Self-report baseline assessments were electronically distributed separately to physicians between June and August 2020, and to nursing staff between September and November 2020. With one exception, only data from questions that were identical between the two surveys were used for this study. Participants were not incentivized to participate. The total number of HCWs who provided baseline assessments were N = 1,491, equating to a response rate of 4.9% and 4.4% for physicians and nursing staff, respectively.

Measures

The Epidemic – Pandemic Impacts Inventory (EPII) Supplemental Healthcare Module – Brief Version (EPII-SHMb) is a 25-item module which was developed to measure exposure to the coronavirus pandemic across personal and social domains among HCWs.³⁵ Items 1-16 ask about exposures to negative experiences during the pandemic with three response choices (‘Yes’, ‘No’, and ‘N/A’). Items 17-25 have responses ranging from ‘Definitely’, ‘Somewhat’, ‘Not at all’, and ‘N/A’. For this study, the factor structure of items 1-16 was considered as they involve direct HCW impacts/experiences, as opposed to questions 17-25 which query emotional experiences as a result of stressors that are more subjective in nature. An example of one of the first 16 questions that is used in the current analysis is: “Have you experienced the following since the beginning of the coronavirus disease pandemic? ‘Inadequate/unhygienic personal protective equipment (PPE)’” Please see Supplemental Table 1 for the full measure. Prior studies using the EPII collapsed “N/A” and “No” into a single “No exposure” category, thereby creating a dichotomous indicator (yes versus no). We followed this practice both to be consistent with scoring in prior studies and also because in this context both ‘N/A’ and ‘No’ represent the same condition of no exposure. The EPII-SHMb has been used in previous studies as a continuous score in which each item is summed with equal weight to have a total score from 0-16.^6,12 Psychometric properties of this healthcare worker version have not been previously examined.

Table 1. Sample characteristics (N = 1,491).

	Total		Physicians		Nursing staff
	n	%	n	%	n	%
Age, years, mean (SD)	45.55	12.89	46.58	13.15	44.81	12.64
Gender
Female	1092	73.7	300	48.8	792	91.5
Male	379	25.6	313	50.9	66	7.6
Non-binary	2	0.1	0	0.0	2	0.2
Prefer not to answer	8	0.5	2	0.3	6	0.7
Race
White only	1087	73.7	434	71.0	653	75.6
Asian only	180	12.2	116	19.0	64	7.4
Black/ African American	93	6.3	18	2.9	75	8.7
Other/Multiple races	115	7.8	43	7.0	72	8.3
Hispanic ethnicity
No	1285	87.6	542	89.3	743	86.4
Yes	182	12.4	65	10.7	117	13.6
Partner status
No (single/separated/divorced/widowed)	461	31.0	144	23.4	317	36.4
Other	18	1.2	7	1.1	11	1.3
Yes (married/engaged)	1006	67.7	464	75.4	542	62.3
Between March 2020 and May 2020, did you work in a COVID-19 hospital unit (Emergency Department, COVID-19 medical floor or COVID-19 ICU)?
No	599	40.2	298	48.1	301	34.6
Yes	892	59.8	322	51.9	570	65.4
Between March 2020 and May 2020, were you deployed to work in a clinical setting that was different from your typical clinical setting/area of specialty?
No	1003	67.7	441	71.1	562	65.3
Yes	478	32.3	179	28.9	299	34.7
Did you directly care for patients with COVID-19 or suspected of having COVID-19?
No	313	21.0	126	20.3	187	21.5
Yes	1178	79.0	494	79.7	684	78.5
During the COVID-19 pandemic, did you choose to live away from your family members to protect them from exposure for any length of time? (i.e., hotel or friend's apartment?)
No	1234	83.3	530	85.8	704	81.6
Yes	247	16.7	88	14.2	159	18.4
Did you use any other ancillary services provided by the hospital that you did not use before? (i.e., housing, scrub machines, parking)
No	1081	73.2	480	78.3	601	69.6
Yes	396	26.8	133	21.7	263	30.4
On a scale of 1–5, how often did you feel supported at work?
1 Not supported	84	5.7	16	2.6	68	7.8
2 Somewhat not supported	121	8.2	26	4.2	95	10.9
3 Neutral	227	15.3	93	15.2	134	15.4
4 Somewhat supported	508	34.3	219	35.8	289	33.2
5 Completely supported	542	36.6	258	42.2	284	32.6
On a scale of 1–5, how did you feel about your clinical workload between March 2020 and May 2020 as compared to your typical workload?
1 Much lighter workload	75	5.1	63	10.3	12	1.4
2 Lighter workload	202	13.6	163	26.5	39	4.5
3 Similar workload	323	21.8	192	31.3	131	15.1
4 Heavier workload	370	24.9	140	22.8	230	26.5
5 Much heavier workload	430	29.0	56	9.1	374	43
6 N/A	83	5.6	0	0.0	83	9.6
I am satisfied with the facilities and resources available to me at work to cope during the pandemic*
Strongly disagree	8	1.7	8	1.7	–	–
Disagree	20	4.1	20	4.1	–	–
Somewhat disagree	20	4.1	20	4.1	–	–
Undecided/Neutral	50	10.3	50	10.3	–	–
Somewhat agree	64	13.2	64	13.2	–	–
Agree	180	37.2	180	37.2	–	–
Strongly agree	106	21.9	106	21.9	–	–
N/A	36	7.4	36	7.4	–	–

* This question was asked among attending physicians only.

In addition to the participants’ responses to the EPII-SHMb, we included several other study variables. These variables allowed us to examine whether the factor structure was stable across sub-groups and how the factor scores associated with other variables. These variables included demographics (age, gender, race, ethnicity, partner status, primary workplace location, COVID-19 patient care provision) and measures of workload, perceived workplace support, use of ancillary services, physical distancing from family, and COVID-19 workplace exposure. Workload was measured by the question: On a scale of 1-5, how did you feel about your clinical workload between March 2020 and May 2020 as compared to your typical workload? Feelings of perceived support at work and use of ancillary support were measured by the questions: On a scale of 1-5, how often did you feel supported at work?, and Did you use any other ancillary services provided by the hospital that you did not use before? (i.e., housing, scrub machines, parking). Family distancing was measured as: During the COVID-19 pandemic, did you choose to live away from your family members to protect them from exposure for any length of time (i.e., hotel or friend's apartment)? Direct COVID-19 care and working on a COVID-19 medical floor, were measured by the questions: Did you directly care for patients with COVID-19 or suspected of having COVID-19? And Between March 2020 and May 2020, did you work in a COVID-19 hospital unit (choice = Emergency Department, COVID-19 medical floor or COVID-19 ICU), respectively. For attending physicians, satisfaction with resources was measured as: I am satisfied with the facilities and resources available to me at work to cope during the pandemic.

Statistical analysis

Internal structure

Bartlett’s test for sphericity indicated that factor analysis was appropriate. The EFA was performed using a tetrachoric correlation matrix, which is appropriate for measuring correlations between binary variables. Only survey respondents with complete data on all items were used. The EFA was initially performed on all 16 items to construct models with 2, 3, and 4 factors using a rotated factor pattern. If a factor loading for an item was less than 0.40 for all factors, the item was removed, and the analysis was repeated. The number of factors chosen were based on several criteria: Kaiser criterion (eigenvalue > 1), factors with at least two items with loadings greater than 0.40 and model fit (root mean square of approximation less than 0.08). The study team discussed and labeled each factor. Internal consistency was assessed with Cronbach’s alpha.

Relationship with other variables

The relationship between each factor and other measures hypothesized to be associated with each construct was examined to collect evidence for validity regarding how the instrument’s scores associate with external variables. To compare each factor to continuous and categorical measures, Spearman’s rank correlations and Wilcoxon-Two Sample tests were performed, respectively. All analyses were performed using SAS software, Version 9.4 (Copyright © 2021 SAS Institute Inc).

Results

Description of sample

The study sample included 1,491 healthcare workers (620 physicians and 871 nursing staff). Participants were, on average, 45.55 (SD = 12.89) years old, and the majority were female (73.7%), White (73.7%), and of non-Hispanic ethnicity (87.6%) (Table 1). There were more females among nurses compared to physicians (91.5% vs. 48.8%). The majority reported working on a COVID-19 hospital unit (59.8%) and working directly with COVID-19 patients (79.0%). Participants endorsed individual items of the EPII-SHMb differentially, with Being at risk of contracting COVID-19 virus from patients or coworkers (89.5%), Contact with distressed family members who cannot be with a loved one (74.2%), Deaths of patients despite heroic efforts by the treatment team (66.7%), and Comforting family members whose loved one is dying or has died (64.6%) being the most frequently endorsed items (Supplemental Table 2). In general, nursing participants endorsed items more frequently than physicians.

Table 2. Factor loadings for items from the The Epidemic – Pandemic Impacts Inventory (EPII) Supplemental Healthcare Module – Brief Version (EPII-SHMb) among Physicians and Nursing Staff (N = 1,491).

	Factors
EPII Item	Workplace Safety (7 items)	Death/Dying of Patients (3 items)	Lack of Outside Support (2 items)
1. Being at risk of contracting COVID-19 virus from patients or coworkers.	0.47	0.13	0.14
2. Inadequate/unhygienic personal protective equipment (PPE).	0.95	−0.10	−0.03
3. Insufficient staffing or equipment to properly care for COVID-19 patients.	0.77	0.12	0.04
4. Insufficient/unavailable viral infection testing kits.	0.58	0.00	−0.08
8. Being in close contact with patients without adequate PPE.	0.83	−0.03	−0.03
9. Having no break from disruptive noise and wearing uncomfortable PPE.	0.48	0.28	0.17
13. Insufficient support from workplace supervisors or administrators.	0.64	−0.04	0.09
5. Deaths of patients despite heroic efforts by the treatment team.	0.10	0.80	−0.05
6. Contact with distressed family members who cannot be with a loved one.	−0.01	0.89	0.07
7. Comforting family members whose loved one is dying or has died.	−0.08	0.96	−0.03
14. Family and friends don't understand the danger you face at work.	0.004	−0.03	0.94
15. Family and friends don't understand the emotional and physical exhaustion caused by your work.	0.001	−0.003	0.94
10. Coworkers treat you or each other with irritability, impatience, or disrespect.
11. Stigma from others because of your (actual or perceived) coronavirus exposure as a healthcare professional/worker (treated like we're lepers).
12. Illness and uncertain recovery, or deaths, of coworkers.
16. Forced separation from your children or spouse/partner for a week or more due to work or self-quarantine.

Note: The bold numbers reflect the items that were identified has having high factor loadings (>0.40) for the 3 factor solution from the factor analysis.

Internal structure

Factor analysis

The EFA was initially performed on all 16 items, with specification for 2, 3 and 4 factors. For each of these solutions, 4 items (Items 10, 11, 12, and 16) had low factor loadings (<0.40). EFA was sequentially performed again, removing items with the lowest factor loadings one at a time, and solutions were reevaluated. The final solution, which met all the a priori fit criteria, was the 3-factor solution with removal of Items 10, 11, 12, and 16. Each of the three factors had eigen values above 1.0 and at least two items with loadings greater than 4.0. The three-factor model demonstrated good fit (RMSEA = 0.027). The 3-factor solution was deemed interpretable, and the factors were labeled Lack of Workplace Safety (7 items), Death/Dying of Patients (3 items), and Lack of Outside Support (2 items) (Table 2).

Internal consistency

Internal consistency was acceptable (Cronbach’s alpha range: 0.70 − 0.81) overall and within subgroups (i.e., physicians versus nurses and male versus female; Supplemental Table 3). There appeared to be slightly lower agreement among physicians and males regarding Workplace Safety items.

Table 3. Association between factors and study variables.

Factor	External variable	Median	Rho^c	p
Lack of Workplace Safety (Factor 1)	Clinical workload		0.19	<.0001^a
	Much lighter workload	−0.59
	Lighter workload	−0.46
	Similar workload	−0.47
	Heavier workload	−0.20
	Much heavier workload	0.50
	Supported at work		−0.49	<.0001^a
	Not supported	1.02
	Somewhat not supported	0.97
	Neutral	0.37
	Somewhat supported	−0.01
	Completely supported	−0.71
	Ancillary services		–	<.0001^b
	No	−0.33
	Yes	0.32
	Distanced from family		–	<.0001^b
	No	−0.27
	Yes	0.39
Death/Dying of Patients (Factor 2)	Direct COVID-19 patient care		–	<.0001^b
	No	−0.86
	Yes	0.60
	Worked on COVID-19 hospital unit		–	<.0001^b
	No	−0.72
	Yes	0.68
Lack of Support (Factor 3)	N/A		N/A	N/A

^a p based on Spearman’s correlation coefficient.

^b p based on Wilcoxon Two-Sample Test.

^c Rho represents Spearman’s correlation coefficient.

Relationship with other variables

Lack of Workplace Safety (Factor 1)

The factor score was weakly positively associated with heavier workload (Spearman’s correlation coefficient r = 0.19, p <.0001) and moderately negatively associated with feeling supported at work (r= −0.49, p <.0001) (Table 3). Participants who used ancillary services (Median: 0.32 vs. −0.33, p <.0001) and those who lived away from family (Median: 0.39 vs. −0.27, p <.0001) had higher median lack of workplace safety factor scores as compared to those who did not use services and who lived closer to family, respectively.

Death/Dying of Patients (Factor 2)

Participants who directly cared for COVID-19 patients (Median: 0.60 vs. -0.86, p <.0001) or worked in COVID-19 hospital units (Median: 0.68 vs. −0.72, p <.0001) had higher median Factor 2 (Death/Dying of Patients) scores as compared to those who did not directly care for COVID-19 patients or those who did not work in COVID-19 hospital units, respectively (Table 3). Among physicians only, satisfaction with coping facilities and resources was not significantly associated with Factor 2, even when stratified by those who directly cared for COVID-19 patients (data not in tabular format).

This analysis was not performed for Factor 3 (Lack of Outside Support) because none of the variables collected were hypothesized to correlate with this construct.

Discussion

The EPII-SHMb was developed to measure the impacts of a pandemic upon healthcare workers. This instrument has been used in a number of COVID-19 related studies. Yet, little is known about its psychometric properties. The current study is the first, to date, to examine the psychometric properties of the EPII brief HCW measure (EPII-SHMb) in a large and diverse physician and nursing sample. The EPII-SHMb holds promise. The results from the EFA and various Cronbach alphas support the internal structure. The factor scores for lack of workplace safety and patient death/dying were associated as expected with other variables. Taken together, these findings provide initial evidence for validity of the EPII-SHMb.

When a crisis such as the pandemic occurs, HCWs experience significant stress. Stress may result from a number of causes, including threat to the worker’s personal or family health (life threat), the loss of colleagues, patients, or professional mastery/identity (loss), an inner conflict between one’s value and aspirations and what they are able to accomplish in their work (inner conflict), and/or fatigue without time for rest and recovery (wear and tear).³⁶ The reaction to this stress may vary from a minor reaction (eg, feeling irritable, difficulty sleeping) to a serious stress injury (eg, excessive guilt, shame or blame, panic, dysthymia) and, if left untreated, the acute stress reaction can evolve into persistent impairment due to depression, anxiety, substance abuse, or PTSD.³⁷ The negative impacts of the stress can affect the quality of the care that an impacted healthcare worker delivers to patients.³⁸

The internal structure of the EPII-SHMb is consistent with contemporary models of occupational safety and health as well as stress and coping. For example, National Academy of Science’s Systems Model of Clinician Burnout and Professional Well-being identifies work system factors (e.g., job demands, moral distress, meaning and purpose in work, alignment of values) that interact with individual factors (e.g. coping strategies, social support) to influence clinician burnout and wellness.¹⁵ The EPII-SHMb factors related to workplace safety, such as inadequate staffing and having no break from noise, real or perceived lack of Personal Protective Equipment (PPE), and death and dying of patients map onto work system factors explained by the above model. The EPII-SHMb factor of outside support and lack of familial/friend understanding, maps onto individual mediating factors of personal relationships and social support. Similarly, Shanafelt and Noseworthy¹⁶ outline drivers of burnout and engagement which conceptually overlap with the three factors identified in this study. This includes their constructs of efficiency/resources, workload/job demands, and social support.¹⁶ The concordance between the EPII-SHMb factors and current models of occupational health, provides further support for the validity of the EPII-SHMb. Other, more traditional models of occupational stress are also worth considering, such as Karesek and Theorell’s model of Job Demand/Control.³⁹ We can draw some parallels with this model, particularly in regard to ‘high demand/burden’ correlating with the two factors identified in our study, ‘death and dying of patients’ and ‘lack of workplace safety’ and the more recent incorporation of support into Karasek’s model maps on to our factor of ‘lack of outside support’.³⁹

These findings have implications for how organizations can mitigate stress during a pandemic. Interventions should focus on optimizing workplace safety, helping clinicians emotionally process moral distress of death/dying under conditions when care is not optimal, and augmenting outside mental health support resources. Shanafelt and Noseworthy¹⁶ developed a conceptual model for how healthcare leaders can create a resilient organization utilizing interventions before, during and after the crisis. In the context of the pandemic, these interventions would include prioritizing the provision of workplace safety items such as adequate PPE, hydration, food, administrative burden relief, and regular communications, the most basic intervention approach as outlined by the National Institute for Occupational Safety and Health’s (NIOSH) Hierarchy of Controls.⁴⁰ While many of these provisions are required by law, the pandemic exposed significant problems with the processes in place to ensure provision. From the authors’ personal experience, there were substantial variability between hospital and health systems in the U.S. in terms of how effectively they communicated with team members, provided administrative relief, and addressed PPE. These are significant learning points in order to better prepare for future pandemics and public health crises.^41,42 To address the death/dying of patients, peer support, mental health support, and training around grief reactions could be integrated. Models, such as Stress First Aid, which focus on increasing the ability to recognize stress in coworkers and knowing how to link affected coworkers into resources, are becoming more widely used, including in healthcare settings, shifting the culture around recognizing and responding to stress and other mental health care needs among coworkers, which is a higher level in NIOSH’s Hierarchy of Controls, and therefore potentially more effective at mitigating stress.^43,44 Organizations may have less control over outside support, but providing childcare, lodging, and transportation can contribute to reducing symptoms of stress and burnout and enhancing healthcare worker resilience.

Limitations

While these findings provide support for the use of the EPII-SHMb brief healthcare worker scale, additional studies are necessary. The three-factor internal structure needs to be replicated in other HCW populations (e.g., disciplines other than physicians and nurses, such as respiratory therapists and technicians and geographic regions other than New York City) and settings (e.g., healthcare workers in private practice or community-based clinics). Moreover, this study did not examine to what extent the internal structure is stable across healthcare worker race and ethnicity, an important feature of validity in general and especially given the disparities in the pandemic impacts on healthcare workers.⁴⁵ In addition, this current study did not examine other important dimensions of validity such as response process, relationships with a broader set of external variables, and consequences. Future studies could also perform confirmatory factor analyses. Although this study was only able to examine how the factor scores related to several other variables, recent studies have provided compelling support for how the overall EPII-SHMb scores correlate with depression, anxiety and PTSD.¹² For example, a one-point increase in the EPII-SHMb score was associated with a 23% increase in probable anxiety and probable depression and a 41% increase in probably PTSD. This suggests that the EPII-SHMb is capturing aspects of the healthcare worker stress exposure that have direct and significant impacts on the nature of their stress reaction.

Additional limitations include the low response rate of nursing staff and physicians. Those who responded may not be representative of the broader population. Although the number of HCWs who agreed to join the longitudinal registry was low compared to the total amount of HCWs who were initially invited, the overall sample size of N = 1,491 was larger than previous studies which examined the EPII.^28,29 Second, the surveys were given at different times for physicians and nursing staff (ie, the physician cohort survey started several months before the nursing cohort) due to researcher bandwidth and logistical issues. While all the participants were asked to report on experiences during the same time period (March–May 2020) during their baseline assessments regardless of when the survey was administered, recall bias may have influenced the participants responses. Third, participants came from different hospitals within the same health system and may have therefore experienced varying levels of exposure or resources for support, though this diversity improves generalizability of study findings. Fourth, the data set did not contain variables that we would hypothesize to be associated with Factor 3, Lack of Outside Support, which limited the evaluation of this factor.

Conclusion

This study provides evidence of validity with respect to the internal structure of the EPII-SHMb and how the factor scores associate with other variables. Combined with recent studies, there is growing evidence to support the psychometric properties of the EPII-SHMb. If replicated, the EPII-SHMb can provide an effective way to continue to measure the impacts of COVID-19 on healthcare workers (as well as any potential additional epidemics) and can inform the development and evaluation of interventions.

Acknowledgments

The authors would like to thank the Northwell Health COVID-19 Research Consortium for their assistance in facilitating this study.

Declaration of interest

We have no conflicts of interest to report.

Funding

The author(s) reported there is no funding associated with the work featured in this article.