Effect of Task Load Interventions on Fatigue in Emergency Medical Services Personnel and Other Shift Workers: A Systematic Review

Abstract

Background: Modifying the task load of Emergency Medical Services (EMS) personnel may mitigate fatigue, sleep quality and fatigue related risks. A review of the literature addressing task load interventions may benefit EMS administrators as they craft policies related to mitigating fatigue. We conducted a systematic review of the peer-reviewed literature to address the following question: “In EMS personnel, do task load interventions mitigate fatigue, mitigate fatigue-related risks, and/or improve sleep?” (PROSPERO 2016:CRD42016040114). Methods: We performed a systematic review of the literature that described use of randomized controlled trials, quasi-experimental studies, and observational study designs. We retained and reviewed research that involved EMS personnel or similar shift worker groups 18 years of age and older. Studies of ‘healthy volunteers’ and non-shift worker populations were excluded. Studies were included where the methodology of the study implied a theoretical framework of task load (or workload) affecting fatigue, and then fatigue related outcomes. Outcomes of interest included personnel safety, patient safety, personnel performance, acute fatigue, and cost to system. We used the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) methodology to summarize findings and assess quality of evidence from very low to high quality. Results: The search strategy yielded 3,394 unique records resulting in 58 records included as potentially eligible. An additional 69 studies were reviewed in full following searches of bibliographies. We detected wide variation in the description and measurement of task load in the retained and excluded research. Among 127 potentially relevant studies reviewed in full, five were judged eligible. None of the retained studies reported findings germane to personnel safety, patient safety, or cost to system. We judged most studies to have serious or very serious risk of bias. Conclusions: The effect of task load interventions on fatigue, fatigue-related risks, and/or sleep quality was not estimable and the overall quality of evidence was judged low or very low. There was considerable heterogeneity in how task load was defined and measured.

Key words::

• task load
• fatigue
• Emergency Medical Services

Background

Fatigue in the Emergency Medical Services (EMS) workplace may be related to high patient care loads, demanding work schedules, and associated stress (1–3). Higher annual EMS patient load has been linked to lower levels of safety climate, teamwork climate, perceptions of management, job satisfaction, and poorer perceptions of working conditions (4). Task load is defined as the perceived difficulty in accomplishing a task or subjective mental workload. In EMS, task load can vary greatly by shift and by EMS system. For example, navigation to and from emergency scenes may have a higher perceived task load for an individual that operates in an organization that utilizes system status management, requiring a breadth of geographic knowledge compared to an individual that consistently responds from a fixed post location to the same geographic area. Conceptually, task load can affect perceived fatigue while on the job and impact sleep quality during or after a shift. As perceived fatigue increases and is potentiated by poor sleep quality, the likelihood fatigue-related risks occurring is increased. Fatigue-related risks in the EMS setting could include patient care errors, decision-making errors, or distracted driving.

Modifying the task load of EMS personnel may mitigate fatigue, sleep quality, and fatigue related risks. Task load modification might include reducing cognitive burden by limiting multi-tasking, increasing the automation of tasks, or decreasing the overall workload. Our understanding of the relationships between task load, fatigue, and outcomes of importance to EMS operations is limited. However, it is likely that interventions to reduce task load that have successfully mitigated fatigue in other professions may translate to EMS. A broad review of the published literature addressing task load interventions may benefit EMS administration by summarizing the evidence in favor of or against use of task load interventions to mitigate fatigue and fatigue related risks in the EMS environment.

We conducted a systematic review of the peer-reviewed literature to address the following question (PROSPERO 2016: CRD42016040114): “In EMS personnel, do task load interventions mitigate fatigue, mitigate fatigue-related risks, and/or improve sleep?” (5).

Methods

We conducted a systematic review of multiple databases. The details of our methodology, study protocol, and procedures for reviewing published and unpublished literature are described in a separate publication (6). In this paper, we provide a brief summary of the study protocol unique to this systematic review.

Study Design

We assessed journal publications that described use of randomized controlled trials, quasi-experimental studies (7), and observational study designs.

Types of Participants

The description of our target population was developed a priori by a panel of experts to be inclusive of EMS personnel and related shift worker groups: “EMS personnel or similar worker groups, defined as shift workers whose job activity requires multiple episodes of intense concentration and attention to detail per shift, with serious adverse consequences potentially resulting from lapses in concentration (5). Studies that involved “healthy volunteers” and non-shift worker populations were excluded.

Types of Interventions

We retained journal articles that reported studies testing or evaluating task load or workload interventions in the operational setting to address fatigue and fatigue-related risks. There is no gold standard measure of task load. For the purposes of this systematic review, we retained literature that included a measure of task load that was defined as an individual's reported difficulty in accomplishing a task or perceived mental workload. This measurement could occur at a single point in time or as a repeated measure.

Types of Outcome Measures

A panel of experts selected outcomes of interest a priori, which include personnel safety, patient safety, personnel performance, acute fatigue, and cost to the system (5).

Search Methods for Studies

We describe the details of our methodology and search strategies in a separate publication (6). In that paper, we identify all databases searched, terms, and a description of vocabulary. Each search for this systematic review incorporated multiple terms covering three concepts: emergency medical services and other critical shift-based occupations; fatigue, sleep, and sleep disorders; task load and workload. All searches included literature from January 1980 to September 2016. See Online Supplemental Material for the details of our search strategy.

Data Collection and Selection of Studies

Screening

Co-investigators (PJC and JPC) independently screened titles and abstracts of search results to identify studies potentially germane to the study objectives. Two additional investigators (PDP and DJS) adjudicated disagreements against inclusion/exclusion criteria: a) the study included the population of interest; b) the title and/or abstract describe one or more outcomes of interest; and, c) the study reports findings stratified by task load (or workload). The Kappa statistic was used to determine inter-rater agreement during screening.

Full-Text Review

Eight investigators (MLR, JPC, KLF, AJK, XX, AAD, ZB, and PDP) worked independently to abstract key information from full-text articles. Key information abstracted included: study design, participant characteristics, intervention characteristics, comparisons, outcome measures, and key findings. Three investigators (MLR, KLF, and AAD) verified data abstractions and disagreements were adjudicated with discussion. We excluded book chapters, conference abstracts, newsletters, dissertations, and theses. Bibliographies of retained literature were searched to identify additional relevant research.

Risk of Bias Assessment

Our team's three senior^{Figure 1} investigators (JRS, AEI, and PDP) used the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) template to assess risk of bias across multiple domains (eligibility criteria, matching, selection, measurement, outcome surveillance, control of confounding, lack of statistical adjustment, and incomplete follow up) (8). Disagreements were resolved through discussion and consensus.

Statistical Analysis

Three investigators (JRS, AEI, and PDP) used the protocol for categorizing findings as favorable, unfavorable, mixed/inconclusive, or no impact described in a separate publication (6). A favorable designation was assigned when co-investigators determined findings favored lower levels of task load. Unfavorable was assigned when findings were unfavorable with lower task load. Four investigators (JRS, AEI, PDP, and ESL) used the GRADE evidence profile tables to summarize and rate the quality of retained research (evidence) (8–10).

Quality of Evidence

Three investigators (JRS, AEI, and PDP) used the GRADE prescribed evidence profile tables to summarize and rate the quality of retained research (6, 10).

Reporting

Our findings are presented in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) (11).

Results

The search strategy yielded 3,394 unique records of which eight duplicates were removed (Figure 1). The remaining titles and abstracts were independently screened by 2 investigators with moderate inter-rater agreement (Kappa = 0.66), resulting in 58^{Table 1} records included as potentially eligible. An additional 70 journal articles were identified during the search of bibliographies, labeled potentially relevant, and reviewed in full-text format. Five non-experimental (prospective observational) studies were retained and judged relevant to our research question (See Online Supplemental Material) (12–19). A total of 123 studies were excluded with reasons given, organized in the Population, Intervention, Comparison, Outcome (PICO) format (See Online Supplemental Material). Following additional review and consensus, five journal articles describing five studies were included in this systematic review. We summarize an assessment of bias for each of the five retained studies in Online Supplemental Material.

Figure 1. PRISMA 2009 Flow Diagram for PICO#7 PROSPERO 2016:CRD42016040114.

Karhula et al. stratified workers into high job strain and low job strain, which we interpreted as one of two techniques used in this study to measure task load (12). A second technique used by Karhula and colleagues was a single survey item that evaluated perceived mental task load and a single survey item to measure physical workload during each shift (12). Baulk et al. used the NASA-TLX tool to evaluate subjective task load during the first half and second half of scheduled shifts (13). Dorrian et al. used the NASA-TLX tool at the mid-point and end of shifts (14). Takahashi et al. used the National Institute for Occupational Safety and Health Generic Job Stress Questionnaire to measure quantitative task load (four items) and variance in workload (three items) (18). Grech et al. measured workload with a single item from the Crew Status Survey (CSS) that measures average workload on a 7-point scale (19).

Impact of Task Load on Personnel Performance Outcomes

Baulk et al. used a prospective cohort study to measure task load with the NASA-TLX tool during the latter half of shifts, and the psychomotor vigilance test (PVT) at the start, mid-point, and end of each shift to evaluate performance (13). Performance was better (faster reaction time) during low task load periods on the first night of a 2-night shift sequence, but not the second night shift and not for either of the 2-day shifts. PVT lapses were less frequent during lower task load periods for the first day shift, but not second day shift or for any of the night shifts. We categorized these findings as favorable for performance during periods of lower task load (Table 1).

Table 1. Synthesis of findings on the impact of task load interventions on critical and important outcomes

OBSERVATIONAL STUDIES
			Important Outcomes
Author, Year	RefID PMID#	Study Design	Personnel Safety	Patient Safety^*	Personnel Performance^†	Acute Fatigue^‡	Cost to System
Karhula et al., 2013 (12)	RefID-1399	Prospective cohort study design	—	—	—	Favorable	—
	PMID-24079918
Baulk et al., 2007 (13)	RefID-2396	Prospective cohort study design	—	—	Favorable	Favorable	—
	PMID-n/a
Takahashi et al., 2006 (18)	RefID-2085	Observational study (panelstudy design)	—	—	—	No Impact	—
	PMID-17190723
Dorrian et al., 2011 (14)	RefID-1057	Prospective cohort study design	—	—	—	Favorable	—
	PMID-20691425
Grech et al., 2009 (19)	RefID-1228	Prospective cohort study design	—	—	—	Mixed/ Inconclusive	—
	PMID-19586219

Findings are classified as favorable in support of lower levels of task load (or workload), unfavorable, mixed/inconclusive, or no impact for mitigating fatigue, mitigating fatigue-related risk, and/or improving sleep.

^* Includes quality of care.

^† Includes external subjective ratings of the study subject's performance including perceived satisfaction with the subject's performance.

^‡ Includes acute states of fatigue, sleepiness, and alertness.

Impact of Task Load on Acute Fatigue Outcomes

Karhula et al. determined that nurses classified as having high job strain were more likely to report being tired when performing a quick return (a morning shift after an evening shift), and had worse subjective recovery between shifts (12). Grech et al. showed that perceptions of fatigue were highest during periods of both low and high levels of task load (19). We categorized the findings of Karhula et al. and Grech et al. as favorable toward lower task load conditions for fatigue mitigation (Table 1) (12, 19). Baulk et al. determined elevated fatigue levels were associated with increased task load at the end of a 12-hour night shift (13). Dorrian et al. showed that the higher a shift worker rated their workload during shifts, the greater the odds of extreme tiredness or exhaustion (odds ratio 1.2; 95% CI 1.04, 1.38) (14). We categorized the findings by Baulk et al. and Dorrian et al. as favorable toward lower task load for fatigue mitigation (Table 1) (13, 14). Takahashi et al. showed no relationship between task load of shift workers and acute fatigue, as measured by the Epworth Sleepiness Scale (18). We categorized these findings as task load having no impact on acute fatigue. None of the retained research addressed personnel safety, patient safety, or cost to systems.

Quality of Evidence

We judged most studies to have serious or very serious risk of bias (Table 2). We downgraded for small sample sizes and indirectness of evidence involving shift workers other than EMS personnel. We downgraded for inconsistency of findings related to one of the 2 outcomes (indicators of acute fatigue). We judged the overall quality of the evidence as low or very low for important outcomes. We detected wide variation in the description and measurement of task load in the retained and excluded research. We present the definitions and/or descriptions of task load/workload used in these studies in Online Supplemental Material.

Table 2. GRADE evidence profile table

Quality assessment
№ of studies	Study design	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	Impact / Effect	Quality	Importance
Personnel Performance (as measured by psychomotor vigilance test [PVT] reaction time and lapses)
1	Observational studies	Very serious^*,†,‡,§,║	Not serious	Serious^#	Serious^**	None	Pooled Effect Not Estimable: One prospective observational study (Baulk et al., 2007) reported better performance on the PVT during low workload periods for one of 2 observed shifts. Findings characterized as favorable towards lower workload to mitigate fatigue and/or fatigue-related risk. [Total sample size n = 20]	⊕◯◯◯ VERY LOW	IMPORTANT
Acute Fatigue (as measured by perceived sleepiness and fatigue instruments)
5	Observational studies	Very serious ^*,†,‡,§,║	Very serious^††	Serious^#	Serious^**	None	Pooled Effect Not Estimable: One observational study (13) showed mixed/inconclusive associations between workload and indicators of acute fatigue (19). Three studies (Karhula et al., 2013; Baulk et al., 2007; and Dorrian et al., 2011) were believed to have shown a positive relationship between workload and indicators of acute fatigue, such that lower workload was associated with lower fatigue and/or fatigue-related risks (12, 13, 14). One study (Takahashi et al., 2006) showed no association between workload and acute fatigue (18). Three of five studies characterized as favorable towards lower workload to mitigate fatigue. [Median sample size n = 90, min = 20, max = 112]	⊕◯◯◯ VERY LOW	IMPORTANT
Personnel Safety
0							not estimable	—	IMPORTANT
Cost to System
0							Not estimable	—	IMPORTANT

^* No concealment of outcome(s).

^† Method/technique for measuring task load (workload) is subjective and may provide inaccurate information.

^‡ The timing of measurement of task load (workload) may lead to very high or very low values of the variable of interest.

^§ No random assignment.

^║ Study participants may differ from non-participants in some meaningful way.

^# Shift workers other than EMS personnel.

^** Small sample size. Below optimal information size (Guyatt et al., 2011, PMID-21839614).

^†† Two of 5 showed mixed/inconclusive associations between workload and acute fatigue measures. One study showed no relationship. Two showed a positive association (higher workload associated with higher acute fatigue).

Discussion

Summary of Main Results

We identified limited evidence to assess the impact of task load interventions on fatigue, fatigue related risks, and/or sleep quality. Our search for experimental and prospective study designs involving EMS personnel and related shift workers resulted in no relevant experimental research, and only five prospective observational studies. None of the retained literature evaluated the relationship between task load, fatigue, and three outcome measures classified as important by the expert panel (personnel safety, patient safety, and cost to the system). One study examined the relationship between task load, fatigue, and personnel performance (13). Findings were categorized as favorable toward lower levels of task load as a means to mitigate fatigue. Five studies examined the relationship between workload and indicators of acute fatigue; however, only three studies presented findings judged by the study authors as favorable toward lower task load as a method for fatigue mitigation (13, 14). These limited results suggest that further research needs to be conducted specific to EMS and task load. While evidence does not exist to make operational decisions on changing task loads, this concept remains important and the evidence that is present suggests that modification to task load may have an impact on fatigue and fatigue related risks.

Inclusion / Exclusion of Prior Research

The decision to include or exclude a research study was based on the study's relevance to our population, intervention, comparison(s), and outcome(s) of interest. We systematically excluded studies that involved healthy non shift-working volunteers and were non-peer-reviewed literature such as book chapters, theses and dissertations, and government reports. Prospective observational studies that did not define and measure task load (or workload), and did not measure an outcome relevant to our PICO question were excluded (a measure of acute fatigue is an outcome for this PICO) (5). Studies were included where the methodology of the study implied a theoretical framework of task load (or workload) affecting fatigue, and then fatigue related outcomes. An outcome was determined to be sufficient for this review if the theoretical framework of the reviewed paper indicated that the outcome was likely the result of fatigue due to task load changes. All outcomes were defined by the individual study.

Agreement and Disagreement with Other Systematic Reviews

Lang et al. completed a systematic review of literature measuring the ratio of nurses-to-patients (20). Our review was not isolated to this ratio as a singular measure of task load. We excluded studies that may have overlapped with Lang et al. (20) if nurse-to-patient ratio was not defined as a measure of task load, and if the measure was not examined in relation to outcomes of interest in an experimental or prospective study design. Morris et al. examined the nursing literature and determined no common/standard definition of nursing workload (21). The aim of our systematic review differs from Morris et al. (21). We did not seek to define workload or task load, nor did we seek to examine differences in definitions. As there was limited available evidence for review, an appropriate next step for researching task load in EMS would be to first gain a consensus definition and method for quantifying task load prior to studying its effect on fatigue.

Limitations

We limited our collection of relevant literature to select databases. Other databases may index literature and research relevant to our PICO question. The judgment of screeners (PJC and JPC) to include or exclude a record (title/abstract) from the initial pool of records was validated against decisions by the principal investigator (PDP) with a random sample of n = 50 titles and abstracts. Findings from this comparison revealed 92% agreement among three investigators (PDP, PJC, and JPC) in the decision to include or exclude a record based on title and abstract alone. Judgments of evidence quality were guided by the GRADE framework and formulated based on consensus between co-investigators (9). Our judgments may differ from others reviewing the same evidence.

Substantial heterogeneity in what defines task load may have negatively impacted the results of our search (See Online Supplemental Material). First, it is possible that other relevant literature using specific definitions or descriptions of task load was not identified and reviewed. Undiscovered research may have impacted our conclusions and assessment of evidence quality. Second, the lack of standardization in the definition or description of task load may have impacted our decision to include or exclude specific research during screening and/or during the review of full-text articles.

None of the retained literature included EMS personnel as the subjects under study. Our findings are limited by indirectness of the populations studied, which is a common problem in systematic reviews and projects focused on development of evidence based guidelines (22). The construct of task load as associated with fatigue, fatigue related risks, and sleep quality has not been widely assessed in EMS. Literature from other areas of healthcare either did not draw the aforementioned associations or did not clearly define task load. While the shift workers assessed in this review were primarily not healthcare providers, they did complete tasks that were of a perceived difficulty or had a high mental workload. While the tasks may differ between job types, the effectiveness of increased or decreased task load ought to remain constant as it is not a third party judging task load but rather the individual. Clearly, further research into defining task load in EMS is needed so that these associations can be further explored.

Our findings are limited by the low yield in research that met our inclusion criteria. Low yield or empty reviews are not uncommon. A recent review of the Cochrane Database of Systematic Reviews revealed that 8.7% of the more than 4,000 archived systematic reviews yielded no studies (23). The benefits of low yield or empty systematic reviews include: 1) identification of gaps in research; 2) aid in honing research questions and searches for related evidence; and 3) reveal the state of the science at a particular point in time (24).

Conclusions

We discovered considerable heterogeneity in how task load was defined and measured and determined that the overall quality of evidence germane to our outcomes of interest was low or very low. Our systematic review reveals considerable gaps in the scientific literature and no research that examines the relationships between task load in the EMS setting, fatigue, and safety or performance outcomes. Given these findings, we were unable to estimate the effect of task load interventions on fatigue, fatigue-related risks, and/or sleep quality for EMS and related shift worker groups. Further research is needed to: 1) define task load in the EMS operational setting; 2) examine the relationship between task load, and outcomes relevant to EMS; and 3) rigorous experimental research is needed to test task load interventions that are feasible to implement in EMS operations.