Effect of Fatigue Training on Safety, Fatigue, and Sleep in Emergency Medical Services Personnel and Other Shift Workers: A Systematic Review and Meta-Analysis

Abstract

Background: Fatigue training may be an effective way to mitigate fatigue-related risk. We aimed to critically review and synthesize existing literature on the impact of fatigue training on fatigue-related outcomes for Emergency Medical Services (EMS) personnel and similar shift worker groups. Methods: We performed a systematic literature review for studies that tested the impact of fatigue training of EMS personnel or similar shift workers. Outcomes of interest included personnel safety, patient safety, personnel performance, acute fatigue, indicators of sleep duration and quality, indicators of long-term health (e.g., cardiovascular disease), and burnout/stress. A meta-analysis was performed to determine the impact of fatigue training on sleep quality. Results: Of the 3,817 records initially identified for review, 18 studies were relevant and examined fatigue training in shift workers using an experimental or quasi-experimental design. Fatigue training improved patient safety, personal safety, and ratings of acute fatigue and reduced stress and burnout. A meta-analysis of five studies showed improvement in sleep quality (Fixed Effects SMD −0.87; 95% CI −1.05 to −0.69; p < 0.00001; Random Effects SMD −0.80; 95% CI −1.72, 0.12; p < 0.00001). Conclusions: Reviewed literature indicated that fatigue training improved safety and health outcomes in shift workers. Further research is required to identify the optimal components of fatigue training programs to maximize the beneficial outcomes.

Key words::

• education
• training
• sleep
• fatigue
• health
• performance

Background

Shift workers, defined as those whose work periods extend outside of the traditional 9 am–5 pm work day (1, 2), may be at an increased risk from the untoward effects of fatigue. A central component of a fatigue risk management system is sleep health education for employees (3, 4). Fatigue education and awareness training programs are required in commercial aviation (5). Other industries, such as railroad and trucking, have instituted similar programs (6, 7). Educating and training Emergency Medical Services (EMS) personnel to mitigate fatigue and manage their sleep may lead to improved safety and worker health. The need for fatigue education and training is evidenced by the high proportion of EMS personnel that report high levels of work-related fatigue and poor sleep (8, 9).

Key components of fatigue education for shift workers may include basic information on sleep and circadian rhythms, sleep disorders, and the use of fatigue countermeasures such as caffeine and nap strategies. Other components of fatigue training may include optimization of schedules or the sleep environment, mindfulness and/or elements of good behavioral health (10–15). A sleep health education and sleep disorders screening program implemented in firefighters resulted in 42% of focus group participants reporting behavior change, including trying to get more sleep and paying attention to fatigue levels (16). EMS administrators and managers would benefit from learning whether fatigue training improves objective fatigue and fatigue-related risk outcomes.

We conducted a systematic review and meta-analysis of the literature evaluating the impact of fatigue education and training on safety and related outcomes for EMS personnel and related shift worker groups. This systematic review was guided by a single research question: “In EMS personnel, does fatigue training and education mitigate fatigue, mitigate fatigue-related risks, and/or improve sleep?” (PROSPERO 2016: CRD42016040110) (17).

Methods

We used a systematic review study design based on literature retrieved from multiple sources. The details of our methodology and study protocol, and procedures for reviewing the literature are described in a separate publication (18). In the following sections, we discuss the components of our search unique to this systematic review.

Study Design

We reviewed publications that describe randomized controlled trials, quasi-experimental studies (19), and observational study designs.

Types of Participants

We retained literature that described studies that included persons 18 years of age and older classified as shift workers, EMS personnel or similar worker groups (17).

Types of Interventions

We retained literature that describes studies evaluating the impact of fatigue training. Each study had to include education on fatigue and/or sleep health as a minimum, but often included education on other related topics and/or interventions.

Types of Outcome Measures

We previously described the selection of outcomes for this systematic review (17). Important outcomes of interest included personnel safety, patient safety, personnel performance, acute fatigue, indicators of sleep duration and quality, indicators of long-term health (e.g., cardiovascular disease), and burnout/stress.

Search Methods for Studies

We describe the details of our search methods in a methods paper that appears in this supplement (18). In that paper, we describe the concepts that guided searches, search terms, vocabulary, and procedures/protocol for searching multiple databases. All searches of five databases and one website included literature from January 1980 to September 2016. Online Supplement Appendix A provides additional information regarding our search strategy specific to this systematic review.

Data Collection and Selection of Studies

Screening

Two co-investigators (PJC and MLR) independently screened titles and abstracts of search results to identify studies potentially germane to study objectives. Two additional investigators (PDP and DJS) adjudicated disagreements on inclusion/exclusion criteria, ensuring that: (a) the study included the population of interest; (b) the title and/or abstract describes one or more outcomes of interest; and (c) the study reports those outcomes of interest stratified by whether or not participants received training and/or education in fatigue or sleep health. The Kappa statistic was used to determine inter-rater agreement during screening.

Full-Text Review

Five investigators (PDP, MLR, KLF, JPC, and AAD) worked independently to abstract key information from full-text articles. Four investigators (MLR, KLF, JPC, and AAD) verified all abstracted information. We adjudicated disagreements with discussion. We excluded book chapters, conference abstracts, newsletters and similar publications, dissertations and thesis documents. Co-investigators MLR, KLF, JPC, and AAD searched bibliographies to identify additional relevant research.

Risk of Bias Assessment

Our team's three senior investigators (LKB, MSR, and PDP) used the Cochrane Collaboration's tool to evaluate experimental studies for risk of bias (20). The same three investigators used the GRADE template to assess risk of bias for studies that used observational designs (21). Disagreements were resolved by discussion and consensus

Statistical Analysis

Three investigators^{Figure 1} (PDP, LKB, and MSR) adapted a categorical system used by Bolster and Rourke to describe the impact of a fatigue training intervention on critical and important outcomes as favorable, unfavorable, mixed/inconclusive, or no impact (22). We describe our protocol for categorizing findings in a separate publication (18).

Multiple studies had pre- and post-intervention (e.g., randomized trial or quasi-experimental design) results for the sleep/sleep quality outcome, enabling us to pool and conduct a meta-analysis. One experimental study reported data for a commonly used performance measure (the psychomotor vigilance test) (23). We abstracted the means at baseline and follow-up and the corresponding standard deviations to calculate the change (and corresponding 95% confidence intervals) in sleep/sleep quality over time. We used RevMan to calculate the standardized mean difference (SMD) and 95% confidence intervals (CIs) of a pooled main effect. The SMD is the estimated change (post minus pre) relative to the variability in the study (20). An SMD of zero indicates no change over time (24). An SMD that is greater than zero suggests worsening in sleep/sleep quality. The SMD is non-significant if the corresponding 95% confidence interval overlaps zero. The I² statistic was calculated as a standard measure of heterogeneity (20). The I² is the percentage of total variation across the included studies related to heterogeneity with values ranging from 0% to 100%. If heterogeneity was deemed significant (Chi-square p-value < 0.05 and high I²), we presented both fixed and random effects SMD estimates and corresponding confidence intervals.

Quality of Evidence

Four investigators (PDP, LKB, MSR, and ESL) used the GRADE framework and prescribed evidence profile tables to summarize and rate the quality of retained research (evidence) (25). The GRADE evidence profile table contains key information about the quality of evidence germane to outcomes rated as critical and important (26). Key information includes: number of studies per outcome; judgments about underlying quality of evidence (e.g., risk of bias, indirectness); statistical results; and a quality rating (very low, low, moderate, or high).

Reporting

We present our findings as prescribed by the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement (27).

Results

The search strategy yielded n = 3,817 unique records of which n = 73 duplicates were removed manually (Figure 1). Two investigators (PJC and MLR) independently screened n = 3,744 titles and abstracts. The interrater agreement for inclusion/exclusion was substantial (Kappa = 0.88). Thirty-nine records were judged potentially eligible based on title and abstract. Sixteen additional studies were identified during the search of bibliographies as potentially relevant and reviewed in full-text format. Eighteen experimental and non-experimental studies were judged relevant and were included in the systematic review, with the key findings abstracted in tables (Online Supplement Appendix B). The reasons for exclusion of the remaining 37 studies are detailed in the online appendix, organized in the Population, Intervention, Comparison, Outcome (PICO) format (Online Supplement Appendix C) (28–30). See Online Supplement Appendix D for assessments of risks of bias.

Figure 1. PRISMA 2009 Flow Diagram for PICO#5 PROSPERO 2016:CRD42016040110.

We detected wide variation in education and/or training programs/interventions (Table 1). Fatigue training and education took on many forms and the amount of detail describing the fatigue training was inconsistent. Programs ranged from one-hour lectures (31, 32) to a one-day workshop (33), to more extended programs, such as 2+ hour weekly classes for eight-weeks (12). Some programs were delivered in-person, led by experts (15, 34), or team-based / peer-led (13), and others were email-based (35) or digital (23). Although all of the programs included an element of fatigue education, the focus varied from mindfulness (12) and self-care practices (33) to exercise prescription (14). Several programs included additional elements or bundles of countermeasures, such as scheduling interventions (10, 23) and an electronic handover tool (36). Smith-Coggins et al. (37) used a complicated intervention protocol such that all participants received education and information germane to sleep physiology and circadian rhythms in the first hour, and the intervention arm received additional information about sleep hygiene, fatigue countermeasures, and scheduling in the second hour. The Arboleda et al. study comprised survey responses from 116 (20.5% response rate) trucking firms that included questions on driver fatigue training but provided no detail on the content, duration, or format of the training (38).

Table 1. Description of the sleep health and/or fatigue-related education/training intervention/program

Author, Year	Refid PMID#	Description
Scott, 2010	RefID-2995 PMID-20467338	A one-hour in-person education including information about fatigue, sleep and circadian rhythms, neurobehavioral and health effects associated with sleep loss or deprivation and misconceptions about sleepiness. Strategies for managing alertness, minimizing fatigue and maximizing sleep duration and quality were highlighted with an emphasis on naps and caffeine.
Rosekind, 2006	RefID-2872 PMID-17183922	A comprehensive alertness management program involved education, a 3.5-hour interactive multimedia CD-ROM including alertness strategies and information on healthy sleep. Innovative schedules were also developed and implemented.
Poulsen, 2015	RefID-2728 PMID-26094782	A one-day workshop on recovery self-care practices with written educational material, including a focus on sleep quality.
Chen, 2010	RefID-924 PMID-21139448	A 5-week education program including topics: arranging good sleep environment and habits; reducing emotional stress; controlling diet, alcohol consumption and tobacco use, exercising regularly and introducing alternative therapies.
Carter, 2013	RefID-872 PMID-23989029	Cognitive-behavioral therapy for insomnia.
Atlantis, 2006	RefID-538 PMID-17059294	Supervised exercise prescription and health education seminars on wide-ranging topics such as disease, nutrition, ergonomics and occupational jet lag. One-on-one, 1-hour counseling sessions available mostly focused on nutrition. Sleep hygiene strategies were promoted for preventing occupational jet lag and included: 1) adjusting sleep schedule; 2) adjust meal times; 3) avoid large meals and caffeine towards end of shift; 4) bright light exposure during breaks; 5) wear sunglasses during travel home; 6) reduce noise/light at home for sleep preparation; 7) use of naps before night shift; 8) exercise therapy.
Holbrook, 1994	RefID-1682 PMID-7808891	One-hour sleep hygiene training as part of a three-hour in-service educational forum on health and nutrition.
Kuehl, 2016	RefID-1997 PMID-27158956	The SHIELD program included twelve 30-minute, team-based, scripted, peer-led sessions over six months. Focused on improving diet, physical activity, body weight, and sleep and reducing unhealthy stress and behaviors such as tobacco and substance abuse. The program targeted multiple sleep disorders and evaluated the impact of the multi-component program on sleep quality.
Hardaway, 2005	RefID-1597 PMID-n/a	Extensive briefings on sleep physiology and fatigue countermeasures, guide of suggested nap and sleep periods and suggested light exposure and avoidance times to facilitate circadian alignment, additional time between flights (42 hours off versus 18 hours off on a layover).
Smith, 2016	RefID-3093 PMID-n/a	Email-based sleep wellness program consisting of eight weekly modules highlighting different aspect of sleep health.
Sullivan, 2016	RefID-n/a PMID- 27692049	In-person sleep health educational seminar with sleep disorders screening; facilitating further evaluation, diagnosis, and treatment for those found at high risk of a sleep disorder.
Fabreau, 2013	RefID-1272 PMID-23987729	A “bundle” that included a night-float system, educational sessions on sleep hygiene, an electronic handover tool, and a simulation-based medical education curriculum.
Christopher, 2015	RefID-946 PMID-n/a	A mindfulness training program with an eight-week curriculum and 2+ hour classes each week. The program was described as being grounded in practices shown previously to enhance / address fatigue and poor sleep.
Lee, 2014	RefID-2074 PMID-24229383	A four-week home-based cognitive behavioral therapy including cognitive restructuring, sleep hygiene, sleep restriction, stimulus control, and relaxation training.
Arora, 2007	RefID-529 PMID-17846392	A 60–90 minute SAFER program lecture. The Sleep, Alertness, and Fatigue Education in Residency (SAFER) program was developed by the American Academy of Sleep Medicine. Elements of the program are available on the academy's website at: (http://www.aasmnet.org/store/product.aspx?pid = 1224. Last Accessed 2/14/17).
Steffen, 2015	RefID-n/a PMID- 25563534	Eight weekly, live participatory group sessions including sleep hygiene, stimulus control, guided imagery, cognitive restructuring, meditation, and relaxation techniques.
Smith-Coggins, 1997	RefID-n/a PMID-9332626	A multi-component intervention that included a 2-hour educational session. The first hour included education on sleep physiology and circadian rhythms. Participants also received information about diet manipulation (active placebo group/control group). Other components of the main intervention included schedule modification and promotion of multiple fatigue countermeasure strategies.
Arboleda, 2003	RefID-517 PMID-12737958	Driver fatigue training. Details of the program's components were not provided in the manuscript.

Impact of Fatigue Education and Training on Patient Safety Outcomes

Two quasi-experimental studies determined the impact of a fatigue education and training type intervention on outcomes germane to patient safety (i.e., medical errors and quality of care delivery) (32, 36). Scott et al. used a quasi-experimental design to study nurses working on medical -surgical units in 3 major acute care Michigan hospitals and found a decrease number of errors following fatigue training (p = 0.01) (32). Fabreau et al. used a quasi-experimental design to study internal medicine residents at two large academic hospitals in Canada and found that fatigue training was associated with a decrease in the residents' perception of their potential for error (p = 0.003) (36). The pooled effect of these studies was not estimable. However, investigators classified the findings from these two studies as favorable for patient safety (Online Supplemental Table 2).

Impact of Fatigue Education and Training on Personnel Safety

Three studies evaluated the impact of fatigue education and training on measures of personnel safety (15, 32, 38). Scott et al. used a quasi-experimental study design and reported episodes of drowsy driving and motor vehicle crashes (MVCs) as measures of personnel safety (32). Findings suggest a 20% reduction in the number of drowsy driving episodes and 80% reduction in MVCs following the fatigue training. Lack of statistical tests of differences pre-to-post intervention led to our categorizing these findings as mixed/inconclusive (Online Supplement Table 2). Sullivan et al. completed a randomized trial and captured MVCs and injuries (15). No differences detected in the rate of MVCs or injuries between the intervention and control groups. Post-hoc analyses showed a 24% lower odds of firefighters filing an injury report among those attending the fatigue education and training session versus firefighters who did not attend (OR 0.76; 95% CI 0.60, 0.98; p = 0.03). We categorize these findings as favorable for personnel safety. One observational study by Arboleda et al. detected a positive association between fatigue training and improvement in perceived safety culture in truck drivers (38). We categorized these findings as favorable for personnel safety (Online Supplement Table 2).

Table 2. Synthesis of findings on the impact of fatigue/sleep education and/or training programs on critical and important outcomes

Experimental Studies
			Important Outcomes
Author, Year	Refid PMID^*	Study Design	Personnel Safety	Patient Safety^*	Personnel Performance^†	Acute Fatigue^‡	Sleep and Sleep Quality^§	Long-Term Health^║	Burnout / Stress
Scott, 2010	RefID-2995 PMID-20467338	Quasi-experimental	Mixed/ Inconclusive	Favorable		Favorable	Favorable	—	—
Rosekind, 2006	RefID-2872 PMID-17183922	Quasi-experimental	—	—	Favorable	—	Favorable	—	—
Poulsen, 2015	RefID-2728 PMID-26094782	Quasi-experimental	—	—	—	—	Favorable	—	Favorable
Chen, 2010	RefID-924 PMID-21139448	Quasi-experimental	—	—	—	—	Favorable	—	—
Carter, 2013	RefID-872 PMID-23989029	Quasi-experimental	—	—	—	—	Mixed/ Inconclusive	—	—
Atlantis, 2006	RefID-538 PMID-17059294	Randomized controlled trial	—	—	—	—	Favorable	—	—
Holbrook, 1994	RefID-1682 PMID-7808891	Quasi-experimental	—	—	—	—	No impact	—	—
Kuehl, 2016	RefID-1997 PMID-27158956	Randomized, longitudinal cohort	—	—	—	—	Mixed/ Inconclusive	Mixed/ Inconclusive	Favorable
Hardaway, 2005	RefID-1597 PMID-n/a	Quasi-experimental	—	—	—	Favorable	Favorable	—	—
Smith, 2016	RefID-3093 PMID-n/a	Quasi-experimental	—	—	—	—	No Impact	—	—
Sullivan, 2016	RefID-n/a PMID- 27692049	Randomized controlled trial	Favorable	—	—	No impact	No impact	Favorable	—
Fabreau, 2013	RefID-1272 PMID-23987729	Quasi-experimental	—	Favorable	—	—	—	Favorable	—
Christopher, 2015	RefID-946 PMID-n/a	Quasi-experimental	—	—	—	Favorable	Favorable	Favorable	Favorable
Lee, 2014	RefID-2074 PMID-24229383	Quasi-experimental	—	—	—	—	Favorable	Favorable	—
Arora, 2007	RefID-529 PMID-17846392	Prospective cohort followed by quasi-experimental	—	—	—	—	No impact	—	—
Steffen, 2015	RefID-n/a PMID- 25563534	Quasi-experimental	—	—	Favorable	Favorable	Favorable	Favorable	Favorable
Smith-Coggins, 1997	RefID-n/a PMID-9332626	Randomized cross-over design	—	—	Mixed/ Inconclusive	Mixed/ Inconclusive	Mixed/ Inconclusive	—	—
Arboleda, 2003	RefID-517 PMID-12737958	Observational, survey	Favorable	—	—	—	—	—	—

Findings are classified as favorable in support of program implementation, unfavorable, mixed/inconclusive, or no impact.

^* Includes quality of care.

^† Includes external subjective ratings of the study subject's performance including perceived satisfaction with the subject's performance.

^‡ Includes acute states of fatigue, sleepiness, and alertness.

^§ includes sleep latency, total sleep time, recovery, and related measures.

^║ General wellness or well-being measures included.

Impact of Fatigue Education and Training on Personnel Performance

Rosekind et al. used a quasi-experimental study design to test the impact of an alertness management program on reaction time of pilots during flights and post-flight during a recovery period (23). The intervention had a large, significant positive effect on reaction time during shift (flights) [Cohen's D = 0.86, 95%CI 0.22, 1.51]. The positive effect was evident with faster reaction time on the Psychomotor Vigilance Test (PVT) among pilots receiving the intervention versus pilots who did not. The intervention had a moderate, yet non-significant effect on reaction time during the recovery period [Cohen's D = 0.45, 95%CI −0.20, 1.09]. We categorize these findings as overall favorable for personnel performance (Online Supplement Table 2). Smith-Coggins et al. used a randomized cross-over design and captured PVT measures during a baseline, post active placebo, and intervention condition (37). An effect is not estimable for this intervention on PVT, from an exposure to non-exposure perspective. Both the active placebo condition and intervention condition were exposed to education of sleep physiology and circadian rhythms. Findings are not reported with the active placebo and intervention conditions pooled and compared to baseline; thus we categorized findings as mixed/inconclusive (Online Supplement Table 2). The impact of fatigue education on other performance measures for this study (i.e., physician intubation and interpretation of electrocardiogram rhythms) was categorized as mixed/inconclusive.

Impact of Fatigue Education and Training on Personnel Performance as Measured by Self-Reported Perception of Performance Affected by Inadequate/Poor Sleep

Steffen et al. used a quasi-experimental study design to evaluate the impact of a multi-component intervention delivered in group sessions over eight weeks (34). Steffen and colleagues evaluated program impact with a 26-item questionnaire that included a measure germane to personnel performance (34). Based on the study's findings (a 20% reduction in number of days when performance is affected by inadequate sleep), we categorize this study's findings as favorable for personnel performance (Online Supplement Table 2).

Impact of Fatigue Education and Training on Measures of Sleep

Polysomnographic recordings were used in the Smith-Coggins et al. study to quantify total sleep time (37). Both the active placebo and intervention study arms received education on sleep physiology and circadian rhythms post randomization. In addition, the intervention group received additional training on good sleep hygiene and chronobiologic principles of scheduling as well as an optimized clinical schedule, and 31 countermeasure strategies to enhance on-duty alertness and performance. Alternative reporting of findings may have allowed for pooling the study arms for a comparison with baseline recordings. We categorized findings as mixed/inconclusive (Online Supplement Table 2). Six additional studies (10, 13, 23, 31, 32, 39) used diverse measurement tools (i.e., survey measures) and different context or reference points for sleep documentation (e.g., average duration over previous month, pre-shift, post-shift, recovery). Arora et al. detected no impact of the fatigue education and training program on the amount of sleep post intervention (31). Although findings were not consistent across studies, most suggest a positive impact on total sleep.

Impact of Fatigue Education and Training on Acute Fatigue

One randomized trial that used an in-person health education and sleep disorder screening showed no differences in multiple measures of alertness between the intervention and control groups at 12 months post-baseline measure (15). We categorize these findings as no impact (Online Supplement Table 2). Scott et al. detected no differences between pre- and post-intervention measures of sleepiness at 4 weeks and 12 weeks (32). Three additional quasi-experimental studies detected lower levels of acute fatigue during the post-intervention period compared to baseline (the pre-intervention period) (10, 12, 34). We categorize the findings of four quasi-experimental studies as favorable for mitigating acute fatigue (Online Supplement Table 2) (10, 12, 32, 34). We categorized Smith-Coggins et al. study findings germane to alertness as mixed/inconclusive for the same reasons listed previously (37).

Impact of Fatigue Education and Training on Sleep Quality

Five quasi-experimental studies measured sleep quality with the reliable and valid Pittsburgh Sleep Quality Index (PSQI) at baseline and within four to eight weeks post initiation of the fatigue education / training program (11, 32, 35, 39, 40). The effect of fatigue education and training on sleep quality at four to eight weeks post baseline was large and statistically significant (SMD −0.87; 95% CI −1.05 to −0.69; p < 0.00001; Figure 2a) using the fixed effects model. Allowing for heterogeneity (ChiSq = 97.28; df = 4; p < 0.00001; I² = 96%), the random effects estimate was similar but significance was attenuated (SMD −0.80; 95% CI −1.72, 0.12). Sensitivity analysis in Online Supplement Table 3 shows the pooled effect is consistently negative (towards better sleep quality) but heterogeneity is driven by the large effect−2.08 (11). When the Chen et al. study was removed, the SMD was smaller and suggests better sleep quality post intervention (fixed effect −0.31 95% CI −0.52, −0.09 and random effect −0.45 95% CI −0.96, 0.06). Given these findings, we interpreted the overall findings for the outcome of sleep quality as favorable (Online Supplement Table 2). We evaluated the effect of a single randomized trial on sleep quality measured at baseline and again at 24 weeks (14). The effect of the study's fatigue education and training intervention on sleep quality among the intervention participants classified as shift workers versus the control participants classified as shift workers was moderate, yet not statistically significant (Cohen's D = 0.55; 95%CI −0.29 to 1.39). We categorized the findings of fatigue education and training interventions on sleep quality as favorable (Online Supplement Table 2).

Figure 2. a) Forest Plot (outcome: sleep quality 4–8 weeks post-intervention follow-up fixed effects model. b) Forest Plot (outcome: sleep quality 4–8 weeks post-intervention follow-up random effects model) The figure reports the standardized mean difference (SMD) for sleep quality as measured by the within-group change in the Pittsburgh Sleep Quality Index (PSQI) during the post-intervention follow-up in five studies occurring between 4 and 8 weeks relative to baseline. The SMDs for these studies are the estimated change from baseline relative to the variability in the study and also known as Cohen's d measurement of effect size. The effect size is not tied to a specific scale or scales used in the pooled analysis. An SMD of zero implies there was no change from baseline. An SMD less than zero indicates the outcome at follow up was less than at baseline. Common delineations or cut-points for interpretation include: 0.2 = small; 0.5 = medium/moderate; 0.8 or greater as large. The SMD is non-significant if the corresponding 95% confidence interval is wide and overlaps 0. For the Carter et al. study we used the PSQI measurements at baseline and 5 weeks follow-up (40). For the Chen et al. study, we used the PSQI measurements at baseline and 5 weeks follow-up (11). For the Scott et al. study, we used the PSQI measurements at baseline and 4 weeks follow-up (32). For the Smith et al. study, we used the PSQI measurements at baseline and 8 weeks follow-up (35). For the Lee et al. study, we used the PSQI measurements at baseline (T1) and (T3) 4 weeks after intervention phase follow-up (39). The standard deviations were estimated using the standard deviations from the baseline and follow up measure assuming a positive, moderate correlation between measurements within the same individual: SD for the change = √(SD² _baseline + SD² _follow-up −2^*0.5^*SD_baseline^*SD_{follow up}).

Impact of Fatigue Education and Training on Burnout/Stress

The Kuehl et al. randomized trial showed that participants in the intervention group reported lower levels of stress at six months post baseline than participants in the control group (13). This difference disappeared at 12 months post baseline. Three quasi-experimental studies also measured burnout and/or stress with diverse measurement tools (12, 33, 34). A pooled effect was not estimable; however, findings suggest a positive impact of the fatigue education and training interventions on indicators of burnout and stress post introduction of the intervention. We categorized findings across all three studies as favorable (Online Supplement Table 2).

Impact of Fatigue Education and Training on Indicators of Long-Term Health

Four quasi-experimental studies evaluated the impact of fatigue education and training interventions on diverse indicators of long-term health (e.g., self-reported mental health, physical health, and depression symptoms) (12, 34, 36, 39). A pooled effect is not estimable given the diversity in measures and measurement timing. We categorized the findings for these four studies, which used diverse fatigue education and training programs, as having a favorable impact on indicators of long-term health (Online Supplement Table 2).

Two additional studies (randomized trials) captured participant self-reported assessments of general health at baseline and 12 months follow up (13, 15). The pooled effect of these trials was not estimable given incomplete reporting of study findings. The measurement tools differed; however, one trial showed a statistically significant improvement of general health at six months, but not at 12 months post baseline (13). Sullivan et al. did not detect a difference in their general health measures at 12 months post-randomization (15). The Sullivan et al. study determined that participants attending the fatigue education session reported fewer injury/disability days per person than participants who did not receive exposure to the intervention (15). On balance, we categorized the findings for these two trials as favorable for the impact of fatigue education and training on indicators of long-term health (Online Supplement Table 2).

Quality of Evidence

Most studies were judged to have a serious or very serious risk of bias due to the limitations associated with quasi-experimental study designs (19). Most did not incorporate randomization, and blinding was not possible or feasible. We downgraded for small sample sizes and indirectness of evidence involving shift workers other than EMS personnel. We judged the overall quality of the evidence as low or very low for important outcomes (Online Supplement Table 3). The evidence quality for selected studies using randomized controlled designs was moderate or high for select outcomes (burnout/stress and indicators of long-term health). Enthusiasm for evidence quality (or confidence in the effect) is lessened when considering the inability to calculate a pooled effect for the indicators of long-term health comprising the two randomized trials (Online Supplement Table 3).

Discussion

Summary of Main Results

Across 36 years of research, we identified favorable evidence from 13 studies including experimental, quasi-experimental, and observational research in support of fatigue education and training for purposes of mitigating one or more of our a priori-specified measures of fatigue, fatigue related risks, and sleep quantity and quality. Findings show a positive short-term effect of fatigue education and training on shift worker sleep quality, even after attenuation when accounting for heterogeneity in fatigue training programs (Figures 2a and 2b). The preponderance of positive results across outcome measures suggests that interventions or programs that include fatigue training and/or sleep health education is beneficial for shift workers.

Quality of Evidence

We judged the quality of evidence for the important outcomes of interest as low to very low. We determined that most studies had serious or very serious risk of bias due to the limitations associated with quasi-experimental study designs (e.g., no randomization or blinding) (19).

Agreement and Disagreement with Other Systematic Reviews

We did not identify previous systematic reviews fitting our PICO.

Limitations

We limited our collection of relevant literature to select databases and study reports published in the English language. Additional databases may index literature and research relevant to our PICO question. Because application of inclusion criteria can be subjective, we examined the judgment of screeners (PJC and MLR) to include or exclude a record (title and abstract) against decisions by the principal investigator (PDP) with a random sample of n = 50 records. Findings from this comparison revealed 100% agreement between co-investigators PDP, PJC, and MLR.

There are limitations with our meta-analysis of sleep quality. Only five quasi-experimental studies reported sleep quality measure data in a format that could be pooled for meta-analysis. The diversity in fatigue education and training interventions raises questions about pooling findings from studies with substantial heterogeneity. We believe that because all three studies qualified for our review based on the inclusion of some form of fatigue training and/or sleep health education, and because sleep quality is a relative measure, it was appropriate to pool the findings.

Fatigue training programs across studies were very disparate. Some have suggested that a comprehensive program to manage fatigue in work settings should include six elements: (a) education and training; (b) compliance with hours of service regulations; (c) appropriate scheduling practices; (d) countermeasures that can be instituted in the work setting; (e) design and technology; and (f) research (32, 41). It is possible that a comprehensive fatigue management program including all of these elements could be more effective in improving the health, safety, performance, and sleep of shift workers and patient safety. Due to the variation in the composition and implementation of the programs in the literature, we were not able to compare the elements of the programs, the length of the programs and the endurance of the favorable outcome measures.

Additionally, the studies we reviewed collected data in operational settings and many lacked parallel control groups. In the reviewed field studies, confounding from other fatigue countermeasures (e.g., caffeine), prior wakefulness or work conditions were not controlled. This lack of standardization may contribute to the variability seen in the results.

Judgments of evidence quality were guided by the GRADE framework and formulated based on consensus between authors (25, 26). Our judgments may differ from those of others reviewing the same evidence.

Conclusions

The effect of fatigue training on select outcomes was stable across studies, showing a favorable impact on important outcomes germane to patient and shift-worker safety, personnel performance, acute fatigue, sleep quality, burnout/stress and indicators of long-term health. The overall quality of evidence was judged low or very low due to lack of randomized clinical trials in operational settings. There was considerable heterogeneity among the included studies and the optimal fatigue-training program content and duration is unclear. Further research evaluating which fatigue training elements are most advantageous is vital.