2022 Systematic Review of Evidence-Based Guidelines for Prehospital Care

Abstract

Introduction

Multiple national organizations and federal agencies have promoted the development, implementation, and evaluation of evidence-based guidelines (EBGs) for prehospital care. Previous efforts have identified opportunities to improve the quality of prehospital guidelines and highlighted the value of high-quality EBGs to inform initial certification and continued competency activities for EMS personnel.

Objectives

We aimed to perform a systematic review of prehospital guidelines published from January 2018 to April 2021, evaluate guideline quality, and identify top-scoring guidelines to facilitate dissemination and educational activities for EMS personnel.

Methods

We searched the literature in Ovid Medline and EMBASE from January 2018 to April 2021, excluding guidelines identified in a prior systematic review. Publications were retained if they were relevant to prehospital care, based on organized reviews of the literature, and focused on providing recommendations for clinical care or operations. Included guidelines were appraised to identify if they met the National Academy of Medicine (NAM) criteria for high-quality guidelines and scored across the six domains of the Appraisal of Guidelines for Research and Evaluation (AGREE) II tool.

Results

We identified 75 guidelines addressing a variety of clinical and operational aspects of EMS medicine. About half (n = 39, 52%) addressed time/life-critical conditions and 33 (44%) contained recommendations relevant to non-clinical/operational topics. Fewer than half (n = 35, 47%) were based on systematic reviews of the literature. Nearly one-third (n = 24, 32%) met all NAM criteria for clinical practice guidelines. Only 27 (38%) guidelines scored an average of >75% across AGREE II domains, with content relevant to guideline implementation most commonly missing.

Conclusions

This interval systematic review of prehospital EBGs identified many new guidelines relevant to prehospital care; more than all guidelines reported in a prior systematic review. Our review reveals important gaps in the quality of guideline development and the content in their publications, evidenced by the low proportion of guidelines meeting NAM criteria and the scores across AGREE II domains. Efforts to increase guideline dissemination, implementation, and related education may be best focused around the highest quality guidelines identified in this review.

Introduction

Since the publication of the first National EMS Research Agenda in 2002 (1), and the 2007 publication “Emergency Medical Services: At the Crossroads” by the National Academy of Medicine (NAM, formerly the Institute of Medicine) (2), there have been extensive efforts to increase the science that guides EMS medicine. Evidence-based guidelines (EBGs) form the basis for translating the latest scientific research into clinical practice (3). Following the recommendations of the National EMS Advisory Council (4) and the Federal Interagency Committee on EMS (5), the National Highway Traffic Safety Administration (NHTSA) and the Health Resources and Services Administration have funded multiple projects to create prehospital EBGs (6–10), and to evaluate their implementation (11–15). Similarly, other national and international organizations have continued their ongoing work to develop and disseminate guidelines relevant to EMS medicine, including the American Heart Association (16) and the European Resuscitation Council (17). Continuously monitoring the release of guidelines, synthesizing recommendations, and maintaining a repository of guidance germane to EMS medicine may contribute substantially to dissemination and implementation.

The National Prehospital EBG Strategy identified a need to establish a sustainable process that promotes the development, implementation, and evaluation of prehospital EBGs (18). In fulfillment of this strategy, the Prehospital Guidelines Consortium (PGC) performed a systematic review of 71 prehospital EBGs published through September 2018 (19). This prior review identified important gaps in the development and reporting of prehospital guidelines, and additional work by the PGC identified specific topic areas that should be areas of focus for new guideline development or updates (20). Further work performed by the PGC evaluated the existing implementation science relevant to prehospital EBGs (21) and along with guideline implementation efforts across multiple states funded by NHTSA (11), identified gaps in knowledge dissemination, among other aspects, limiting implementation of EBGs across EMS systems. To further improve this knowledge dissemination, multiple national organizations have recommended that an updated repository of prehospital EBGs be maintained (18), an effort initiated by the PGC in 2017 (www.prehospitalguidelines.org/ebgs).

Considering the importance of evidence-based prehospital care, the National Continued Competency Program from the National Registry of Emergency Medical Technicians (NREMT) contains requirements for education on EMS research and evidence-based guidelines (22). With the intent of informing related initial certification and continued competency activities for EMS personnel, and to continually update the repository of prehospital EBGs consistent with the National Prehospital EBG Strategy, NREMT and the PGC formed a memorandum of understanding in 2021 establishing a sustainable process to continuously monitor and biennially identify the latest prehospital EBGs. Subsequently, we performed an interval systematic review of prehospital EBGs to inform the EMS community about the latest guidelines published since our inaugural systematic review of prehospital EBGs. This systematic review update aimed to identify new EBGs relevant to EMS clinical medicine and non-clinical operations published since 2018. We further aimed to evaluate the quality of published prehospital guidelines and identify top-scoring guidelines that could be incorporated into initial certification and continued competency activities for EMS personnel.

Methods

Study Design & Search Strategy

We performed a systematic review and structured appraisal of published guidelines relevant to prehospital care (defined below). This work was performed by members of the PGC’s Research and Development Committees, along with co-investigators who assisted with screening records, full-text reviews, and manuscript preparation. Our systematic review involved a research librarian who searched Ovid MEDLINE and EMBASE for articles published from inception to April 30, 2021, excluding EMBASE conference abstracts. The keywords and search strategy are described in Supplementary Tables 1 and 2. Search terms were developed using the search terms used in 2018, with the addition of “abstract search (ab.)” for the EMS search string and the search term “first responder*.ti,ab,kw.” Additionally, we manually searched bibliographies and position statements published by the National Association of EMS Physicians during the search period for additional articles that met our screening criteria for prehospital EBGs. Records retained in these searches were screened for topical relevance and full-text articles were examined in detail if they were published after January 1, 2018, and not included in the prior systematic review (19). Non-English language publications were excluded from the initial search and duplicates were subsequently excluded.

Guideline Selection and Categorization

Two investigators (RMH and BTP) independently screened records (titles and abstracts) that met three inclusion criteria for our definition of a “prehospital evidence-based guideline”: the record addresses a clinical or operational topic relevant to prehospital care or EMS medicine (23), provides recommendations/guidance for clinical care or operations, and describes performance of an organized review of the literature as a basis for forming those recommendations (e.g., systematic review, narrative review, rapid review, scoping review, or simply “literature review”). We used the first 50 records/articles from the searches to train co-investigators how to systematically apply review criteria during the screening process as described by Ng et al. (24). In terms of EBG content and scope, a guideline was included/retained if at least one section of the guideline was determined to be relevant to prehospital care, even if the bulk or rest of the guideline document was related to hospital-based care. We also evaluated position statements from professional organizations using the above-mentioned criteria.

Initial screening was conducted using DistillerSR (Evidence Partners, Ontario, Canada) and articles were retained if they met all three criteria. Conflicts were resolved by discussion of two senior investigators (CMG and PDP). Retained articles and articles found by bibliography and related searches underwent full-text review with adjudication of conflicts by discussion. The final decision for including or excluding a guideline document was based on consensus of five co-investigators (KMB, REC, CMG, PDP, and CTR).

We then used the American Board of Emergency Medicine 2019 Core Content of EMS Medicine to categorize each guideline by content area (23). Guidelines addressing multiple clinical or operational topics were categorized into multiple topic areas where appropriate and based on investigator judgment.

Guideline Appraisal

We appraised the evidence evaluation, development, and reporting of each guideline using the National Academy of Medicine (NAM) criteria for clinical practice guidelines and the Appraisal of Guidelines for Research and Evaluation (AGREE) II tool, as previously described (19). Tables describing the qualitative NAM criteria for clinical practice guidelines and the six AGREE II guideline domains can be found in this prior publication.

The NAM criteria were previously adapted based on criteria for clinical practice guidelines established for the National Guidelines Clearinghouse by the U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality (AHRQ) (25). These criteria were adapted from the NAM publication “Clinical Practice Guidelines We Can Trust” (26). Guidelines were assessed on all NAM criteria, as summarized in Table 1. Appraisals were determined by independent full-text review of three co-investigators (KMB, REC, and CTR), with review of disagreements and final consensus of four investigators (KMB, REC, CMG, and CTR). Supplementary content that was referenced or linked within the cited guideline as an evidentiary basis for the guideline’s recommendations (e.g., a separate systematic review of the literature that directly linked to the guideline’s recommendations) was considered as part of the guideline for NAM scoring. All retained guidelines met the additional NAM criteria of being published in English, available to the public (for free or for a fee), and being the most recent versions.

Table 1. Criteria for clinical practice guidelines assessment.

Guideline	Systematically Developed Recommendations	By an Association or Similar	Systematic Review	Description of Search Strategy	Study Selection	Synthesis of Evidence	Summary of Evidence Synthesis	Assesses Benefits/ Harms and Alternate Options	Meets All NAM Criteria
Adeoye 2019	No	Yes	No	No	No	No	Yes	Yes	No
Ahmed 2019	Yes	Yes	No	No	No	No	Yes	Yes	No
Atkins 2018	Yes	Yes	Yes	No	No	No	Yes	Yes	No
Babl 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Bennett 2020	Yes	Yes	No	Yes	No	No	Yes	Yes	No
Berg 2020	Yes	Yes	Yes	No	No	No	Yes	Yes	No
Best 2020	Yes	No	No	No	No	No	Yes	Yes	No
Blancher 2018	Yes	Yes	No	Yes	Yes	No	Yes	Yes	No
Bouzat 2020	Yes	Yes	No	No	No	Yes	Yes	Yes	No
Charlton 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Cheng 2020	Yes	No	No	Yes	No	No	Yes	Yes	No
Cicero 2021	Yes	Yes	No	No	Yes	Yes	Yes	Yes	No
Cook 2020	No	Yes	No	No	No	No	Yes	Yes	No
Cottey 2019	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	No
Craig 2020	No	Yes	No	No	No	No	No	Yes	No
Doherty 2018	Yes	Yes	No	No	No	No	Yes	Yes	No
Dow 2019	Yes	Yes	No	Yes	No	No	Yes	Yes	No
Duff 2018	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	No
Duff 2019 (BLS)	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Duff 2019 (PALS)	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Eggleston 2020	No	Yes	No	No	No	No	No	Yes	No
Escobedo 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Gowens 2018	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Grasner 2021	Yes	Yes	No	No	No	No	Yes	No	No
Greif 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Hachimi-Idrissi Coffey 2020	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	No
Hachimi-Idrissi Dobias 2020	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Hart 2020	No	No	No	No	No	No	No	Yes	No
Hodgson 2022	Yes	Yes	No	Yes	No	No	No	Yes	No
Hodroge 2020	Yes	Yes	No	No	No	Yes	Yes	Yes	No
Hughes 2021	Yes	Yes	No	No	No	No	Yes	Yes	No
Incagnoli 2019	Yes	Yes	Yes	No	No	No	Yes	Yes	No
Kimura 2019	Yes	Yes	No	No	No	No	Yes	Yes	No
Kleinman 2018	Yes	Yes	Yes	No	No	Yes	Yes	Yes	No
Le Conte 2019	Yes	Yes	No	No	No	No	Yes	Yes	No
Lott 2021	Yes	Yes	Yes	No	No	No	Yes	Yes	No
Lou 2020	Yes	Yes	No	No	No	No	No	Yes	No
Lyng 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Madar 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Maschmann 2019	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	No
Medley 2019	Yes	Yes	No	No	No	No	Yes	Yes	No
Mégarbane 2020	Yes	Yes	No	No	No	No	Yes	Yes	No
Mentzelopoulos 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Mills 2020	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Mitchell 2018	Yes	Yes	No	No	No	No	Yes	Yes	No
Munjal 2020	Yes	Yes	No	Yes	No	No	No	Yes	No
Nasr 2019	Yes	No	Yes	Yes	No	No	No	Yes	No
Nolan 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Olasveengen 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Onifer 2019	Yes	No	No	Yes	No	Yes	Yes	Yes	No
Panchal 2018	Yes	Yes	Yes	No	Yes	No	Yes	Yes	No
Panchal Berg Cabanas 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Panchal Berg Hirsch 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Perkins 2018	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Powers 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Rodriguez-Pardo 2020	Yes	Yes	No	No	No	No	No	No	No
Roquilly 2020	Yes	Yes	No	No	No	No	Yes	Yes	No
Schmidt 2019	Yes	Yes	No	No	No	No	Yes	Yes	No
Schon 2020	Yes	Yes	No	No	No	No	No	Yes	No
Schwellnus 2019	Yes	Yes	No	Yes	Yes	No	Yes	Yes	No
Semeraro 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Smith 2021	Yes	No	No	No	No	No	No	Yes	No
Soar 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Spahn 2019	Yes	Yes	No	Yes	No	No	Yes	Yes	No
Strapazzon 2018	Yes	Yes	No	No	Yes	Yes	Yes	Yes	No
Sumann 2020	Yes	Yes	No	Yes	No	No	Yes	Yes	No
Terheggen 2021	Yes	Yes	No	No	No	No	No	Yes	No
Van de Voorde 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Vanhoy 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Williams 2019	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Willmore 2020	Yes	No	No	Yes	No	No	Yes	Yes	No
Wong 2019	Yes	Yes	No	Yes	No	No	Yes	Yes	No
Yokobori 2020	Yes	Yes	No	No	Yes	No	Yes	Yes	No
Zideman 2021	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Zileli 2020	Yes	Yes	No	Yes	Yes	No	No	Yes	No
Total Meeting Criteria, n (%)	70 (93%)	68 (91%)	35 (47%)	41 (55%)	35 (47%)	31 (41%)	63 (84%)	73 (97%)	24 (32%)

Consistent with our prior systematic review, we modified the NAM criteria to include operational guidelines not otherwise described as “clinical practice guidelines,” such as the guideline by Schwellnus et al. describing methods of data recording and reporting during mass gathering sports events that are relevant to EMS quality improvement and research (27). A guideline was identified as “produced under the auspices of an association or similar entity”’ if it was described as created, sponsored, supported, or endorsed by one or more medical specialty associations; relevant professional societies, public or private organizations; government agencies at the federal, state, or local level; or health care organizations or plans (25). A guideline was determined to be based on a systematic review if explicitly performed for the guideline and stated as such, or if recommendations were explicitly based on a separate systematic review that was tied to the individual recommendations, such as in the guideline by Williams et al. (10). If not explicitly stated in the guideline document, we considered a systematic review to have been performed if all of its key elements (search strategy, study selection, synthesis of evidence, and summary of evidence synthesis) were reported and this evidence review informed the recommendations. When a guideline’s literature search identified a relevant systematic review but did not form the basis for the guideline’s recommendations, we did not consider this criterion fulfilled (e.g., guideline by Ahmed et al.) (28). In adjudicating the “presence of a synthesis of evidence from selected studies” (e.g., a detailed description or evidence tables) or a “summary of evidence synthesis” (e.g., a descriptive aggregate summary of studies or summary table), we considered these criteria fulfilled if they were present for at least one of the recommendations in the guideline, even if other recommendations in the guideline were consensus-based. Similarly, we considered a guideline to be “informed by a systematic review of the literature” if any of the recommendations were informed by a systematic review, even if other recommendations were included based on expert opinion, such as the guideline by Incagnoli et al. (29).

Each guideline was appraised further by the independent review of three investigators (KMB, REC, and CTR) across the six AGREE II domains summarized in Table 2. The AGREE II appraisal tool consists of 23 domain items, and individual appraisals were performed using the My AGREE PLUS platform (AGREE Collaboration, available at www.agreetrust.org/resource-center/agree-plus) (30, 31). We combined individual appraisals for each domain by averaging the reviewers’ scores for each item within domains and displaying this as a proportion of the maximum score available for each domain. We then averaged these totals across all domains to provide an overall proportional score for each guideline. For Domain 6 (Editorial Independence), if no technical expert panel or recommendations-developing group was specified, we assumed the recommendations were developed by the listed authors, and we determined “reporting of conflicts relevant to the guideline” based on the reported conflicts of the authors.

Table 2. Appraisal of guidelines for research and evaluation (AGREE) II assessment.

Guideline	Domain 1 Scope & Purpose	Domain 2 Stakeholder Involvement	Domain 3 Rigor of Development	Domain 4 Clarity of Presentation	Domain 5 Applicability	Domain 6 Editorial Independence	Domain Average
Adeoye 2019	73%	56%	46%	78%	52%	71%	61%
Ahmed 2019	75%	57%	55%	84%	43%	83%	63%
Atkins 2018	95%	57%	83%	95%	52%	88%	77%
Babl 2021	87%	84%	91%	95%	77%	88%	87%
Bennett 2020	87%	59%	64%	90%	55%	76%	71%
Berg 2020	90%	78%	89%	89%	61%	95%	81%
Best 2020	71%	49%	56%	78%	43%	83%	59%
Blancher 2018	67%	59%	70%	76%	33%	67%	61%
Bouzat 2020	92%	71%	58%	89%	39%	88%	70%
Charlton 2019	81%	76%	73%	92%	58%	93%	76%
Cheng 2020	86%	46%	60%	89%	40%	79%	64%
Cicero 2021	86%	57%	67%	84%	62%	55%	71%
Cook 2020	76%	54%	38%	65%	46%	76%	56%
Cottey 2019	78%	43%	58%	81%	43%	76%	61%
Craig 2020	65%	51%	39%	75%	54%	74%	57%
Doherty 2018	83%	63%	55%	79%	56%	69%	67%
Dow 2019	83%	54%	63%	84%	50%	60%	67%
Duff 2018	89%	73%	81%	87%	50%	90%	76%
Duff 2019 (BLS)	89%	73%	80%	90%	38%	74%	74%
Duff 2019 (PALS)	87%	70%	88%	90%	50%	88%	77%
Eggleston 2020	62%	57%	43%	57%	35%	67%	51%
Escobedo 2019	89%	70%	86%	90%	60%	83%	79%
Gowens 2018	76%	65%	60%	59%	49%	64%	62%
Grasner 2021	63%	59%	46%	71%	46%	81%	57%
Greif 2021	84%	81%	80%	84%	58%	76%	77%
Hachimi-Idrissi Coffey 2020	83%	63%	67%	83%	44%	88%	68%
Hachimi-Idrissi Dobias 2020	84%	59%	71%	78%	61%	79%	70%
Hart 2020	71%	43%	34%	57%	37%	71%	48%
Hodgson 2022	76%	73%	70%	75%	52%	74%	69%
Hodroge 2020	87%	51%	63%	89%	49%	67%	68%
Hughes 2021	76%	68%	58%	73%	55%	71%	66%
Incagnoli 2019	79%	63%	52%	86%	52%	74%	67%
Kimura 2019	67%	57%	48%	75%	58%	64%	61%
Kleinman 2018	86%	71%	77%	89%	56%	88%	76%
Le Conte 2019	87%	56%	60%	83%	36%	67%	64%
Lott 2021	90%	79%	80%	90%	67%	79%	81%
Lou 2020	78%	59%	39%	83%	60%	55%	63%
Lyng 2019	90%	67%	68%	70%	71%	67%	73%
Madar 2021	89%	79%	84%	89%	63%	81%	81%
Maschmann 2019	94%	78%	80%	89%	62%	71%	81%
Medley 2019	94%	68%	74%	90%	70%	76%	79%
Mégarbane 2020	86%	67%	63%	81%	45%	74%	68%
Mentzelopoulos 2021	90%	79%	88%	87%	65%	81%	82%
Mills 2020	87%	67%	79%	87%	51%	26%	74%
Mitchell 2018	78%	67%	45%	76%	40%	52%	61%
Munjal 2020	78%	57%	52%	60%	56%	64%	61%
Nasr 2019	79%	59%	65%	75%	50%	69%	66%
Nolan 2021	86%	81%	88%	89%	64%	81%	82%
Olasveengen 2021	86%	81%	85%	89%	71%	79%	82%
Onifer 2019	59%	40%	43%	78%	33%	38%	50%
Panchal 2018	86%	70%	82%	92%	61%	86%	78%
Panchal Berg Cabanas 2019	87%	75%	85%	92%	58%	86%	79%
Panchal Berg Hirsch 2019	84%	71%	80%	94%	62%	83%	78%
Perkins 2018	79%	79%	85%	94%	77%	55%	83%
Powers 2019	92%	68%	90%	92%	65%	79%	82%
Rodriguez-Pardo 2020	67%	41%	39%	71%	45%	62%	53%
Roquilly 2020	48%	44%	47%	62%	33%	48%	47%
Schmidt 2019	81%	56%	55%	83%	49%	71%	65%
Schon 2020	75%	48%	36%	63%	51%	71%	55%
Schwellnus 2019	86%	56%	62%	78%	44%	90%	65%
Semeraro 2021	83%	76%	79%	87%	74%	81%	80%
Smith 2021	70%	44%	45%	51%	57%	76%	53%
Soar 2021	83%	79%	88%	87%	67%	86%	81%
Spahn 2019	78%	59%	76%	87%	62%	81%	72%
Strapazzon 2018	81%	56%	69%	73%	51%	71%	66%
Sumann 2020	94%	67%	71%	81%	56%	74%	74%
Terheggen 2021	81%	57%	45%	71%	48%	69%	60%
Van de Voorde 2021	92%	83%	90%	90%	71%	81%	85%
Vanhoy 2019	86%	57%	79%	81%	58%	40%	72%
Williams 2019	94%	73%	80%	76%	58%	74%	76%
Willmore 2020	76%	54%	52%	59%	42%	26%	56%
Wong 2019	87%	75%	78%	90%	73%	67%	81%
Yokobori 2020	76%	51%	54%	71%	37%	60%	58%
Zideman 2021	86%	83%	89%	90%	69%	76%	83%
Zileli 2020	67%	46%	48%	70%	37%	24%	53%
AVERAGE	81%	63%	66%	81%	54%	72%	69%

Reporting

Findings are presented in accordance with the preferred reporting items for systematic reviews and meta-analyses (PRISMA; Supplementary Table 3).

Results

Guideline Search

Our search strategy yielded 1,161 articles after removal of duplicate records (Figure 1). Of these citations, we excluded two non-English records and 900 based on inclusion/exclusion criteria through screening of titles and abstracts, with substantial inter-rater agreement (kappa = 0.65). Of the remaining citations, 192 were excluded by full-text review for reasons in Figure 1. An additional nine articles meeting review criteria were added based on bibliography review and search of organizational position statements that met the inclusion criteria. Ultimately, 75 guidelines were included for appraisal (10, 27–29, 32–102).

Figure 1. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) diagram.

Guideline Topics

Almost all appraised guidelines (n = 73, 97%) addressed at least one clinical aspect within the classification of the core content of EMS medicine (Supplementary Table 4) (23). There were many guidelines addressing multiple topics, including a guideline by Zideman et al. that provided recommendations related to 20 PICO topics, which were categorized into 12 core content areas of EMS medicine (101). The most frequently addressed components of EMS medicine among included guidelines were time/life-critical conditions (n = 39, 52%), injury (n = 26, 35%), medical emergencies (n = 26, 35%), and special clinical considerations (n = 26, 35%). Few guidelines were specific to pediatric prehospital care (n = 16, 21%), and fewer than half addressed non-clinical aspects of EMS medicine (n = 33, 44%).

Guideline Appraisal

Results of the guidelines appraisal based on NAM criteria and AGREE II scoring are summarized in Tables 1 and 2, respectively. Fewer than a third of prehospital guidelines (n = 24, 32%) contained all NAM criteria for high-quality guidelines (Table 1). Most guidelines contained assessments of benefits and harms (n = 73, 97%), had systematically developed recommendations (n = 70, 93%), and were developed or endorsed by one or more associations or professional organizations (n = 68, 91%). Fewer than half (n = 35, 47%) reported performing or contained the key elements of a systematic review of the literature. Only 31 (41%) reported explicit synthesis of the evidence leading to recommendations, which was the most commonly missing NAM criterion for clinical guidelines.

Most guidelines (n = 46, 65%) had average domain scores between 50-75% (Table 2, Figure 2). Only 27 (38%) guidelines scored above 75%. Across prehospital guidelines, the highest domain scores (% of available points ± SD) were found for Domain 1 (Scope and Purpose) and Domain 4 (Clarity of Presentation), which scored at 81% (±9%) and 81% (±11%), respectively. Domain 5 (Applicability) scored the lowest at 54% (±11%). The average score across all domains was 69% (±10%). Averaged individual domain item scores are provided in Supplementary Table 5.

Figure 2. Average scores across AGREE II domains (average of three appraisers ± standard deviation).

Domain 1 – Scope and Purpose (description of objectives, questions, and population)

Domain 2 – Stakeholder Involvement (description of input gained from stakeholders including target population)

Domain 3 – Rigor of Development (description of methodology of evidence evaluation and development of recommendations)

Domain 4 – Clarity of Presentation (clarity of recommendations including options for management)

Domain 5 – Applicability (description of implementation and evaluation)

Domain 6 – Editorial Independence (description of potential conflicts from funding body or guideline development group members)

Discussion

Our systematic review identified 75 EBGs germane to prehospital emergency care published between January 1, 2018, and April 30, 2021, after or otherwise not included in the prior systematic review completed in September 2018 (19). Our findings identified more EBGs than the total number of guidelines reported in the prior systematic review (n = 71) (19), demonstrating a high level of interest and substantial effort toward creating and updating EBGs for prehospital care. Almost all guidelines (97%) provided recommendations for clinical aspects of EMS medicine with only 33 guidelines (44%) focusing on operations. The high number of prehospital EBGs identified in this interval suggests an ongoing need to periodically identify and evaluate the quality of new guidelines to support the dissemination and implementation of new evidence-based recommendations, such as on a biennial basis.

Similar to the prior systematic review of prehospital EBGs (19), we found specific deficiencies in the methodology for developing or reporting guidelines based on NAM criteria. Fewer than one third contained all elements of high-quality guidelines. Importantly, many prehospital EBGs are based on consensus recommendations and provide only limited descriptions of their literature reviews or how published scientific evidence was used to inform specific recommendations. Fewer than half stated that recommendations were based on systematic reviews of the literature or reported all the key elements of a systematic review. Commonly missed systematic review elements included search strategy and study selection, which are essential to understanding what literature was retained and being able to reproduce the review’s findings. Similarly, many guidelines did not meet the NAM reporting threshold for a “synthesis of evidence” describing individual studies (e.g., a detailed narrative description or evidence tables) or a “summary of the evidence synthesis” that aggregates individual studies and relates the evidence to the recommendations (e.g., a descriptive summary of included studies in narrative form or as summary tables). These elements are needed for readers to understand the basis and strength of recommendations and to relate the evidence supporting specific recommendations to their own EMS systems.

The AGREE II Domain 1 (Scope and Purpose) and Domain 4 (Clarity of Presentation) continued to score highly in this review compared to the previous review (19), demonstrating this is a continued strength of EMS guidelines. On the other hand, aggregate scores across Domain 2 (Stakeholder Involvement), Domain 3 (Rigor of Development), and Domain 6 (Editorial Independence) made notable improvements yet continue to be opportunities for improvement in many guidelines. In domains that had higher aggregate scores overall compared with the prior systematic review, specific items that continued to score poorly were the “incorporation of statements related to the target population’s preferences and views” (Domain 2, Item 5), and a “specific statement describing the procedure and timeline for updating the guideline” (Domain 3, Item 14) (30).

A minority of guidelines (n = 27, 38%) scored above 75% across AGREE II domains, though this compared favorably with the prior systematic review in which only 13% had scores in the top quartile. While better, this demonstrates a continued opportunity to improve the performance and reporting of key elements of guideline development that informs the EMS community. Specifically, guidelines continued to score poorly in Domain 5 (Applicability). This domain relates largely to how well the guideline addresses the ability to implement the guideline. To score highly, guidelines must describe facilitators and barriers to application of the guideline, provide implementation advice or tools, describe resource implications, and provide monitoring or auditing criteria. This finding further emphasizes the need to strengthen implementation science for prehospital EBGs, as identified in a prior systematic review of how prehospital EBGs are implemented in EMS systems (21). The low score of Domain 5 across guidelines was largely driven by a lack of monitoring or auditing criteria being included in guidelines. Examples of such content provided in the AGREE II checklist include: “Criteria to assess guideline implementation or adherence to recommendations, criteria for assessing impact of implementing the recommendations, advice on the frequency and interval of measurement, and operational definitions of how the criteria should be measured” (30). EBGs can improve in this domain by incorporating specific content to facilitate guideline implementation, and performance metrics that can be used to determine how well a guideline was implemented and if it was effective at achieving the intended goals. An example of such implementation tools can be found for the fatigue in EMS guideline by Patterson et al. (9, 103).

Our review identified additional valuable publications that did not meet our criteria for prehospital EBGs and were not retained in this review. Publications with recommendations that merely cited peer-reviewed literature but did not report organized searches of the literature were not retained. For example, the position statement by Fischer et al. did not report performing or being based on a specific organized review of the literature in developing its recommendations, but provided citations of relevant publications (104). Other articles citing relevant studies but not describing organized reviews or synthesis of the literature were also excluded (105–107). A guideline by Koenig et al. that reported performing a literature search but found no relevant published studies, with recommendations based solely on consensus opinion, was similarly excluded (108). On the other hand, we included the reviews and related position statements by Cicero et al. and Lyng et al. because they involved new organized literature reviews to inform practice recommendations contained within the position statements (42, 67). These examples highlight the challenge of defining an “evidence-based guideline” that differentiates such guidelines from review articles, editorials, or consensus statements informed by, but not based on, an organized review of the literature.

The content of prehospital EBGs published between 2018 and 2021 continues to be focused on clinical aspects of EMS medicine (Supplementary Table 4). More than half of guidelines had to do with time/life-critical conditions, largely represented by guidelines relevant to cardiopulmonary resuscitation, such as updated guidelines from the American Heart Association (33, 36, 40, 48–50, 52, 63, 80–82, 84) and the European Resuscitation Council (54, 55, 65, 68, 72, 77, 78, 83, 89, 91, 96, 101). Fewer than half of guidelines we found (n = 33, 44%) involved operational topics, such as medical oversight, quality management, research, and special operations, though this was three times as many as were found in the prior review (n = 11, 15%) (19). A variety of articles relevant to COVID-19 were screened, but ultimately not retained as there was very limited research to be reviewed early in the pandemic, thus not meeting inclusion criteria. We also identified new guidelines that at least in part addressed content relevant to operational topics and questions, including system finance (67); public health (36, 59, 78, 89); data collection, management, and analysis (27, 36, 54); epidemiology (54); and a variety of previously unaddressed special operations topics (27, 38, 44, 59, 61, 79, 87, 93, 94). Across both systematic reviews, we continue to not find prehospital EBGs addressing several core content areas of EMS medicine including assault/abuse, renal emergencies, dermatology, behavioral emergencies, geriatrics, or bariatric issues.

While all gaps in knowledge represent areas of potential focus for future guideline development, members of the PGC collaborated on a gap analysis of prehospital guidelines that identified top areas for both clinical and operational prehospital guideline development, informed by the prior systematic review (20). The prioritized list of clinical EBG gaps included airway management, care of the pediatric patient, and management of behavioral emergencies, while operational EBG gaps included defining and measuring the effects of EMS care on patient outcomes, practitioner wellness, and practitioner safety. This most recent effort, which includes the time during which this gap analysis was completed, reveals seven new guidelines relevant to airway compromise and respiratory failure and 16 guidelines focusing on pediatric patients, demonstrating these are indeed focus areas for clinical guideline development within the EMS community. A new systematic review of airway management by the Evidence-based Practice Center of AHRQ supported by NHTSA is anticipated to yield a new evidence-based guideline on airway management for EMS (109). Similarly, six new guidelines were identified relevant to EMS personnel health and wellness. On the other hand, our review identified no additional guidelines relevant to behavioral emergencies or the other high-priority areas identified in the EBG gap analysis. These findings can be used to inform the EMS community of priority areas where new guidelines have been developed and specific topics that should continue to be areas of focus for future guideline development.

Limitations

There are several limitations associated with this systematic review. Our search criteria, though refined from the prior review, missed some guidelines found later in bibliography searches. Including searches of bibliographies is a strength of this systematic review. Other guidelines may have been missed, particularly those in other medical disciplines whose titles and abstracts did not indicate any potential relevance to prehospital care. On the other hand, the agreement between screeners was substantial (kappa = 0.65) and improved from the prior review (kappa = 0.35), identifying improvements in knowledge and training within our group for identifying prehospital EBGs. Additionally, the total number of guidelines found in the 3-year span of this review compared to prior work suggests that our search methods were robust. Members of our investigative team performing guideline appraisals identified specific challenges in appraising guidelines based on the NAM criteria or AGREE II domains. Specifically, they reported that quality assessments of guidelines were often subject to reviewer interpretation. Even use of standardized criteria such as that promoted by NAM and AHRQ, or the individual items and domains of the validated AGREE II scoring tool are subject to reviewer variability. This variability may result in differing judgements and assessments of guidelines by other investigators. Criteria were particularly challenging to identify in guideline publications based on separate systematic reviews of the literature and especially where key elements were reported in supplementary material. Overall, the structured individual review, training of co-investigators, and consensus approach described herein mitigated reviewer discrepancies, though adjudication of NAM criteria by consensus review was required for many publications. Furthermore, subjectivity to the AGREE II scoring mechanism was mitigated by using an average of three independent reviewers for each individual item (following recommendations from the AGREE Collaborative, which is to have two to four reviewers rate each guideline) (31), and these were performed by reviewers with extensive experience in systematic reviews and prehospital guideline development.

Conclusions

This updated systematic review of EBGs identified many prehospital guidelines published between 2018 and 2021. We identified numerous guidelines that could inform the EMS community of evidence-based approaches to clinical care and operations. We highlight guideline quality and discuss gaps that remain for guidelines germane to prehospital care. The highest quality guidelines identified may benefit the EMS community with focused efforts supporting dissemination, implementation, and evaluation. To optimize their quality, future prehospital guidelines should ensure they incorporate and report existing criteria for high-quality guidelines, such as those promoted by NAM and by the AGREE II checklist.

Acknowledgment

The authors would like to thank Rose Turner from the Health Sciences Library System, University of Pittsburgh, for her assistance in completion of the systematic literature search.

Disclosure Statement

This work was supported through a cooperative agreement between the National Registry of EMTs (NREMT) and the Prehospital Guidelines Consortium, with funding from NREMT. KMB, REC, BTP, CTR, and PDP received support through this cooperative agreement. RMH was supported through an internal grant from the University of Cincinnati Department of Emergency Medicine. CMG declares no financial support for this work. Separate from this work, CMG reports funding from CDC, NIOSH, NIH, US Department of Defense, and Kaiser Foundation Hospitals. PDP reports funding from CDC, NIOSH, NIH, and the ZOLL foundation. REC reports funding from NIH.