Recommendations for Improving the Quality of Prehospital Evidence-Based Guidelines

Abstract

Evidence-based guidelines that provide recommendations for clinical care or operations are increasingly being published to inform the EMS community. The quality of evidence evaluation and methodological rigor undertaken to develop and publish these recommendations vary. This can negatively affect dissemination, education, and implementation efforts. Guideline developers and end users could be better informed by efforts across medical specialties to improve the quality of guidelines, including the use of specific criteria that have been identified within the highest quality guidelines. In this special contribution, we aim to describe the current state of published guidelines available to the EMS community informed by two recent systematic reviews of existing prehospital evidenced based guidelines (EBGs). We further aim to provide a description of key elements of EBGs, methods that can be used to assess their quality, and concrete recommendations for guideline developers to improve the quality of evidence evaluation, guideline development, and reporting. Finally, we outline six key recommendations for improving prehospital EBGs, informed by systematic reviews of prehospital guidelines performed by the Prehospital Guidelines Consortium.

Introduction

Guidelines that provide recommendations intended to inform clinical care are ubiquitous and increasingly being used across medical specialties (1–3). These documents aim to decrease practice variation and improve quality of care by translating evidence-based knowledge into practice (4). Numerous medical specialties have advocated for the development of more guidelines to ensure that health policy and practice are informed by formal, structured, and comprehensive reviews of pertinent peer-reviewed primary research (5). Systematic literature reviews, used to inform guidelines, summarize available evidence through methodologically rigorous processes. The proliferation of systematic literature reviews is notable (5). Hoffmann et al. identified a 20-fold increase in the number of systematic reviews published between 2000 and 2019, with an average of 80 published per day in 2019; a much higher rate of increase than primary scientific publications (6). Unfortunately, many of these reviews lack methodological rigor and are therefore of low quality (7–10). Similarly, clinical practice guidelines have been found to have varying quality and rigor, which leads end users to question the validity of guideline recommendations. This hesitancy may also extend to high-quality guidelines that are based on rigorous systematic reviews, further hindering effective and timely implementation of evidence-based guidelines (11, 12).

In 2020 and 2022, the Prehospital Guidelines Consortium (PGC) completed two overlapping systematic reviews of all identified evidence-based guidelines (EBGs) pertaining to prehospital clinical care and operations published through mid-2021. These systematic reviews identified an increasing number of guidelines relevant to EMS medicine (13, 14). Some guidelines are well known to EMS professionals and widely used to inform education and clinical protocols, such as those for cardiovascular emergencies from the American Heart Association (15) and the trauma triage guidelines from the Centers for Disease Control and Prevention (16), recently updated by the American College of Surgeons (17). While EBGs are intended to represent the highest level of recommendations based on organized reviews of the best available evidence, the true quality of published EBGs varies substantially (13, 14). The variable quality and increasing number of guidelines available to the EMS community represent challenges to dissemination and education efforts and can have negative effects on implementation (18, 19). In addition to the variable quality of existing guidelines, there remain gaps in clinical and operational topics where new guidelines are needed to inform the best possible delivery of prehospital medicine (20).

While the future of EBGs specific to EMS is not known, a trend toward increased activity in EBG development is likely. First, not all clinical or operational topics germane to EMS have been examined in detail, and many current practice recommendations are not guided by EBGs. These topics are ripe for investigation, and protocols or processes linked to these topics will undoubtedly benefit from evidence-based guidance. Second, national organizations and professional groups are increasingly supporting guideline development and assisting with dissemination of published guidelines. Finally, the ever-increasing specialization of medicine suggests that guidance developed for one specialty may not directly translate to guidance for another specialty or be readily accepted by practitioners in certain settings. This creates a need for guidelines tailored to unique sub-specialty practice environments.

Given these trends, we see an urgent need to address the variable quality of prehospital guidelines and outline strategies to improve processes for guideline development, dissemination, and implementation. The overarching goal of this paper is to stimulate improvement in guideline quality available to and created for the EMS community. We describe key elements of EBGs, outline methods available for determining the quality of guidelines, and propose concrete recommendations for guideline authors to improve the quality of evidence evaluation and guideline publication. We summarize significant findings resulting from our two systematic reviews of prehospital EBGs, informed further by the collaborative work of the PGC, a nonprofit organization with representatives from 37 national and international organizations collaborating to improve the development, implementation, and evaluation of prehospital EBGs.

What Is an Evidence-Based Guideline?

EBGs should be comprised of reviews of the best available peer-reviewed published literature and provide recommendations for clinical care of patients or operational actions in care delivery systems. When published evidence is limited, EBGs may involve reviews of grey literature, such as technical reports, non-peer reviewed literature, or unpublished works. The primary intent of EBGs is to improve quality of care and improve patient outcomes (7). The Committee to Advise the Public Health Service on Clinical Practice Guidelines from the Institute of Medicine (now the National Academy of Medicine, NAM) defined practice guidelines as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances,” referring to publications that include recommendations based on an analysis of scientific evidence and/or consensus of a panel of experts (21). While authors who embark on EBG development share a common goal (to improve care), the approaches (methodologies) and quality of reviews can vary significantly among author groups. One study found that half of clinical practice guidelines describe nonsystematic methods in developing recommendations (22). In addition, in some EBGs, recommendations are based on expert opinion alone rather than the result of a rigorous and transparent process. Furthermore, dissemination and implementation of guidelines is often challenging (19), particularly for the EMS community (18).

The NAM provides a “frame of reference” and a benchmark or standard to achieve for EBG development. In their landmark publication “Clinical Practice Guidelines We Can Trust,” the NAM expanded on the concept of practice guidelines to specifically define clinical practice guidelines as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” (23). They further summarized that “to be trustworthy, guidelines should

• be based on a systematic review of the existing evidence;

• be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups;

• consider important patient subgroups and patient preferences, as appropriate;

• be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest;

• provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of the recommendations; and

• be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.”

These “trustworthy” guidelines based on a robust process, including a systematic review of the evidence, highlight the value of high-quality evidence evaluation and a systematic and transparent approach to forming recommendations. Unfortunately, development of such guidelines can be hampered by the amount of time required to create a well-researched and vetted guideline, the need to include experts in both evidence evaluation and a specific clinical practice (often not the same individuals), and the lack of a robust evidence base. These limitations may lead authors to use less rigorous approaches for developing recommendations that, while less robust, may fill a need and equally influence practice (e.g., position statements based on expert consensus). In addition, recommendations developed in one practice setting may or may not apply to another setting, such as between the in-hospital and out-of-hospital environments (24). Given these challenges and conflicting needs for developing recommendations, even the term evidence-based guideline is used loosely in the published EMS-related literature and a consensus around its definition could be beneficial to better recognize and then stratify the quality of EBGs. These findings led the PGC to adopt a three-part definition of a prehospital EBG (Figure 1).

Figure 1. Definition of a prehospital evidence-based guideline.

Another important feature of a high-quality EBG is that recommendations result from a rigorous and careful review of the evidence. While the NAM emphasizes the importance of “systematically developed” recommendations as a key feature of “guidelines we can trust,” (23) they do not explicitly define or outline what is meant by “systematically developed” (23). The phrase implies a detailed, transparent, and reproducible process, which are all key features of rigorous systematic reviews and meta-analyses. Other structured methods for developing recommendations are possible, however, with consensus-like processes such as the Delphi approach. The lack of a clear definition or instructions for “systematically developed” recommendations challenges EBG-development teams and limits the judgment of quality by those critiquing the rigor of EBG work. Given this void, we define a “systematically developed” recommendation linked to an EBG effort as having the following components, which are informed by the NAM description:

• Recommendations are focused on specific questions or hypotheses that are answered through a review of the literature.

• A literature search is performed to identify the available scientific evidence that can inform those questions or hypotheses. Ideally, the literature search is performed using a systematic keyword search of relevant and widely known scientific databases, with details of the search described so that replication may be possible.

• A review of the available literature is performed, which may follow a more formal systematic review approach or include only a summary of the evidence in a narrative format. Optimally, a systematic review is performed, and it adheres to known steps for performing rigorous and transparent reviews of the evidence, including known and accepted methods for quality assessment.

• Ideally, an expert panel or stakeholder group representing diverse perspectives is charged with and led through a prescribed process for creating recommendations. Given that the overall body of evidence may be small, it is ideal that EBG recommendations be created only when a sizeable body of supportive evidence exists. Optimally, recommendations are informed by levels of evidence, and the guideline assigns a strength to individual recommendations.

Optimally, the evidence evaluation and development of recommendations are performed using established processes for guideline development such as the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) (25, 26) or the GIN-McMaster Guideline Development Checklist (27).

Current State of Prehospital Evidence-Based Guidelines

In the past 10 to 15 years, great progress has been made to link out-of-hospital clinical care to the best available evidence. In 2011, through leadership from the National Highway Traffic Safety Administration (NHTSA), the National EMS Advisory Council, the Federal Interagency Committee on EMS, and many other national organizations, a national model was created for developing, implementing, and evaluating EBGs for prehospital care (25). Stemming from this effort, NHTSA funded multiple efforts to create new EBGs for EMS (28–33), a process that is ongoing (3). In 2015, the National Association of EMS Physicians (NAEMSP), through a cooperative agreement with NHTSA and supplemental funding from the Health Resources and Services Administration (HRSA), published a National Prehospital Evidence-based Guidelines Strategy (34). This strategy aimed to engage EMS stakeholder organizations, institutions, agencies, and leaders in a sustainable process that promotes the development, implementation, and evaluation of EBGs for prehospital care and outlined key actions that included creation of the PGC (www.prehospitalguidelines.org).

Multiple national organizations have engaged in initiatives to support the development and implementation of prehospital EBGs. In 2016, the National Registry of EMTs expanded its requirements for education on evidence-based practice within the National Continued Competency Program to include education on EBGs (35). Similarly, NAEMSP has identified that EMS curricula must incorporate elements of evidence-based practice (36), which was also identified as a key element of the EMS Agenda for the Future (37).

Despite these collaborative efforts, the PGC has identified important limitations and critical differences in the quality of guidelines available to the EMS community (13, 14). For example, the national model process for prehospital guideline development provides a critical roadmap and structured process from guideline initiation to evaluation (25). Unfortunately, adherence to the process appears limited and details of how high-quality guidelines are implemented is lacking (18). Other medical specialties have identified similar limitations and in response, developed tools to inform EBG assessments, guide future guideline writers, and facilitate implementation (2, 4, 7, 11, 19, 38–43).

Variability in the Quality of Prehospital Guidelines

The most critical element of developing recommendations that comprise an EBG is the process used to identify and evaluate the scientific literature. Guidelines commonly provide a wide array of descriptions for how evidence was identified and evaluated, ranging from a simple statement that a guideline was based on a “literature review” to providing a detailed description of all key elements of a systematic review. Some guideline author groups have performed a systematic review of the literature as part of the same effort as developing the guideline (44), while others have based recommendations on a previously completed systematic review of the literature performed by a separate or overlapping group (32, 33). In other cases, recommendations are informed by a technical expert group, or individuals associated with evidence-based practice centers connected to the U.S. Agency for Healthcare Research and Quality (AHRQ) or other entity. These groups perform systematic reviews of the literature on topics of relevance to EMS and findings are then used for multiple projects, including those that inform separate clinical expert panels working on diverse projects (32, 33, 45).

Separating the evidence evaluation from forming recommendations based on the evidence may address the common need to engage separate groups of experts from diverse organizations or from unique specialties. This approach may alleviate the time burden of each step, yet it may lead to multiple guidelines with differing recommendations based on the same systematic review. Examples include guidelines for cardiovascular and other emergency care published by the American Heart Association (15) and the European Resuscitation Council (46), both based on systematic literature reviews by the International Liaison Committee on Resuscitation (47). Creation of separate guidelines should not necessarily be avoided. Developers of EBGs may be charged with ensuring recommendations are in-line with their own stakeholder groups and end users, or that they be applied to their unique national or regional needs. Different groups must address diverse scopes of practice, educational needs, and other operational considerations, and differences in the characteristics of the population, prevalence of diseases, and values of the community.

For the valid reasons noted above, differing approaches to guideline development are likely to continue. Furthermore, as groups balance the goals of detailed methodological reporting with creating guidelines that foster action and facilitate dissemination and education, guidelines may vary in the reporting of key elements of high-quality guidelines (14). Thus, the most important of all actions may be standardizing and optimizing how guidelines are reported in publications (peer-reviewed or other). Standardizing reporting can best assist the end user with understanding how recommendations are created and reconcile differences in process, as described above, including when guidelines should be updated with new evidence.

Any discussion of standardizing the reporting of guideline development would be incomplete without a discussion of position statements. Position statements often originate from professional organizations and are another resource that may or may not meet the definition of an EBG. It is possible that a position statement is based on a robust and systematic evaluation of the evidence, with or without a separately published resource document, and ultimately meets the NAM criteria of a high-quality EBG (48). Other position statements are created to address important changes in practice or controversial topics where expert guidance is of great value to other clinicians and where high-quality evidence is limited or non-existent. (49). More recently, as the EMS-specific scientific evidence base continues to grow, organizations like NAEMSP have performed organized reviews of scientific evidence when developing recommendations (48, 50). As these changing practices unfold and as position statements are increasingly based on more robust evaluations of scientific evidence, professional organizations should consider the key elements of high-quality guidelines when developing their recommendations, especially in the reporting of position statements.

Evaluating the Quality of Evidence-Based Guidelines

Numerous “criteria” and tools for evaluating the quality of EBGs have been published and are widely used across medical specialties (41). These resources are used to retrospectively evaluate the quality of guidelines and to prospectively support guideline developers to ensure that key elements of guideline development are reported in their publications. Within their publication “Clinical Practice Guidelines We Can Trust” (23), NAM established criteria that were adapted by AHRQ to inform what guidelines should be accepted into the National Guidelines Clearinghouse (51). Separately, in 2003 the AGREE Collaboration published the Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument, a tool comprising 23 items across six quality domains, each describing a different aspect of guideline quality (52). This tool was revised by the AGREE Next Steps Consortium to include a seven-point response scale and was evaluated for construct validity, resulting in the AGREE II tool (39). The AGREE and AGREE II tools have been endorsed by multiple organizations including the World Health Organization and used in hundreds of publications (7, 53, 54). Several other tools for evaluating clinical practice guidelines exist and have also been used extensively to evaluate guidelines across medical disciplines (Table 1) (41). As with the AGREE II instrument, while these tools have been designed primarily for retrospective guideline appraisal, they provide guideline writers information on the core components that should be considered when developing and reporting a guideline. When formatted as checklists, such as the AGREE Reporting Checklist (38), they can be incorporated into guideline publications to ensure key elements are performed and documented. These guideline evaluation and reporting tools mirror similar efforts to evaluate the quality of evidence evaluations (e.g., systematic literature reviews) that form the basis of high-quality guidelines (Table 1) (55–57). Use of these types of tools in reporting of systematic reviews is widely encouraged and journals with higher impact factors have been associated with endorsement of reporting recommendations such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (58).

Table 1. Tools and criteria used to evaluate evidence synthesis and guidelines.

Tools to evaluate guidelines
ADAPTE process
Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument
German Instrument for Methodological Guideline Appraisal (DELBI)
GuideLine Implementability Appraisal (GLIA)
National Academy of Medicine Criteria for Clinical Practice Guidelines
Tools to evaluate evidence synthesis
Assessing the Methodological Quality of Systematic Reviews (AMSTAR)
International Network of Agencies for Health Technology Assessment (INAHTA) checklist
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)

The NAM criteria were previously adapted (Table 2) and along with the AGREE II tool (Table 3) have been used to evaluate the quality of prehospital EBGs in two prior systematic reviews by the PGC (13, 14). Together, these respectively provide a categorical set of criteria by which we can identify high-quality guidelines “we can trust” and scored criteria that stratify additional details relevant to stakeholder involvement, rigor of development, implementation, and other key guideline publication elements. End users of guidelines should be educated in these key components, which can not only differentiate guidelines on similar topics based on their rigor of evidence evaluation but can assist in stratifying what specific guidelines and individual recommendations should be a focus for education and implementation efforts within EMS systems. Considering that over 30 prehospital guidelines may be published annually, it is just as important for EMS clinicians, as it is for medical directors and agency leaders, to be well-versed in methods of assessing the quality of guidelines. This parallels efforts to increase the dissemination and facilitate the interpretation of primary scientific literature across the EMS community.

Table 2. Adapted National Academy of Medicine criteria for clinical practice guidelines.

Criteria	Description
1. Systematically developed recommendations	The clinical practice guideline contains systematically developed statements including recommendations intended to optimize patient care and assist physicians and/or other health care practitioners and patients to make decisions about appropriate health care for specific clinical circumstances.
2. By an association or similar organization	The clinical practice guideline was produced under the auspices of a medical specialty association; relevant professional society; public or private organization; government agency at the Federal, State, or local level; or health care organization or plan. A clinical practice guideline developed and issued by an individual(s) not officially sponsored or supported by one of the above types of organizations does not meet the criteria.
3. Systematic review	The clinical practice guideline is based on a systematic review of evidence as demonstrated by documentation of each of the following features in the clinical practice guideline or its supporting documents.
a. Statement	An explicit statement that the clinical practice guideline was based on a systematic review.
b. Search strategy	A description of the search strategy that includes: - A listing of database(s) searched, - A summary of search terms used, and - The specific time period covered by the literature search including the beginning date (month/year) and end date (month/year)
c. Study selection	A description of the study selection that includes: - The number of studies identified, - The number of studies included, and - A summary of inclusion and exclusion criteria.
d. Synthesis of evidence	A synthesis of evidence from the selected studies, e.g., a detailed description or evidence tables.
e. Summary of evidence synthesis	A summary of the evidence synthesis (see 3d above) included in the guideline that relates the evidence to the recommendations, e.g., a descriptive summary or summary tables.
4. Assessment of benefits/harms and alternative care options	The clinical practice guideline or its supporting documents contain an assessment of the benefits and harms of recommended care and alternative care options.
5. English and to the public	The full text guideline is available in English to the public upon request (for free, or for a fee).
6. Current	The guideline is current and the most recent version.

Reproduced from Turner et al. (13).

Table 3. Appraisal of guidelines for research and evaluation (AGREE) II instrument.

Domain	No.	Item
1. Scope and purpose	1	The overall objective(s) of the guideline is (are) specifically described.
	2	The health question(s) covered by the guideline is (are) specifically described.
	3	The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.
2. Stakeholder involvement	4	The guideline development group includes individuals from all relevant professional groups.
	5	The views and preferences of the target population (patients, public, etc.) have been sought.
	6	The target users of the guideline are clearly defined.
3. Rigor of development	7	Systematic methods were used to search for evidence.
	8	The criteria for selecting the evidence are clearly described.
	9	The strengths and limitations of the body of evidence are clearly described.
	10	The methods for formulating the recommendations are clearly described.
	11	The health benefits, side effects, and risks have been considered in formulating the recommendations.
	12	There is an explicit link between the recommendations and the supporting evidence.
	13	The guideline has been externally reviewed by experts prior to its publication.
	14	A procedure for updating the guideline is provided.
4. Clarity of presentation	15	The recommendations are specific and unambiguous.
	16	The different options for management of the condition or health issue are clearly presented.
	17	Key recommendations are easily identifiable.
5. Applicability	18	The guideline describes facilitators and barriers to its application.
	19	The guideline provides advice and/or tools on how the recommendations can be put into practice.
	20	The potential resource implications of applying the recommendations have been considered.
	21	The guideline presents monitoring and/or auditing criteria.
6. Editorial independence	22	The views of the funding body have not influenced the content of the guideline.
	23	Competing interests of guideline development group members have been recorded and addressed.

Reproduced from Turner et al. (13).

Common Limitations of Existing Prehospital Guidelines

Our prior assessments of published guidelines identified limitations of existing prehospital guidelines (13, 14). Despite over 90% of the latest prehospital guidelines being produced under the auspices of medical specialty associations, professional societies, or public, private, or governmental organizations, fewer than one-third of prehospital guidelines met the NAM criteria for high-quality guidelines. Most commonly this was due to not being based on systematic reviews of the evidence, which was reported in fewer than half of guidelines. Even when systematic reviews were performed, guidelines were commonly missing key reporting elements of a systematic review. Full reporting of the systematic review process is important for the reader to know clearly and transparently what scientific evidence was included, why specific studies were excluded, and how specific studies were interpreted individually or in aggregate. These elements contribute to what the NAM identified as the most common reasons why conflicting guidelines exist; when evidence is weak, the approach to evidence evaluation differs, and differing assumptions about intervention benefits and harms influence recommendations (23). This is consistent with the wide variability in systematic review quality when evaluated outside of the context of guideline development (59). When reporting an EBG, providing a comprehensive and transparent description of the literature review methodology, including all key elements of the search strategy and evidence synthesis, is fundamental to interpretation of the guideline recommendations.

Prehospital guidelines have consistently scored lowest in the AGREE II domain of “applicability,” which primarily relates to how the guideline authors address implementation and subsequent evaluation of the use of the guideline in clinical practice, an issue that has been identified across other clinical specialties (2, 12, 19). Challenges to guideline implementation have previously been highlighted (18) and remain an important consideration for EMS systems. NHTSA and the HRSA EMS for Children program have funded several projects to evaluate implementation of guidelines, which identified several barriers to successful guideline implementation across EMS systems (60–62). Some guidelines include specific content relevant to implementation of the guideline including performance measures (32, 63), though this remains a minority of guidelines. In the most recent review of prehospital guidelines by the PGC, the lowest scoring individual AGREE II checklist items were the “inclusion of monitoring and/or auditing criteria to measure the application of guideline recommendations,” “incorporation of the target population preferences and views as part of stakeholder involvement,” and “describing the procedure for updating the guideline” (14).

Despite the availability of training materials for use of AGREE II, the judgment of individuals appraising guideline documents is subjective and likely biased. This bias is likely influenced by several different factors, many of which are unknown or undisclosed. Additionally, identifying specific guideline elements may be further limited by references to content in separate publications or online documents, especially when online links to such materials are not consistently maintained. When evaluating a guideline, it is therefore beneficial to have multiple reviewers that may both identify hard to find key elements of a guideline, and reach consensus on otherwise potentially subjective criteria. However, the best method for guideline developers to ensure that end users can identify and benefit from all key reporting elements of a guideline is to include a standardized checklist with the guideline, such as the PRISMA checklist (64) for systematic reviews that are published independently with the intent to inform guidelines, and the AGREE reporting checklist (38) for guideline publications that include systematic reviews of the literature.

Key Recommendations for Improving Prehospital Evidence-Based Guidelines

Based on the review of 146 prehospital guidelines across two systematic reviews (13, 14), using established and validated criteria for high-quality guidelines (e.g., AGREE II), and focused on specific opportunities for improving guidelines relevant to EMS medicine, we propose the following key recommendations and areas of emphasis for future prehospital EBG development and publication:

1. High-quality guidelines should be based on systematic reviews of the evidence.

   A systematic literature review may be performed and published as part of a published guideline or as an independent publication. In either form, the following key components, as recommended by NAM, should be readily identifiable in the main text, readily accessible supplementary material, or stand-alone publication:

   1. A detailed description of the search strategy, including databases used, search terms, and time period of the search. Ideally, searches are informed by one or more research questions framed in a structured format (e.g., PICO format).

   2. A description of the study selection, including the number of studies identified in searches, number of studies included, and the inclusion and exclusion criteria.

   3. A synthesis of the evidence, whereby individual studies are described in detail or via evidence tables (i.e., with study details provided and not merely included as citations for author statements).

   4. A summary of the evidence synthesis, whereby studies are summarily described in aggregate or via summary tables. This is where investigators comment on the “body of evidence” or “weight of evidence” germane to a specific structured question or recommendation.

      When based on a systematic review, high-quality methodological tools such as GRADE or a similar toolkit should be used to assess the quality of the evidence and risk of bias (26, 65). When a guideline is not based on a systematic review, such as a rapid or narrative review of the literature, inclusion of any of the key reporting elements of the literature search will strengthen the guideline by identifying how the science was identified and influenced the recommendations. Recommendations based on expert consensus should clarify what recommendations are based on scientific evidence versus expert opinion; guidelines comprising recommendations based solely on expert consensus should not be considered to meet the definition of evidence-based guidelines.

2. Guidelines should be published with standardized checklists of key reporting elements, such as the AGREE reporting checklist (38). When a guideline is based on a systematic review, a PRISMA, Assessing the Methodological Quality of Systematic Reviews (AMSTAR), or similar checklist published with the systematic review will similarly assist guideline users in identifying key reporting elements in those referenced publications (55–57). This checklist should be included or referenced in its final published format and reference back to the location or pagination of the final published guideline.

3. Guidelines should include timelines and processes for when they will be updated. Even high-quality guidelines may become outdated within 3 to 4 years after release, and half may be outdated within 6 years (66). A simple statement of when the guideline will or should be updated and the methodology for the updating procedure can benefit the longevity and implementation of the guideline. However, concrete guidance on when EBGs should be updated is limited (43), and identifying a date for updating a guideline may be challenging based on future effort limitations and the uncertainty of when new relevant scientific evidence will be published that can inform a new literature review (10, 67). As an alternative to an exact timeline for updating the guideline, developers may describe a date or timeline for when the existing evidence will be considered to decide if the guideline should be updated, such as by the periodic performance of a scoping review (68) or through expert consultation.

4. Guidelines should incorporate specific content related to guideline implementation. This includes identifying potential facilitators and barriers to using the guideline, implementation advice or tools, and a description of resource implications. Additionally, guidelines should incorporate monitoring or auditing criteria, such as by creating performance measures that can be used to determine if the guideline was implemented successfully and if it led to the outcome change that was desired. A detailed implementation plan including these elements should ideally be disseminated with any published guideline and incorporated into educational content.

5. Expert panels used in guideline development should include one or more representatives of the target population. For a clinical guideline, these individuals (e.g., representatives of EMS patients) can assist in identifying the target population’s preferences and views, an item that is commonly missing in published guidelines. This inclusion will further influence the ability to assess the benefits and harms or alternative care options relevant to specific guidance and help to inform and address potential disparities in care. Similarly, for operational guidelines, inclusion of EMS professionals as representative(s) will ensure their viewpoint is maximally considered.

6. Guidelines should ideally be published as open access to facilitate dissemination and implementation. While many scientific journals where guidelines are commonly published make material freely available one year after publication, EMS personnel, administrators, medical directors, and other end users often do not have immediate access to these journals, hampering early implementation of new guidelines. Providing guidelines, including the associated implementation advice and tools, in open access formats can ensure access to the EMS community and may improve dissemination and implementation. Guidelines that are funded through grants or similar sources should routinely budget to ensure the resulting publications are openly available to the public.

Conclusions

The rapid growth in prehospital-focused EBGs holds promise to improve care quality, safety, and clinical outcomes. However, variability in EBG rigor, transparency, and reporting demands that the developers of EBGs adhere to a consistent and transparent approach and reporting of guideline development processes. Optimally, guidelines germane to prehospital care need to adhere to a core set of steps that fosters confidence among end users, leading to rigorous dissemination and implementation. Attention to standardized criteria for the development of high-quality guidelines and use of a reporting checklist in guideline publications are among key proposed actions that can improve the quality of prehospital evidence-based guidelines.

Disclosure statement

This work follows from a systematic review of prehospital EBGs supported through a cooperative agreement between the National Registry of EMTs (NREMT) and the Prehospital Guidelines Consortium, with funding from NREMT. KMB, REC, CTR, and PDP received support through this cooperative agreement; CMG and ARP declare no financial support for this work. Separate from this work, CMG reports funding from CDC, NIOSH, NIH, US Department of Defense, and Kaiser Foundation Hospitals. PDP reports funding from CDC, NIOSH, NIH, and the ZOLL foundation. REC reports funding from NIH.

Additional information

Funding

The authors report no funding for this work.