Methodology for Evidence Evaluation and Reporting of the NAEMSP Trauma Compendium Position Statements

Abstract

Background

Position statements from national organizations commonly vary in methodology for the evaluation of existing literature and the development of recommendations. Recent national recommendations have highlighted important components for evidence-based guidelines that can be feasibly incorporated in the creation of position statements and their resource documents. We describe the methodology developed to guide the creation of a compendium of 16 trauma-related position statements led by NAEMSP and partner organizations.

Methods

Each position statement group developed trauma-related topic areas, primarily guided by the Population, Intervention, Comparison, and Outcome (PICO) framework. A structured literature search comprised of search terms aimed to identify relevant EMS and trauma-related scientific publications was performed for each topic area. Resource documents for each position statement included a description of the literature considered in forming recommendations, reported through evidence tables and a narrative description of the available literature. Where evidence was limited, consensus-based recommendations were developed using content experts and reviewed by the NAEMSP Standards and Clinical Practice Committee.

Conclusion

We report a standardized methodology for literature review and development of recommendations as part of a compendium of trauma-related position statements from NAEMSP and partner organizations. This methodology can serve as a template for future position statements with ongoing refinement.

Introduction

Guidelines, position statements, and other publications providing recommendations for clinical care or operations are increasingly being published in the medical literature (1). Improving guideline development for EMS has been a specific interest of the National Association of EMS Physicians (NAEMSP) and partner organizations for over a decade (2). Two systematic reviews by the Prehospital Guidelines Consortium (PGC) identified a combined 146 prehospital evidence-based guidelines, including some position statements meeting the PGC’s criteria for evidence-based guidelines (1, 3). However, these investigations identified that the quality of evidence evaluation and reporting, as well as the way recommendations are developed, are highly variable across publications aimed to guide EMS clinicians and administrators.

Position statements from medical associations can be a useful contribution to the literature. Some position statements are informed by systematic literature reviews and meet criteria for a high-quality evidence-based guideline. Others, however, are written with unstructured literature reviews without keyword searches, or are guided only by expert consensus. Identified literature may not be described in detail, such as through evidence tables, drawing questions to the evidence base for specific recommendations. Furthermore, position statements rarely have funding for systematic reviews, focusing instead on rapid publication of guidance rather than comprehensive literature-based recommendations. As a result, position statements vary substantially in the level of evidence that supports them.

For its trauma compendium, members of the NAEMSP Standards and Clinical Practice Committee developed a templated evidence review methodology aimed at optimizing the consistency of evidence review across these papers. The goal of this templated process was to incorporate key components of evidence-based guidelines in forming recommendations, while balancing the need for rapid development (1, 4).

Methods

This guidance for developing position statements and resource documents was adapted from the 23 items of the Appraisal of Guidelines for Research and Evaluation (AGREE) II reporting checklist for clinical practice guidelines (5). We also incorporated key reporting elements of high-quality evidence-based guidelines outlined by the National Academy of Medicine (NAM; formerly the Institutes of Medicine) and the Agency for Healthcare Research and Quality (AHRQ), as summarized by Turner et al., following recommendations from the PGC (3, 4). The AGREE II instrument is a validated tool supported by multiple international organizations designed to evaluate the quality of evidence-based guidelines. Its reporting checklist has been suggested as a reference tool to inform guideline developers of key criteria that should be incorporated when developing and reporting recommendations within guidelines or position statements (4). While incorporation of all 23 items of the AGREE II instrument is not typical or practicable for most position statements, we used essential components for developing high-quality guidelines to identify key elements for resource documents that inform recommendations within position statements (Figure 1). We provided additional recommendations for the optimal reporting of position statements and resource documents (Figure 2).

Figure 1. Framework for the development of a position statement and resource document.

Figure 2. Additional recommendations for the reporting of position statements and resource documents.

Scope and Purpose

The NAEMSP Standards and Clinical Practice Committee established an editorial board for the trauma compendium comprised of experts in emergency medical services, emergency medicine, pediatrics, pharmacology, obstetrics, and trauma. The editorial board included an editor-in-chief (JWL) and methodology lead (CMG) that assisted all project teams, along with an editor assigned to each position statement group. The editorial board, with input from the committee, established a list of 16 topic areas for position statements that would comprise the trauma compendium (Table 1). Areas of focus included patient population-specific topics, injury and intervention-specific topics, a focus on out-of-hospital traumatic arrests, and a focus on clinically driven systems-based trauma management principles. Concurrent or recent work on several topic areas that were considered by the editorial board as important to include in a comprehensive trauma compendium had been recently published by other entities and were thus excluded, such as the management of field triage of trauma patients, external hemorrhage control, trauma airway management, pain management, and management of traumatic brain injury (6–12).

Table 1. Position statement topic areas for the trauma compendium.

Position Statement Topic

Antibiotic Use for Open Fractures

Blood Product Administration

Entrapment and Crush injury

Femur Fracture Care

Fluid Management

Geriatric Trauma Care

Medically Directed Rescue

Pediatric Trauma Care

Pelvic Fracture Care

Pneumothorax Assessment and Management

Obstetrical Trauma Care

Spinal Injury Management

Termination, Withholding Resuscitation, and Palliation, Adult

Termination, Withholding Resuscitation, and Palliation, Pediatric

Tranexamic Acid Use

Vasopressor Use

Author Group Composition and Stakeholder Involvement

The trauma compendium editorial board solicited volunteers from the NAEMSP Standards & Clinical Practice Committee to serve as contributing authors for the various topics of interest. Lead authors were selected from the pool of interested volunteers and were briefed by the editor-in-chief and the methodology lead regarding the scope and purpose of the project. The overall framework (Figure 1) and reporting recommendations (Figure 2) for the development of the position statements and resource documents was reviewed by the methodology lead with each project team. When previous work on specific topics had been published by NAEMSP in conjunction with other collaborating medical associations the editor-in-chief, with the approval of the NAEMSP Board of Directors, engaged with leadership at those respective associations to request their nominations for collaborating authors. Lead authors for each topic area were encouraged to solicit additional subject matter experts from other sources to ensure their workgroups were well rounded.

Literature Search

The literature searches and evidence evaluations for individual position statements were guided by a rapid review framework. Rapid reviews use key elements of a systematic review approach while simplifying or omitting certain components to complete the evidence evaluation in a timely manner or when there is limited time or resources for completing traditional systematic reviews (13–15). Modifications may include reducing the number of databases searched or streamlining the evidence evaluation process.

Each literature search began with the development of research questions or subtopic areas, favoring the Population, Intervention, Comparison, and Outcome (PICO) format when applicable. PICO questions or subtopic areas were outlined using a standardized form (Supplemental Figure) and each team was ultimately encouraged to complete a structured literature search and evidence evaluation to form systematic recommendations. Relevant PICO questions or subtopic areas were available for an open comment period by members of the NAEMSP Standards and Clinical Practice Committee and the trauma compendium editorial board prior to initiation of the literature search.

Project teams performed a structured keyword search in one or multiple databases, with or without the use of a research librarian, based on the anticipated breadth of available literature and as determined by consensus of each working group with input from the editorial board. All literature searches were guided by a common search strategy for the identification of EMS-related literature (Table 2), guided by the works of Olaussen et al. (16), Burgess et al. (17), and Martin-Gill et al. (1). Each group was provided and asked to modify a proposed list of trauma-related search terms (Table 3) to address individual topics. For example, population-specific restrictors (pediatrics, geriatrics, obstetrics, etc.) were used to limit the literature searches for position statements focused on those populations.

Table 2. Proposed search terms for EMS-related literature.

	Search Terms
MESH Terms	Emergency Medical Dispatch [Mesh] Emergency Medical Dispatcher [Mesh] Emergency Medical Service Communication Systems [Mesh] Emergency Medical Technicians [Mesh] Transportation of Patients [Mesh]
Text Search	air medical [Title/Abstract] air medicine [Title/Abstract] aeromedical [Title/Abstract] aero-medical [Title/Abstract] aeromedicine [Title/Abstract] aero-medicine [Title/Abstract] ambulance* [Title/Abstract] emergency medical service* [Title/Abstract] emergency medical technician* [Title/Abstract] emergency services [Title/Abstract] ems [Title/Abstract] emt [Title/Abstract] field triage [Title/Abstract] first responder* [Title/Abstract] paramedic* [Title/Abstract] prehospital [Title/Abstract] pre-hospital [Title/Abstract]

Table 3. Proposed search terms for trauma-related literature.

	Search Terms
MESH Terms	Advanced Trauma Life Support Care [Mesh] Wounds and Injuries [Mesh]
Text Search	concussion* [Title/Abstract] contusion* [Title/Abstract] fracture* [Title/Abstract] injury [Title/Abstract] injuries [Title/Abstract] trauma* [Title/Abstract] wound* [Title/Abstract]

Literature was screened by title and abstract, followed by full-text review. When relevant, publications were identified through bibliography searches or other methods as determined by each writing group and then incorporated. The number of reviewers and method of adjudicating conflicts was determined by each position statement group based on the amount and type of literature identified for screening, as described in each resource document.

Evidence Evaluation

Retained publications were reviewed by the writing groups with key data extracted into evidence tables organized by research question and/or subsequent recommendation. Aggregate data were then summarized in a narrative evidence synthesis. As part of the rapid review framework, risk of bias assessments and grading of the quality of the literature were deferred, except as narratively described in each resource document. Common reporting elements included a PRISMA diagram to illustrate the evidence evaluation process, which can be found in each respective resource document.

Development of Recommendations

An expert panel for each writing group was identified to draft recommendations informed by the literature review. When evidence was limited or altogether absent, recommendations were guided by expert consensus and labeled as such. In some cases where evidence was lacking or equivocal, the author groups could choose to make no recommendation, sometimes highlighting these areas as topics of import for future research.

Position statements were reviewed by the entire NAEMSP Standards and Clinical Practice Committee, comprised of physician and professional (front-line clinician) members of NAEMSP, with amendments suggested as pertinent to each writing team. Draft resource documents were made available to the committee, with the resource documents themselves undergoing standard journal peer review. When partner organizations were engaged in writing these documents, those entities also passed the manuscripts through their own internal processes for approval. There was no dedicated community consultation or other stakeholder review required for each position statement. The final position statements were reviewed and approved by the NAEMSP Board of Directors.

Discussion

In this manuscript, we describe the methodology used in the development of a trauma-related compendium of 16 position statements led by NAEMSP. Key components of high-quality guidelines were incorporated and adapted for the creation of position statements and resource documents. The methodology described for this compendium represents an improvement in the varied literature analysis that has been previously incorporated in position statements of many national organizations, including NAEMSP. Performing a structured and comprehensive literature search using well-described search terms is essential to ensuring that most literature relevant to a topic is adequately considered prior to making recommendations on behalf of a national organization. While consensus and expert-driven recommendations are often necessary in position statements due to a lack of scientific evidence, a literature review is indicated prior to making this assertion. Similarly, incorporating a description of the individual literature that informs recommendations has been previously incorporated in position statements or their resource documents, but has not been universally applied. This description of the evidence, optimally through evidence tables and even in narrative format, is essential for individuals to understand both the literature that is currently available on a topic and how that literature was used in framing recommendations. This is a substantial improvement over simply reporting that a group of experts reviewed the literature and reached consensus on a topic.

The optimal process for reviewing and reporting the literature supporting position statements is dynamic and evolving. Necessarily, this process needs to be tailored to the topic of interest, the resources available to the organization(s), and the timeliness of developing the recommendations. For example, even when the resources needed for a systematic literature review are available, it may be too time-intensive for the development of time-critical recommendations, such as during a pandemic response. Similarly, a scoping review of the literature may identify if sufficient literature exists to inform a position statement or if a purely expert consensus-based approach is necessary. The approach herein described for the trauma compendium may serve as a model for future NAEMSP work, with refinements and improvements anticipated to be made by the organization and its members on a continuous basis.

This unified methodology for the trauma compendium has several important limitations. Though guided by prior investigations aimed at identifying the optimal search strategy for EMS-related scientific literature, relevant prehospital trauma-related literature may have been missed (1, 16, 17). Involving a research librarian at the discretion of the statement authors may have limited the primary research included in the rapid reviews that did not involve a research librarian. Similarly, individual search terms for each position statement (described in each resource document) may have been too restrictive or otherwise missed relevant studies. Authors were encouraged to include additional works from bibliography searches or with the input of content experts, and non-EMS specific literature from other relevant practice environments (e.g., emergency medicine, trauma, and critical care). Our rapid review methodology also has limitations compared to a systematic review approach, including not performing a grading of the quality of evidence. This limitation is balanced by the comparative improvement represented by undergoing a structured evidence evaluation with inclusion of evidence tables versus the common consensus-only approach or more limited literature reviews with inadequate reporting often encountered in other position statements. Individual statement authors were asked to describe how their rapid review differed from a comprehensive systematic review for transparency. Our methodology also incorporated additional key aspects for the reporting of evidence-based guidelines identified from the AGREE II reporting checklist and in recommendations from the PGC (4, 5). Finally, future position statements could be enhanced by incorporating performance or quality measures with recommendations.

Conclusions

A unified methodology comprising a structured search of the literature and evidence evaluation was developed for the compendium of trauma-related position statements and resource documents created by NAEMSP and partner organizations. This methodology may serve as a model for future position statements, with a focus on balancing a need for rigorous and well-reported rapid reviews of the literature with practical limitations inherent to developing timely recommendations informed through expert guidance.

Disclosure statement

CMG reports he is the President of the Prehospital Guidelines Consortium but has received no financial support from this organization.