Defining the benefits and challenges of stakeholder engagement in systematic reviews

Abstract

Background:

Although there is a growing literature on the process of engaging stakeholders in medical research, there are a lack of clearly-defined measures for reporting and evaluation, which limits the ability to learn from past experience, understand the effectiveness of engagement, or identify which approaches work best. Clearly defining the benefits and challenges of engaging stakeholders in the systematic review process is an integral first step toward developing a set of criteria that can be used to evaluate the impact and effectiveness on the conduct, quality, and dissemination of systematic reviews.

Methods:

We utilized two complementary approaches to examine the benefits and challenges of engaging stakeholders in the systematic review process: 1) a literature scan to understand the overall state of the field; and 2) a series of key informant interviews with systematic reviewers, program/policy officials, and stakeholders.

Results:

We identified six main expected benefits and five primary challenges of involving stakeholders in systematic reviews. Benefits included: establishing credibility; anticipating controversy; ensuring transparency and accountability; improving relevance; enhancing quality; and increasing dissemination and uptake of findings. Challenges included: time; training and resources; finding the right people; balancing multiple inputs; and understanding how to match the right type of stakeholder to the right time in the systematic review process.

Discussion:

The results of this study are an important first step toward developing mechanisms for evaluating the effectiveness of stakeholder engagement in systematic reviews. Future work should seek both to verify the appropriateness of these benefits and challenges and identify concrete criteria for evaluating the effectiveness of different methods, timing, and intensity of stakeholder engagement.

Keywords::

• stakeholder
• medical research
• key informants
• AHRQ
• Effective Health Care Program
• evidence based medicine

Acknowledgments

This project was funded under Contract 290-2012-00004-C from the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services.

Author contributions

EKC conducted all interviews, participated in conception and design of the project, analyzed and interpreted project material, and drafted the manuscript. EPW helped analyze and interpret project material and critically revised the manuscript. EK helped analyze and interpret project material and critically revised manuscript. SU helped analyze and interpret project material and critically revised the manuscript. SB helped analyze and interpret project material and critically revised the manuscript. CC helped analyze and interpret project material and critically revised the manuscript. TH helped analyze and interpret project material and critically revised the manuscript. DOM helped analyze and interpret project material and critically revised the manuscript. HN helped analyze and interpret project material and critically revised the manuscript. KAR helped analyze and interpret project material and critically revised the manuscript. MM helped analyze project material and helped to draft the manuscript. JA helped analyze project material and helped to draft the manuscript. RAP helped analyze and interpret project material and critically revised the manuscript. JMG participated in conception and design of the project and helped draft the manuscript. All authors read and approved the final manuscript.

Disclosure

The author reports no conflicts of interest in this work. This paper is based on a Research White Paper produced for the Agency for Healthcare Research and Quality (AHRQ), and was posted to the AHRQ EHC Program Web site on March 18th 2014. The authors of this document are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.