Challenges to generating evidence-informed policy and the role of systematic reviews and (perceived) conflicts of interest

Abstract

Background: Multiple efforts to generate evidence-informed policy have attempted to teach policymakers how to understand and apply scientific research findings in their decision-making. These efforts have had limited success, because policymakers generally do not understand scientific methods.

Objective: We piloted efforts to teach policy intermediaries – specifically consumer advocacy groups – how to understand and apply health research, anticipating that they might offer such evidence to policymakers in more accessible forms.

Design: Four workshops focusing on research design and methods were conducted with consumer advocacy groups in 2010. We coded and analyzed participant responses regarding their confidence in interpreting research findings and assessments of research credibility, and the extent to which their knowledge about research findings changed after completing the workshops.

Results: Our findings suggest that although participants expressed strong interest in understanding scientific research, their ability to develop confidence about scientific research methods was limited. However, like policymakers, consumer advocacy group members intuited that financial conflicts of interest could compromise scientific findings, although they initially underestimated their effects on research results. After training, consumer advocates also saw the value of using systematic reviews rather than individual studies.

Discussion and Conclusions: Our findings suggest that although advocates may not feel confident in their understanding of scientific research, they found it easier to understand the value of systematic reviews and the risks of conflicts of interest than other statistical concepts and terminology. Focusing on making these types of information available may offer a useful way for policymakers and consumer advocates to interpret the validity of policy-relevant scientific research.

Keywords:

• Policy
• Conflict of interest
• Consumer advocacy
• Evidence-based practice
• Tobacco

Disclaimer statements

Contributors The authors acknowledge David Krauth (UCSF Department of Clinical Pharmacy) for carefully reviewing drafts of this manuscript.

Funding This work was supported by National Cancer Institute [grant CA140236]; The Flight Attendant Medical Research Institute; and the UCSF Resource Allocation Program. The funders played no role in the conduct of the research or preparation of the article.

Conflicts of interest No conflicts of interest have been declared.

Ethics approval Ethical approval was not required. The research was certified as exempt by the institutional review board of the University of California, San Francisco.

Additional information

Notes on contributors

Dorie E. Apollonio

Author information

Dr Dorie Apollonio is an Associate Professor at the University of California, San Francisco, Department of Clinical Pharmacy. Her research considers the role of scientific evidence and interest group lobbying in decision making on public health.

Lisa A. Bero

Dr Lisa Bero is a Professor at the University of Sydney and Faculty of Pharmacy and Charles Perkins Centre. Dr. Bero studies the integrity of research evidence used to influence health policy, and the manner in which evidence is communicated to key groups such as physicians, policy-makers, journalists and the community.