Reconsidering the ‘idea’ of evidence in evidence-based policy and practice

Abstract

Evidence-based policy and practice (EBP) has become an important social work conceptual framework. Yet, the core EBP concept, the concept of evidence, remains ill-defined. I propose a modification of the concept of evidence as applied to EBP effectiveness questions. As a basis for this reformulation ideas about evidence are examined from cross-disciplinary and multidisciplinary perspectives including epistemology, philosophy of science, evidence-science, and law. I propose that for EBP effectiveness questions: (1) to be considered ‘relevant evidence’ an explanatory connection between an intervention and an outcome must be established rather than a mere association; (2) the EBP definition of ‘best available evidence’ should include total available evidence (rather than a subset) about effectiveness, causal roles (i.e., mechanisms), and support factors and be inclusive of high-quality experimental and observational studies as well as high-quality mechanistic reasoning; (3) the familiar five-step EBP process should be expanded to include formulation of warranted, evidence-based arguments and that evidence appraisal be guided by three high level criteria of relevance, credibility, and strength rather than rigid evidence hierarchies; (4) comparative effectiveness research strategies, especially pragmatic controlled studies, hold promise for providing relevant and actionable evidence needed for policy and practice decision-making and successful implementation.

Die evidenzbasierte Politik und Praxis (EBP) bildet in der Sozialarbeit einen wichtigen Begriffsrahmen. Der Kernbegriff der EBP, der Begriff Evidenz, ist jedoch immer noch ungenau definiert. Ich schlage eine Modifizierung des in EBP-Effektivitätsfragen benutzten Begriffs Evidenz vor. Als Grundlage dieser Neuformulierung dient eine Untersuchung der Evidenz-Konzepte aus bereichsübergreifender und interdisziplinärer Perspektive unter Einschluss der Epistemologie, Wissenschaftsphilosophie, Evidenzwissenschaft und der Gesetzgebung. Ich schlage folgendes für EBP-Effektivitätsfragen vor: (1) als ‘relevante Evidenz’ sollte gelten, wenn eine selbsterklärende Verbindung zwischen einer Intervention und einem Ergebnis existiert und nicht nur einfache Assoziation; (2) die EBP-Definition der ‘besten verfügbaren Evidenzen’ sollte die gesamten verfügbaren Evidenzen (keine Teilmenge) über Effektivität, kausale Rollen (d. h. Mechanismen) und Unterstützungsfaktoren beinhalten sowie erstklassige Experimentier- und Beobachtungsstudien und anspruchsvolle mechanistische Begründungen; (3) der bekannte fünfstufige EBP-Prozess sollte um die Formulierung einer begründeten, evidenzbasierten Argumentation erweitert werden, und die Beweiswürdigung sollte sich durch drei strenge Kriterien steuern lassen: Relevanz, Glaubwürdigkeit und Stärke anstatt feststehender Evidenzhierarchien; (4) Forschungsstrategien zur vergleichenden Bewertung der Effektivität, besonders pragmatische kontrollierte Studien, sind vielversprechend bei der Schaffung von relevanten und umsetzbaren Evidenzen, die für Entscheidungsfindung und erfolgreichen Einsatz in Politik und Praxis erforderlich sind.

Keywords:

• evidence-based practice
• evidence-based policy
• evidence-based medicine
• evidence-based social work
• evidence
• philosophy of evidence-based practice

Schlüsselwörter:

• Evidenzbasierte Praxis
• evidenzbasierte Politik
• evidenzbasierte Medizin
• evidenzbasierte Sozialarbeit
• Evidenz
• Philosopie der evidenzbasierten Praxis

Acknowledgments

I thank Professor Peter Achinstein for his helpful suggestions in early email correspondence as well as his willingness to review the first draft of this manuscript. I also thank Professors Jennifer Vick Bellamy, Haluk Soydan, and Mike Fisher for their insightful comments on early drafts of this manuscript. I am grateful to the coeditors of this special issue, Walter Lorenz and Ian Shaw as well as to the two manuscript reviewers for their most helpful suggestions. The helpful comments of Gerald Hanley throughout has made this manuscript a bit more reader friendly. This manuscript is April 2014, Free University of Bozen-Bolzano, Bozen/Bolzano, Italy.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes on contributor

Edward J. Mullen is Willma and Albert Musher Professor Emeritus, Columbia University School of Social Work. He is Editor-in-Chief, Oxford University Press Bibliographies in Social Work and Member of the American Academy of Social Work and Social Welfare.

Notes

1. I assume readers already are familiar with the meaning of EBP so in this paper my comments focus on suggested modifications only. For those wishing information about the basic fundamentals of EBP in the context of social work (see E.J. Mullen, 2010a, 2010b, 2010c).

2. In addition to effectiveness questions EBP includes questions about prevention, diagnosis or assessment, prognosis or expectations of possible outcomes, etiology or causation, and harm or negative effects that might occur, among others.

3. These examples are described in Achinstein (2010, pp. 8–13).

4. Achinstein writes:

when speaking of evaluating explanations, I focus on illocutionary evaluations. These are ‘strongly pragmatic’ because they are concerned with determining whether E is a good explanation for an explainer to give in explaining q to some type of audience. And whether this is the case will vary with explainer and audience. There is, however, another important kind of evaluation, which I introduced in chapter 1. Its aim is to determine whether an explanation is a correct one, which may be the case even if it is not a good explanation for a certain type of explainer to give to a certain type of audience. … Such an explanatory connection exists if and only if either h correctly explains e, or e correctly explains h, or something correctly explains why both e and h are true. Whether the latter obtains is not pragmatic in the strong sense. Its truth does not depend upon who, if anyone, is explaining what to whom. (2010, p. 142)

5. ‘From any premises that don't need further serious support—like those that are self-evident or that you just see by looking or are already well established or attested to by a reliable expert—draw roots into the ground’ (Cartwright & Hardie, 2012, p. 16).

6. Unlike the multiple regression case wherein several independent variables can be shown to contribute to a dependent variable's value, in the case of INUS conditions the support variables must be present for the policy or intervention to have a measure effect.

7. It is of interest to note that in the Spoth et al. study two brief interventions were implemented and contrasted with a minimal contact control condition, a 5-session Preparing for the Drug Free Years Program, and a 7-session Iowa SFP. Both active interventions were found to be significantly more efficacious than the control condition, and not significantly different from one another in outcomes achieved. This may serve as a good example of the INUS condition described above suggesting that each program may be a member of different causal clusters (Spoth et al., 2001).

8. See program website: http://www.extension.iastate.edu/sfp10–14/

9. Adapted with permission from EBBP.org at www.ebbp.org.