On the sidelines of what works: scientifically based indifference

ABSTRACT

A team of systematic reviewers successfully completed a government-commissioned review of ‘what works to improve post-school outcomes for youth with disabilities’ in 2012. Despite its success, interviews with 10 review team members revealed dissatisfaction with the process and indifference to its outcomes. The purpose of our analysis was to examine how the systematic review process itself led to review team members’ feelings of indifference, resignation, and pessimism. Drawing on the writings of Henry Giroux, Gert Biesta, and Hanna Arendt that warn of the death of democracy and the rise of totalitarianism, we explored how the systematic review certification process, examinations, rules, and structures deadened democratic deliberation and critique necessary, we argue, to conducting good educational science. We end with a call for systematic reviews in education whose researchers, products, and processes remain ethically oriented to keeping democracy alive.

KEYWORDS:

• Critical theory
• democracy
• education
• research methodology
• systematic reviews

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Two of the 12 team members were not interviewed because they could not be reached or were unable to schedule a convenient time for the interview. The first author also sought to include systematic review project documents (e.g. meeting minutes, report drafts and email correspondence) in the analysis. Since these documents were produced under a government subcontract, and, therefore, owned by the government, she needed government permission to use them for research purposes. Her request was denied by a representative of the IES in July 2013. Therefore, all information about the systematic review process is sourced from interviews with the review team members and, on occasion, the first author’s memory.

2 All review team members who participated in the interviews are referred to as Reviewers 1–10. This is to protect the identities of team members who would be readily identifiable by their titles (e.g. PI, Statistician).

Additional information

Funding

This research was supported by a New Researcher grant from the University of South Florida awarded to the first author in May, 2013.