Sharing successes and hiding failures: ‘reporting bias’ in learning and teaching research

ABSTRACT

When researchers selectively report significant positive results, and omit non-significant or negative results, the published literature skews in a particular direction. This is called ‘reporting bias’, and it can cause both casual readers and meta-analysts to develop an inaccurate understanding of the efficacy of an intervention. This paper identifies potential reporting bias in a recent high-profile higher education meta-analysis. It then examines a range of potential factors that may make higher education learning and teaching research particularly susceptible to reporting bias. These include the fuzzy boundaries between learning and teaching research, scholarship and teaching; the positive agendas of ‘learning and teaching’ funding bodies; methodological issues; and para-academic researchers in roles without tenure or academic freedom. Recommendations are provided for how researchers, journals, funders, ethics committees and universities can reduce reporting bias.

KEYWORDS:

• Reporting bias
• publication bias
• research synthesis
• academic freedom
• Scholarship of Teaching and Learning

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 ‘d’ is an effect size referring to standardised mean difference. This is commonly calculated by subtracting the sample mean of the treatment group from the sample mean of the control group, and then dividing by the pooled standard deviation (Cohen 1988).