Research anomalies in criminology: How serious? How extensive over time? And who was responsible?

ABSTRACT

A variety of ways to detect questionable research practices in small sample social science surveys have been discussed by a variety of authors. However, some of those approaches (e.g., GRIM test, SPRITE test) do not work well for results obtained from larger samples. Here several approaches for detecting anomalies in larger samples are presented and illustrated by an analysis of 78 journal articles in the area of criminology, 59 by Dr. Eric Stewart, published since 1998 with similar methods and/or authors. Of all 59 articles, 28 (47.5%, p < .001, d = 0.94) had two or more major anomalies compared to none of the 19 control group articles. It was also found that the larger the role of Dr. Stewart in article authorship, the greater the number of anomalies detected (p < .001, d = 1.01) while for his coauthors, there were few significant relationships between their roles and total anomalies. Our results demonstrate that extensive problematic results can remain undetected for decades despite several levels of peer review and other scientific controls; however, use of two types of control groups and the use of statistical methods for measuring and evaluating anomalies can improve detection.

KEYWORDS:

• Fraud
• Professor Eric Stewart
• retractions
• anomalies
• research misconduct

Acknowledgments

The opinions, findings, and conclusions or recommendations made in this report are those of the authors, and may not reflect those of the Department of Applied Human Sciences (formerly the School of Family Studies and Human Services), the College of Health and Human Sciences (formerly the College of Human Ecology), or of Kansas State University or the National Council on Family Relations. An earlier version of this report (with a smaller number of articles assessed) was presented at the annual conference of the Society for the Improvement of Psychological Science, Victoria, British Columbia, June 22, 2020, by the first author. Appreciation is expressed to Professor Justin Pickett for his helpful insights and comments.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was not supported by any external or internal funding.