Science in peril: the crumbling pillar of peer review

Click to increase image sizeClick to decrease image size

Notes

1 Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99 (4), 178-182. And see: Benos, D.J., Bashari, E., Chaves, J.M., Gaggar, A., Kapoor, N., LaFrance, M., Mans, R., Mayhew, D., McGowan, S., Polter, A., Qadri, Y., Sarfare, S., Schultz, K., Splittberger, R., Stephenson, J., Tower, C., Walton, G., & Zotov, A. (2007). The ups and downs of peer review. Advances in Physiology Education, 31, 145-152. Huisman, J., & Smits, J. (2017). Duration and quality of the peer review process: the author’s perspective. Scientometrics, 113 (1), 633-650.

2 See e.g., Hancock, P.A. (2019). In praise of civicide. Sustainable Earth, 2 (8), 1-6. Hancock, P.A. (2019). The humane use of human beings. Applied Ergonomics, 79, 91-97. Hancock, P.A. (2020). The humanity of humanless systems. Ergonomics in Design, 28 (3) 4-6. De Winter, J., & Hancock, P.A. (2021). Why human factors science is demonstrably necessary: Historical and evolutionary foundations. Ergonomics, 64 (9), 1115-1131.

3 And see e.g.,: Moreno-Fernández, M. M., Ramos-Álvarez, M. M., Valdés-Conroy, B., & Catena, A. (2008). Criteria of the peer review process for publication of experimental and quasi-experimental research in Psychology: A guide for creating research papers. International Journal of Clinical and Health Psychology, 8 (3), 751-764.

4 e.g., Dempsey, P.G., Wogalter, M.S., & Hancock, P.A. (2000). What’s in a name? Using terms from definitions to examine the fundamental foundation of Human Factors and Ergonomics science. Theoretical Issues in Ergonomic Science, 1 (1), 3-10. Hancock, P.A. (2022). Machining the mind to mind the machine. Theoretical Issues in Ergonomic Science, in press. Hancock, P.A. (2022). How Human Factors and Ergonomics saves lives. Applied Ergonomics, 98, 103585.

5 Kassirer, J.P., & Campion, E.W. (1994). Peer review: Crude and understudied, but indispensable. Journal of the American Medical Association, 272 (2), 96-97. Jefferson, T., Alderson, P., Wager, E., & Davidoff, F. (2002). Effects of editorial peer review: a systematic review. Journal of the American Medical Association, 287 (21), 2784-2786.

6 see Hancock, P.A., & Hoffman, R.R. (2015). Keeping up with intelligent technology. IEEE Intelligent Systems, 30 (1), 62-65.

7 Tight, M. (2010). Are academic workloads increasing? The post‐war survey evidence in the UK. Higher Education Quarterly, 64 (2), 200-215, cf., Soliman, I., & Soliman, H. (1997). Academic workload and quality. Assessment & Evaluation in Higher Education, 22 (2), 135-157. It is by no means the claim here that Faculty are unique in respect of the never-ending pursuit of greater effort for lesser remuneration by a system specifically oriented to that purpose. Rather, the proximal point is that non-obligatory tasks are readily ‘shed’ as obligatory requirements occupy more and more of the time available.

8 Daruka, I. (2014). Confined to grow?-Publication dynamics and the proliferation of scientific journals. Europhysics News, 45 (1), 19-22. And: Goel, R.K., & Faria, J.R. (2007). Proliferation of academic journals: effects on research quantity and quality. Metroeconomica, 58 (4), 536-549.

9 Dekker, S.W.A., Hancock, P.A., & Wilkin, P. (2013). Ergonomics and the humanities: Ethically engineering sustainable systems. Ergonomics, 56 (3), 357-364.

10 Beall, J. (2016). Essential information about predatory publishers and journals. International Higher Education, (86), 2-3.

11 Hancock, P.A., & Warm, J.S. (1989). A dynamic model of stress and sustained attention. Human Factors, 31, 519-537.

12 It can obviously be argued that the Editor constitutes at least one other set of eyes on the submission. However, the pressure on the latter individuals may induce them to ‘go with the Reviewer.’ After all, an Editor, even of a medium size Journal cannot diligently review every article that is submitted.

13 Ewing, J. (2006). Measuring journals. Notices of the AMS, 53 (9), 1049-1053. This can be an especially challenge when trying to make assessments of the quality and impact of an individual who publishes in a wide range of journals, some of which are relatively new and difficult to assess.

14 See e.g., Parker, M. (2013). The ethics of open access publishing. BioMed Central: Medical Ethics, 14 (16), 1-4.

15 The present work has itself fallen prey to this very strategy. The following response was received: “Thank you for submitting your manuscript "Science in Peril: The Crumbling Pillar of Peer Review." I am sorry to inform you that we cannot recommend your manuscript for (publication). Space in our Perspectives section is extremely limited, and most articles are invited, leaving very little room for uninvited contributions. In the context of other articles under consideration we did not find your inquiry to be competitive, our recommendation is that you seek publication in a more specialized journal.” Since the submission is manifestly about the whole of science, it is hard to see why the recommendation to a ‘specialized journal’ is offered, except as a standard pabulum for a journal which, in actuality, cannot now claim itself to be fully peer-reviewed! And see: Jefferson, T., Wager, E., & Davidoff, F. (2002). Measuring the quality of editorial peer review. Journal of the American Medical Association, 287 (21), 2786-2790.

16 The propensity is then to create self-isolated ‘echo chambers’ in which the permeation of the chamber walls becomes ever less achievable. An example from global warming is given in: Walter, S., Bruggeman, M., & Engesser, S. (2018). Echo chambers of denial: Explaining user comments on climate change. Environmental Communication, 12 (2), 204-217.

17 Hancock, P.A. (2019). The humane use of human beings. Applied Ergonomics, 79, 91-97.

18 But see: Kaplan, A.D., Kessler, T.T., Brill, J.C., & Hancock, P.A. (2022). Trust in Artificial Intelligence: Meta-analytic findings. Human Factors, in press. Hancock, P.A. (2022). Advisory adumbrations about autonomy’s acceptability. Human-Computer Interaction, 37 (3), 263-280. Hancock, P.A. (2014). Automation: How much is too much? Ergonomics, 57 (3), 449-454. Salmon, P.M., Carden, T., & Hancock, P.A. (2021). Putting humanity into inhuman systems: How Human Factors can be used to manage the risks associated artificial general intelligence. Human Factors Ergonomics in Manufacturing and Service Industries, 31 (2), 223-236.

19 Hancock, P.A. (2017). Imposing limits on autonomous systems. Ergonomics, 60 (2), 284-291. And see: Hancock, P.A. (2022). Avoiding autonomous agents’ adverse actions. Human-Computer Interaction, 37 (3), 211-236.

20 Li, Y. (2013). Text-based plagiarism in scientific publishing: issues, developments, and education. Science and Engineering Ethics, 19 (3), 1241-1254. And see: Suls, J., & Martin, R. (2009). The air we breathe: A critical look at practices and alternatives in the peer-review process. Perspectives on Psychological Science, 4 (1), 40-50. Of course, there remains a wide gap between a flawed design and analysis and an intentional attempt at plagiarism. The point here is that pattern-matching (widely writ) can accomplish both of the tasks to hand.

21 Braga, A., & Logan, R.K. (2017). The emperor of strong AI has no clothes: limits to artificial intelligence. Information, 8 (4), 156. And see: Hancock, P.A. (2022). Advisory adumbrations about autonomy’s acceptability. Human-Computer Interaction, 37 (3), 263-280.

22 It may be of more than passing interest to know that the present work was ‘desk rejected’ by one of the leading journals in science as not of general interest to the readership.