The Future of Peer Review

Since the 1960s, and maybe before, the acronym STS was used to identify a curriculum or teaching approach known as Science–Technology–Society (National Science Teachers Association, 1982). The original acronym of STS has evolved over the years into Science Technology Society and Environment (STSE). Most recently the STS and STSE movements have given way to the use of socioscientific issues as a platform for teaching science subject matter, science practices, argumentation, and the nature of science. The purpose of this editorial is not to discuss teaching approaches or curriculum but rather to discuss something near and dear to beginning and experienced faculty members (i.e., scholarship, teaching, and service). These are the categories of accomplishments that are used to make decisions about tenure and promotion. Often teaching is the first category listed in tenure and promotion guidelines, but we have all learned that scholarship is the most important and service the least.

Service typically refers to committee work on one’s university campus and committee membership and leadership positions in professional organizations. Most universities require faculty members to serve on university committees. In terms of becoming active in professional organizations, those who are around and have networked with other faculty in the professional community for 5 years or more become involved in various professional organizations without much effort. However, beginning faculty members must make a more concerted effort to become involved and are usually mentored during their PhD programs to sign up for various committees, review conference proposals, and maybe review manuscripts for journals. The latter is not always possible because some journals do not permit graduate students to be part of the editorial review board (ERB). We are glad to say that the Association for Science Teacher Education (ASTE) has just approved advanced PhD students to become members of its ERB for the Journal of Science Teacher Education (JSTE). Many of us involve our students in reviewing manuscripts on an informal basis as part of our PhD programs, as we believe that reviewing manuscripts assists students in writing and submitting their own research articles. As a consequence of this type of informal mentoring (which in some PhD programs is part of a formal course) we enthusiastically agree with the ASTE Board of Directors decision to allow PhD students to serve as members of the ERB of JSTE. The story is different for experienced faculty members, as they should already have a record of publications in peer-reviewed journals to support their applications to serve as reviewers or editorial board members.

With this background, let us move on to the primary focus of this editorial. The peer-review process requires that each submitted manuscript be reviewed by at least two reviewers, one associate editor, and the editor(s)-in-chief. Reviewers and associate editors perform their tasks voluntarily. Editor(s)-chief may not receive financial compensation (depending on the journal), but these individuals do get compensation in the form of a graduate assistantship, managing editor, and so on. As editors we rely on the professional integrity of our reviewers. Being a reviewer is a thankless job, with the only tangible reward being that you can list your editing experience as an item on your vita. From our experience as editors-in-chief of JSTE, our membership on other editorial boards, and our network of colleagues in similar situations, we can confidently say that reviewers are usually responsible for six or more manuscripts per year. The science education research community is relatively small compared to other professional communities. Nevertheless, given the pressure to publish in peer-reviewed journals, editors can never have enough reviewers. Editors face many problems with respect to the peer-review process, which include severely delayed reviewers, reviews of poor quality, and reviewers declining the invitation to review. Often a poor review provides the editor with little more help than a reviewer who declines the invitation to review. For example, Norman received the following review of a manuscript submitted to an unnamed journal: “This manuscript reminds me of a cheeseburger, except the cheese is missing and the meat has not been cooked yet. But, the buns look pretty good.”

What can an editor do with a review like this? It may be humorous to some, but it is really no different from the more common review of “Accept as is.” From our vantage point there is an increasing frequency of reviewers who simply decline any requests to review manuscripts. This is not just a JSTE problem; it is a growing problem for each of the journals for which serve as associate editors, editorial board members, or simply reviewers. All journals keep records of how often reviewers accept invitations to review, how often reviewers decline invitations, and how many reviews have been completed during the year. In some cases, reviewers are removed from the list of reviewers, but generally reviewers complete their terms as reviewers regardless of their track record. Again, remember that journals can never have enough reviewers and that the work reviewers do is voluntary. So editors-in-chief are hesitant to nudge their poor reviewers too much. There was a time when reviewers wore their membership on a journal’s review board as a badge of honor, not just as a line entry on their vita. Providing feedback on a submitted article and potentially influencing what research finds its way into the published literature in your professional community is a heavy responsibility and a reward in and of itself (Hamilton, 1991). Years ago, when Norman first served on the editorial board of the Journal of Research in Science Teaching, the editor kept track of how long it took each editorial board member to complete his or her review. These data were shared at the annual editorial board meeting, and people became very competitive and wanted to decrease their review times. Some international members even complained that it took longer for their reviews to be received by the journal editor because of the time it took to send international reviews through the postal service. Yes, Norman is old enough to remember the days when manuscripts were received though regular mail and reviews were submitted through regular mail as well. In any case, it is doubtful that this behavioristic tactic would be effective today.

Delayed reviews and/or reviewers who decline the invitation to review delay the editorial process, and it potentially affects the tenure and promotion process. We are sure that reviewers who decline the invitation to review or who submit severely delayed reviews would not be receptive if their own manuscript were handled in this manner. We are not sure whether it was a serious suggestion, but a reviewer for a journal other than JSTE mused about handling manuscripts from reviewers who were not fulfilling their responsibilities differently in some nondisclosed manner. We are not sure what has created the problem, but there was a time when reviewers took their task as reviewers as a critically important professional responsibility. Please do not misconstrue what we are saying. There are certainly many excellent reviewers who diligently complete their reviews in a timely manner. These same individuals rarely decline an invitation, unless there are extenuating circumstances, such as illness, a temporarily heavier workload than usual, or hard deadlines associated with the writing of a grant. All editors-in-chief know who these obsessive-compulsive people are, and the reward is usually getting more manuscripts to review. We do not intend to exploit our hardworking reviewers, but it happens much too often. If we extrapolate the problem, we worry that the peer-review process is relying on a continually shrinking pool of reviewers (Siegelman, 1991). The situation runs contrary to the overarching goal of the peer-review process. We want a diversity of perspectives, specialized knowledge, and a plurality of viewpoints having input into the process as opposed to a small group of like-minded reviewers. The National Science Foundation selects panel members for proposal review cognizant of this issue. It intentionally selects panel members so that a plurality of viewpoints is represented. In short, all authors want and deserve thorough and fair reviews of their work. Just to be clear, the issue discussed is not isolated to JSTE but is an increasing problem with all peer-reviewed journals.

We applaud the conscientious reviewers for their dedication to the peer-review process. For those reviewers who are not stepping up to the plate, we desperately need your expertise and help.