From the Editor—Writing a Good Peer Review to Improve Scholarship: What Editors Value and Authors Find Helpful

The following quotes were recently published in Buzzfeed (Oakes, 2014), an online media and technology company, and ostensibly were culled from actual peer reviews published on another website:

I am afraid this manuscript may contribute not so much toward the field’s advancement as much as toward its eventual demise.

I am generally very happy to provide extensive suggestions and comments on manuscripts, but this submission was an absolute waste of my time.

It is early in the year, but difficult to imagine any paper overtaking this one for lack of imagination, logic, or data it is beyond redemption.

As is the case with much content on the Internet, it is hard to determine whether these are fabricated or real. Nonetheless, despite the intended humor for the general public, they clearly demonstrate the type of mean-spirited review comments that are not helpful to authors or editors.

With increasing demands for faculty to publish in top-ranked journals, editors continually are faced with the challenge of selecting high-quality manuscripts that not only meet the specific criteria for their journals but also retain or raise their journal’s prestige. The peer-review process plays an indispensable role in these decisions, and editors base their final decisions on their reviewers’ comments combined with their own expertise, assessment, and judgment.

A 2009 large-scale international peer-review survey of more than 4,000 authors and reviewers found that most respondents valued the peer-review process and deemed it to be essential, and almost all researchers believed that the peer-review process improved the quality of their papers. The vast majority of reviewers not only enjoyed reading other scholars’ works but also appreciated the ability to help authors improve their manuscripts. Notably, they also saw this as an important role as a member of the academic community and were committed to conducting reviews in the future. However, they also noted that to improve the peer review process, the training of new reviewers was needed (Mulligan, Hall, & Raphael, 2013).

Although most journals provide specific guidelines for reviewers, in general there is little or no mentorship when faculty are selected to conduct peer reviews. Although some may be fortunate to learn from colleagues or mentors, it is not uncommon for new reviewers to simply learn on their own by conducting reviews. By and large, there are some fairly standard criteria that most editors look for in a review, and you will see some of these common themes reflected here. However, because there is variation across journals in focus and scope, I have invited the editors (or immediate past editors) of Social Work Education: The International Journal, Social Work, Families in Society: The Journal of Contemporary Social Services, Affilia: The Journal of Women and Social Work, and Social Work: The Journal of the National Association of Social Workers to join me in this collaborative editorial.

*****

Sondra J. Fogel, PhD, LCSW

Editor-in-Chief, Families in Society: The Journal of Contemporary Social Services

Editors of academic publications that use a peer-review process depend on their reviewers to provide quality assessments of submitted manuscripts so that they can provide final dispositions that are as informed, fair, and justifiable as possible. Indeed, the key players in peer-reviewed journals—authors, reviewers, and editors—agree that participatory scholarship is key to co-creating a body of literature that positively affects the development of social work and allied disciplines. This process expects reviewers (who, most often, volunteer their time) to critique the submitted work against established research standards and on criteria determined important to the journal. Reviewers are also typically offered an opportunity to expand on their analysis with detailed comments that provide guidance for improving the submission.

Although this seems like a very straightforward process, not all reviews that are provided to the editor are as helpful to the disposition process as the reviewer would like to believe. And, as many reviewers know because they also submit material for publication consideration, not all reviews provide sufficient guidance or justification for revisions requested or the comments received. This, too, can make the decision process for the editor excruciating. Therefore, to help reviewers provide their best work for the editor and manuscript authors, the following key points will be highlighted for potential new reviewers (and existing reviewers) of Families in Society.

Of course, it is important to know who the audience is for the journal. The readership of Families in Society includes academic scholars and researchers in a variety of fields, administrators, practitioners, and students in social work and human services. Then, it is critically important for reviewers to know about the journal and the organization that sponsors the journal. Founded as The Family in 1920 by social casework pioneer Mary Richmond, Families in Society is the most enduring scholarly social work journal. The mission of the journal is to advance translational research and critical analysis on a broad array of issues related to all areas of practice, including consideration of the various bio-psycho-social, economic, and cultural factors that affect functioning and well-being. That said, this journal accepts submissions on a broad range of interdisciplinary topics related to research, policy, theory, direct-practice issues, and relevant international works.

Families in Society does this because it is sponsored by the Alliance for Strong Families and Communities. Established in 1911, the Alliance is one of America’s earliest national charity organizations. Currently, there are nearly 500 association members, a collection of private, nonprofit social service agencies and community centers in the United States and Canada. Because of this history and affiliation, the reviewer must be able to judge whether the manuscript makes a significant contribution to the implications for practice.

This implies, therefore, that the reviewer must be knowledgeable about the content area, including recent developments; research methods and standards; innovations in treatment methods or interventions if available; and the significant literature that supports the work. Helpful reviews come from those who are comfortable with their expertise in the manuscript area. Specificity of guidance, collegial tone of response, and applicability to the criteria rating form are also very important. Although sometimes reviewers are asked to provide commentary on manuscripts that stretch their scholarly knowledge (for example—on advanced statistical procedures), alerting the editor that a specific section or area of the manuscript is outside one’s expertise is important. This can be done in a confidential way and is always appreciated.

Providing comprehensive comments for the editor, which are shared with the authors, is very useful. In fact, there are several other tangible steps reviewers can take to ensure that they are providing actionable feedback: (a) organize the review by the manuscript’s section headings and specify where suggested changes are needed; (b) give specific examples of the strengths as well as the weaknesses of the manuscript when providing constructive criticism; (c) suggest additional literature sources (with full bibliographic details) to the authors that might inform or enhance their manuscripts; (d) focus on whether the manuscript can contribute to improving and advancing practice, policy, or theory; and (e) value everyone’s time and decline to review if other pressing commitments could interfere with quality and timeliness, or request an alternate deadline from the journal team.

Good reviewers, like authors of good manuscripts, dedicate time and effort to ensure quality work. This dedication and a commitment to excellence likewise reflect some of the central tenets of our profession—respect, empowerment, and collaboration. We hope this challenge appeals to you and that these values mirror your own.

*****

Noël Busch-Armendariz, PhD, LMSW, MPA

Editor-in-Chief, Affilia: The Journal of Women and Social Work

and

Karin Wachter, MEd

Editorial Assistant for Publications, Affilia: The Journal of Women and Social Work

In 1986 the founding editors created Affilia: Journal of Women and Social Work as a dedicated space for feminist women’s voices, underrepresented in mainstream social work journals, and the topical and methodological challenges that these marginalized voices raised for the field and the discipline. Today, Affilia is a living record of feminist social work. We seek exceptional scholarship—groundbreaking, thought-provoking works that challenge taken-for-granted knowledge; raise new questions; generate innovative theories and methodologies; reflect feminist social work’s global diversity; and illuminate alternative pathways for social work theory, research, practice, and teaching.¹

We accept a broad range of feminist social work scholarship through a variety of media in four issues per year. Our articles include empirical qualitative, quantitative, and mixed-methods research and substantive conceptual pieces. Affilia also publishes four peer-reviewed columns (On Practice, Women Creating Change, Past and Present, and On the Bias), book reviews, and poetry. These works are managed by editors or co-editors and maintain our feminist values.

Our commitment as a feminist journal is to both process and outcome; we aim for high-quality reviews through a helpful learning process and approach the peer-review process based on feminist principles, recognizing that the relationship between authors and reviewers, although anonymous, can be quite intimate. The review process involves multiple perspectives and feedback from editorial or consulting board members, guest reviewers, and the editorial leadership team.

A good review is systematic. To help in this process, we spent a considerable amount of time developing a guideline for our reviewers to ensure consistency and thoroughness and communicate our expectations. In turn, we have also developed a process for authors to respond to reviewers’ comments to facilitate the anonymous communication between author and reviewer while acknowledging the intimacy of that discussion.

Year after year we see that the articles that are theoretically strong receive the most citations; therefore, a good review encourages authors to develop their theoretical orientation.

A good review is strong in its methodological critique. Methodology and methods are often where beginning authors, in particular, need the most guidance. It follows, then, that a good review also comments on the weightiness of the research effort. A good review also insists on intersectional and multicultural perspectives and analyses. This involves encouraging authors to be reflexive and to inform their readers of their positionality. Social work’s legacy is liberatory, and a good review encourages the authors to speak with authority, take a strong position, use her or his voice, and expand her or his vision.

Finally, we recognize that a good review also builds the reputation of the journal. Although reviewers are anonymous, the journals we review for are not. Thus, it is important that reviewers are thoughtful, collegial, and respectful, because reviews will do much to enhance authors’ assessment of the journal and increase the likelihood of future contributions. Often we fail to communicate our positive reactions and consider that a good review praises as well as critiques.

Finally, a good review is returned on time. In good social work practice, check it off your To Do List! Reviewers owe it to the authors, the editor, the journal, and themselves.

*****

Hugh McLaughlin, PhD

Editor-in-Chief, Social Work Education: The International Journal

Peer reviewers are critical to the success of any academic peer-reviewed journal. Peer reviewers are responsible for holding the line about what is acceptable for publication in the journal and for supporting authors to improve their papers to the journal’s accepted standard. In identifying reviewers, the editors in Social Work Education: The International Journal look for a board member and another reviewer whose area of expertise is aligned with the article under consideration. If one of the reviewers is new to the process, a third reviewer will be asked to assess the manuscript. It is also important, if asked to review, that you consider whether you have the time, or subject knowledge, to undertake the review and, if the answer is no, then to refuse the request to review. Editors and authors prefer timely responses to requests to review even if the assessment is negative.

If the two reviewers arrive at the same assessment, this is helpful for editors, but it is also not unusual to have two very different evaluations—for example, one assessor considers the article requires only minor revisions, whereas the other assessor considers that it should be rejected. In these cases the editor, or a third reviewer, will assess the manuscript. Social Work Education: The International Journal asks reviewers to judge articles as publishable, publishable with minor or major revisions, a rejection and resubmit, or reject.

To do this, they are asked to consider the manuscript in line with the reviewer’s questions on ScholarOne covering overview, methodology, and structure and content and a commentary on the article that will go to the authors and the editor. The overview consists of seven questions covering issues such as does the “subject matter fall within the scope of the journal?” Does it “make a significant and original contribution to knowledge?” The methodology section for empirical articles contains six questions, including asking reviewers to assess whether the “methodology is appropriate to address the research question and the aims and objectives of the research?” “Have ethical considerations been clearly addressed?” and, importantly, “Are the interpretations and conclusions sound and justified?” The third section focuses on structure and presentation and includes questions about whether the manuscript is well-organized with a clear introduction and conclusions: “Is the paper free of discriminatory language or assumptions?,” “Is any article content relating to a particular country appropriately contextualized for an international audience?,” and is the article within the journal’s 6,000-word limit? These questions are used to help direct reviewers to important issues before they complete their review of the articles.

In completing their review commentary, reviewers are asked to be mindful of the dictum, “Do unto others as you would wish them to do unto you.” This is to ask reviewers to consider how others have structured their feedback to help them improve their manuscripts. A helpful review is one that begins with an overview of the article identifying its strengths, structure, contribution to new knowledge, and suitability of topic for the journal. This is often best compiled at the end of the review when the rest of the review has been completed. The reviewer should then identify constructive criticisms section by section or page by page. It is helpful to be as specific as possible both in terms of the comments concerning the content and identifying the lines where possible (e.g., P3 Lines 35–40). Very general comments are often unhelpful to authors, making it difficult for them to be addressed. Reviewers may suggest alternative literature for consideration that may enhance the manuscript and should also be aware of whether the journal has recently published articles on the same theme, which should be included, if the author has not already done so. A common problem for Social Work Education: The International Journal is the international dimension, and reviewers need to consider whether the authors have provided sufficient detail for readers to understand and benefit from the article. The reviewers should also consider issues of grammar and spelling throughout the article, again by being very specific where these occur. This can either be completed as a separate section at the end of their review or as part of the ongoing commentary. The individual reviewer’s comments are shared blind, between the reviewers, and many reviewers have commented how helpful this has been in helping them become more effective and how it has contributed to improving their own manuscripts.

The journal allows reviewers to make confidential comments to the editors. Although these comments can be helpful where reviewers are uncertain over particular issues or assessments, they are not a substitute for providing honest critical feedback to the author. Reviewers are finally asked, where appropriate, to indicate whether they would be willing to assess a resubmission.

Many reviewers are also authors, and most authors have also been reviewers. Reviewing is a skill that benefits both the author and the reviewer and is critical for the reputation of the journal.

*****

Elizabeth C. Pomeroy, PhD, LCSW

Former Editor-in-Chief, Social Work: The Journal of the National Association of Social Workers

As past editor-in-chief of Social Work, I have read a plethora of peer reviews for scholarly manuscripts that were submitted for publication. The peer-review process is an invaluable component of evaluating a manuscript’s merit for adding new knowledge to the field. The goal of the peer-review process is to provide authors with an objective critical analysis of their writing and work so that the best manuscripts are disseminated to the professional world of scholars and practitioners. Due to the large volume of manuscript submissions, the acceptance rate for Social Work is fairly low, which makes the work of peer reviewers an extremely important part of the process.

Peer reviewers must meet certain criteria to be considered for the position. They must have a record of scholarly publication and be National Association of Social Workers members. They must also be able to review manuscripts in a timely manner and show sound, independent judgment in critiquing others’ work. Peer reviewers are matched by area of expertise to manuscripts under review. For example, a peer reviewer who is an expert in substance use would be asked to review a manuscript pertaining to this specialized field of social work. In sum, peer reviewers are the bedrock of success for any scholarly journal. Without good reviewers, the journal would be unable to meet its goals of contributing to the knowledge base of social work.

A good peer review provides the author with a clear and comprehensive analysis of the manuscript. It specifically addresses both the strengths and the limitations of the content and the writing. Vague or sweeping generalizations such as “the study had some interesting findings but not enough to warrant publication” with no further specifics are not helpful to the author or the editor. Peer reviews that address each section of the manuscript and that suggest specific modifications to the writing as well as more substantive improvements to the content can provide clear direction for enhancements to the manuscript. Often, excellent studies or conceptual manuscripts can be greatly improved when the author is able to specifically address some of the concerns raised by the peer reviewers. In addition, good peer reviewers may ultimately recommend rejecting the manuscript for publication but have provided the author with enough information to know whether the manuscript is salvageable or not.

Some of the best peer reviews contain references to articles that may be useful to the author during the revision process. The additional information can add substance to a study that may have conceptual limitations or that is lacking in implications for social work. Peer reviewers who are aware of the existing gaps in the literature are able to assist authors by pointing out the limitations of the existing research and how their manuscripts may be a necessary addition to the knowledge base of a particular field.

Finally, a good review is timely. Peer reviewers are volunteers from the professional community, and it takes time to carefully analyze a manuscript and write a review. Good peer reviewers get manuscripts read and critiqued within a month of receiving them. It is unfair to authors, who may be under pressure to publish or perish, for the review process to take an unusually long period of time. On the other hand, the production of a journal is a large undertaking, with many busy professionals involved; therefore, it is incumbent on the individual reviewer, rather than journal staff, to get a manuscript reviewed in a timely manner. Both authors and good reviewers are respectful of deadlines and understand the importance of giving and receiving feedback in a courteous, thoughtful manner.

In conclusion, good peer reviews can be extremely beneficial to the overall quality of a manuscript. Authors should carefully analyze the recommendations of the peer reviewers. Not all recommendations need to be accepted by the author; however, a clear explanation for rejecting the feedback should accompany any revision that is submitted for examination. By doing so, the author will prevent any misunderstandings between the reviewers and manuscript writers. As an author, reviewer, and editor, I have the utmost respect for the peer-review process and view it as a very positive system for introducing current knowledge to the field.

*****

As noted above, there are a number of overlapping themes that are central to any good review. These include, but are not limited to, a clear understanding of the focus of the journal; timely submission of the review; well-balanced and comprehensive feedback that is presented in a respectful and civil manner; constructive criticism that gives clear guidance on how to improve the manuscript; originality and contribution to the existing literature; suggestions for additional literature that the author(s) may have missed; and an in-depth review of the adequacy of the methodology and conceptual/theoretical framework used, as relevant. It is important to keep in mind that reviews have essentially two distinct functions: (1) to help the editor make a decision about the manuscript and (2) to provide sufficient feedback that will help the author improve it.

I would add to this several other considerations that are somewhat specific to the Journal of Social Work Education (JSWE). Reviewers must be members of the Council on Social Work Education and have, at a minimum, a solid beginning record of peer-reviewed publications to review manuscripts for the journal. We publish several categories of manuscripts, including full-length, general papers that can be empirical, conceptual, theory based, or systematic reviews, as well as shorter Teaching Notes, Field Notes, and Research Notes. It is important for reviewers to understand the requirements for each of these categories and review the paper strictly in accordance with the guidelines for each. A good review is developmental in nature, and it is therefore essential that you provide very concrete suggestions for needed revisions. That said, it is equally important for the reviewer to not interject her or his positive or negative biases (either explicit or implicit) into the review. Reviews should also be clear about the major issues that must be addressed, in contrast to suggestions that are helpful but not essential.

In my experience, reviewers use a variety of styles in their reviews. Some provide a summary of the article prior to their review comments. Others use the review form that we provide, and yet others simply delve straight into their review. Any of these styles is acceptable as long as the review comments are thorough, thoughtful, and address both the strengths and necessary areas for improvement. Some reviewers give very extensive feedback on typos, grammatical errors, and reference errors, whereas others simply note that there are a number of errors and suggest that the authors carefully address this in their revisions. I imagine that authors appreciate extensive feedback of this sort, but, as an editor, I am most concerned with an assessment of the overall quality of the article and, most importantly, whether or not it makes a significant contribution to our knowledge base. That said, a poorly written paper is one that is not likely to get a positive review.

As noted above, most journals, including JSWE, allow for confidential comments to the editor in which reviewers can give an honest opinion about the merits of the paper. It is important to include not only all substantive comments in the review itself, but confidential comments also should be congruous with the review. Finally, it is important to not place the recommendation in the body of the review. It is not uncommon for reviewers to disagree about the merits of a paper, and the final decision made by the editor may not concur with a specific reviewer’s recommendations. Typically, editors strip recommendations from the review prior to sending it to the author, and this simply creates more work for the editor. Finally, it is extremely uncommon for most manuscripts to be accepted without any required changes. In the rare instance that a manuscript is of such high quality that all reviewers recommend acceptance, it is equally important to provide substantive feedback to the authors that address the strengths of the manuscript. One- and two-sentence reviews are neither helpful nor valued by the editor or author.

If you are interested in becoming a reviewer for any of these journals, please contact the journal directly through their respective websites.

Susan P. Robbins, PhD, LCSW

University of Houston

Editor-in-Chief

Notes

This editorial is being published simultaneously in the Journal of Social Work Education and Social Work Education: The International Journal.

¹ In 2014 About AFFILIA was written by a small group of current editorial board members and was ratified by the board to replace our mission statement. It currently appears on our website and is used in promotional materials.