Reliability of Reviewer Ratings in the Manuscript Peer Review Process: An Opportunity for Improvement

REFERENCES

• Abbott , A. 2011 . A how-to for peer review . Nature , 473 : 17
   PubMed Web of Science ®Google Scholar
• Armstrong , J. S. 1996 . We need to rethink the editorial role of peer reviewers . Chronicle of Higher Education , 43 : B3 – B4 .
   Google Scholar
• Blomme , E. A. 2011 . The ARRIVE guidelines: A resource for authors and reviewers to ensure that submissions to The Veterinary Journal meet minimal expectations of completeness, accuracy and transparency . The Veterinary Journal , 189 : 237 – 238 .
   PubMed Web of Science ®Google Scholar
• Callaham , M. L. 2002 . Research into peer review and scientific publication: Journals look in the mirror . Ann. Emerg. Med. , 40 : 313 – 316 .
   PubMed Web of Science ®Google Scholar
• Campion , M. A. 1993 . Article review checklist: A criterion checklist for reviewing research articles in applied psychology . Personnel Psychology , 46 : 705 – 718 .
   Web of Science ®Google Scholar
• Curzon , M. E. and Cleaton-Jones , P. E. 2011 . Reviewing scientific manuscripts . Eur. Arch. Paediatr. Dent. , 12 : 184 – 187 .
   PubMedGoogle Scholar
• Doi, S. A. (2011). Editors must have a mandate that excludes the tradition of confirmatory bias. October 18, 2011. http://www.bmj.com/rapid-response/2011/11/03/editors-must-have-mandate-excludes-tradition-confirmatory-bias (http://www.bmj.com/rapid-response/2011/11/03/editors-must-have-mandate-excludes-tradition-confirmatory-bias) (Accessed: 23 January 2013 ).
   Google Scholar
• Duchesne , S. and Jannin , P. 2008 . Proposing a manuscript peer-review checklist . Neuroimage , 39 : 1783 – 1787 .
   PubMed Web of Science ®Google Scholar
• Fiske , D. W. and Fogg , L. 1990 . But the reviewers are making different criticisms of my paper! . American Psychologist , 45 : 591 – 598 .
   Web of Science ®Google Scholar
• Heddle , N. M. and Ness , P. M. 2009 . Reviewing manuscripts: tips and responsibilities . Transfusion , 49 : 2265 – 2268 .
   PubMed Web of Science ®Google Scholar
• Henly , S. J. and Dougherty , M. C. 2009 . Quality of manuscript reviews in nursing research . Nurs. Outlook , 57 : 18 – 26 .
   PubMed Web of Science ®Google Scholar
• Howard , L. and Wilkinson , G. 1998 . Peer review and editorial decision-making . Br. J. Psychiatry , 173 : 110 – 113 . discussion 114–115
   PubMed Web of Science ®Google Scholar
• Ioannidis , J. P. , Tatsioni , A. and Karassa , F. B. 2010 . Who is afraid of reviewers’ comments? Or, why anything can be published and anything can be cited . Eur. J. Clin. Invest. , 40 : 285 – 287 .
   PubMed Web of Science ®Google Scholar
• Jackson , J. L. , Srinivasan , M. , Rea , J. , Fletcher , K. E. and Kravitz , R. L. 2011 . The validity of peer review in a general medicine journal . PLoS One , 6 : e22475
   PubMedGoogle Scholar
• Jefferson , T. , Rudin , M. , Brodney Folse , S. and Davidoff , F. 2007 . Editorial peer review for improving the quality of reports of biomedical studies . Cochrane Database Syst. Rev , 2 : MR000016
   PubMedGoogle Scholar
• Landis , J. R. and Koch , G. G. 1977 . The measurement of observer agreement for categorical data . Biometrics , 33 : 159 – 174 .
   PubMed Web of Science ®Google Scholar
• Loonen , M. P. , Hage , J. J. and Kon , M. 2005 . Who benefits from peer review? An analysis of the outcome of 100 requests for review by Plastic and Reconstructive Surgery . Plast. Reconstr. Surg. , 116 : 1461 – 1472 . discussion 1473–1475
   PubMed Web of Science ®Google Scholar
• Mahoney , M. 1977 . Publication prejudices: An experimental study of confirmatory bias in peer review system . Cognitive Therapy and Research , 1 : 161 – 175 .
   Google Scholar
• Marotti , M. 2011 . Primum non nocere: Authors or reviewers? Rev. Assoc. Med . Bras. , 57 ( 477 ) : 487
   Google Scholar
• Neufeld , J. 1970 . To amend refereeing . Physics Today , 23 : 9 – 10 .
   Web of Science ®Google Scholar
• Newton , D. P. 2010 . Quality and peer review of research: an adjudicating role for . Account. Res. , 17 : 130 – 145 .
   PubMed Web of Science ®Google Scholar
• Ploegh , H. 2011 . End the wasteful tyranny of reviewer experiments . Nature , 472 : 391
   PubMed Web of Science ®Google Scholar
• Prieto , L. , Lamarca , R. , Casado , A. and Alonso , J. 1997 . The evaluation of agreement on continuous variables by the intraclass correlation coefficient . J. Epidemiol. Community Health , 51 : 579 – 581 .
   PubMed Web of Science ®Google Scholar
• Ramulu , V. G. , Levine , R. B. , Hebert , R. S. and Wright , S. M. 2005 . Development of a case report review instrument . Int. J. Clin. Pract. , 59 : 457 – 461 .
   PubMed Web of Science ®Google Scholar
• Rothwell , P. M. and Martyn , C. N. 2000 . Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? . Brain , 123 : 1964 – 1969 .
   PubMed Web of Science ®Google Scholar
• Schroter , S. , Black , N. , Evans , S. , Carpenter , J. , Godlee , F. and Smith , R. 2004 . Effects of training on quality of peer review: Randomised controlled trial . BMJ , 328 : 673
   PubMed Web of Science ®Google Scholar
• Sosa , J. A. , Mehta , P. , Thomas , D. C. , Berland , G. , Gross , C. , McNamara , R. L. , Rosenthal , R. , Udelsman , R. , Bravata , D. M. and Roman , S. A. 2009 . Evaluating the surgery literature: can standardizing peer-review today predict manuscript impact tomorrow? . Ann. Surg. , 250 : 152 – 158 .
   PubMed Web of Science ®Google Scholar
• Starbuck , W. H. 2003 . Turning lemons into lemonade: Where is the value in peer reviews? . Journal of Management Inquiry , 12 : 344 – 351 .
   Web of Science ®Google Scholar
• Strayhorn , J. Jr , McDermott , J. F. Jr. and Tanguay , P. 1993 . An intervention to improve the reliability of manuscript reviews for the . Journal of the American Academy of Child and Adolescent Psychiatry. Am. J. Psychiatry , 150 : 947 – 952 .
   PubMed Web of Science ®Google Scholar