Exploring the Relationship Between Interrater Correlations and Validity of Peer Ratings

REFERENCES

• Borman , W. C. 1975 . Effects of instructions to avoid halo error on reliability and validity of performance evaluation ratings . Journal of Applied Psychology , 60 : 556 – 560 .
   Web of Science ®Google Scholar
• Buckner , D. N. 1959 . The predictability of ratings as a function of interrater agreement . Journal of Applied Psychology , 43 : 60 – 64 .
   Web of Science ®Google Scholar
• Cronbach , L. J. 1970 . Essentials of psychological testing , New York : Harper & Row .
   Google Scholar
• Freeberg , N. E. 1969 . Relevance of rater-ratee acquaintance in the validity and reliability of ratings . Journal of Applied Psychology , 53 : 518 – 524 .
   PubMed Web of Science ®Google Scholar
• Ghiselli , E. E. , Campbell , J. P. and Zedeck , S. 1981 . Measurement theory for the behavioral sciences , San Francisco : W.H. Freeman. .
   Google Scholar
• Hanson , R. K. , Hunsley , J. and Parker , K. C. H. 1988 . The relationship between WAIS subset reliability, g Loadings, and meta-analytically derived validity estimates . Journal of Clinical Psychology , 44 : 557 – 562 .
   Web of Science ®Google Scholar
• Heneman , R. L. 1986 . The relationship between supervisory rating and result-oriented measures of performance: A meta-analysis . Personnel Psychology , 39 : 811 – 826 .
   Web of Science ®Google Scholar
• Horowitz , L. M. , Inouye , D. and Siegelman , E. Y. 1979 . On averaging judges' ratings increase their correlation with an external criterion . Journal of Consulting and Clinical Psychology , 47 : 453 – 458 .
   PubMed Web of Science ®Google Scholar
• Kane , J. S. and Lawler , E. E. III . 1978 . Methods of peer assessments . Psychological Bulletin , 85 : 555 – 586 .
   Web of Science ®Google Scholar
• McGraw , K. O. and Wong , S. P. 1996 . Forming inferences about intraclass correlation coefficients . Psychological Methods , 1 : 30 – 46 .
   Web of Science ®Google Scholar
• McNemar , Q. 1962 . Psychological statistics , New York : Wiley .
   Google Scholar
• Murphy , K. R. and De Shon , R. 2000 . Interrater correlations do not estimate the reliability of job performance ratings . Personnel Psychology , 53 : 873 – 900 .
   Web of Science ®Google Scholar
• Peter , P. J. and Churchill , G. A. 1986 . Relationships among research design choices and psychometric properties of rating scales: A meta-analysis . Journal of Marketing Research , XXIII : 1 – 10 .
   Google Scholar
• Saal , F. E. , Downey , R. G. and Lahey , M. A. 1980 . Rating the ratings: Assessing the psychometric quality of rating data . Psychological Bulletin , 88 : 413 – 428 .
   Web of Science ®Google Scholar
• Schmidt , F. L. and Kaplan , L. B. 1971 . Composite vs. multiple criteria: A review and resolution of the controversy . Personnel Psychology , 24 : 419 – 434 .
   Web of Science ®Google Scholar
• Schmidt , F. L. , Viswesvaran , C. and Ones , D. S. 2000 . Reliability is not validity and validity is not reliability . Personnel Psychology , 53 : 901 – 912 .
   Web of Science ®Google Scholar
• Shrout , P. E. and Fleiss , J. L. 1979 . Intraclass correlations: Uses in assessing rater reliability . Psychological Bulletin , 86 : 420 – 428 .
   PubMed Web of Science ®Google Scholar
• Suen , H. , Williams , B. L. and Naidoff , L. 1995 . Does raters' homogeneity enhance validity? . Perceptual and Motor Skills , 80 : 254
   Web of Science ®Google Scholar
• Thorndike , R. L. 1982 . Applied psychometrics , Boston : Houghton Mifflin .
   Google Scholar
• Tinsley , E. A. and Weiss , D. J. 1975 . Interrater reliability and agreement of subjective judgments . Journal of Counseling Psychology , 22 : 358 – 376 .
   Web of Science ®Google Scholar
• Viswesvaran , C. , Ones , D. S. and Schmidt , F. L. 1996 . Comparative analysis of the reliability of job performance ratings . Journal of Applied Psychology , 81 : 557 – 574 .
   Web of Science ®Google Scholar
• Weekley , J. A. and Gier , J. A. 1989 . Ceilings in the reliability and validity of performance ratings: The case of expert raters . Academy of Management Review , 32 : 213 – 222 .
   Google Scholar