SPSS macros for assessing the reliability and agreement of student evaluations of teaching

References

• Bartko , J.J. 1966 . The intraclass correlation coefficient as a measure of reliability . Psychological Reports , 19 : 3 – 11 .
   PubMed Web of Science ®Google Scholar
• Berk , R.A. 1979 . Generalizability of behavioral observations: A clarification of interobserver agreement and interobserver reliability . American Journal of Mental Deficiency , 83 : 460 – 72 .
   PubMedGoogle Scholar
• Brennan , R.L. 2001 . Generalizability theory , New York : Springer .
   Google Scholar
• Ebel , R.L. 1951 . Estimation of the reliability of ratings . Psychometrika , 16 : 407 – 24 .
   Google Scholar
• Feldman , K.A. 1998 . “ Reflections on the study of effective college teaching and student ratings: One continuing quest and two unresolved issues ” . In Higher education: Handbook of theory and research , Edited by: Smart , J.C. Vol. 13 , 35 – 74 . New York : Agathon Press .
   Google Scholar
• Hayes , A.F. and Krippendorff , K. 2007 . Answering the call for a standard reliability measure for coding data . Communication Methods and Measures , 1 : 77 – 89 .
   Google Scholar
• Jackson , S.E. and Brashers , D.E. 1994 . Random factors in ANOVA , Thousand Oaks, CA : Sage .
   Google Scholar
• Krippendorff , K. 1970 . Estimating the reliability, systematic error and random error of interval data . Educational and Psychological Measurement , 30 : 61 – 70 .
   Web of Science ®Google Scholar
• Krippendorff , K. 1980 . Content analysis: An introduction to its methodology , Beverly Hills, CA : Sage .
   Google Scholar
• Krippendorff , K. 2004a . Reliability in content analysis: Some common misconceptions and recommendations . Human Communication Research , 30 : 411 – 33 .
   Web of Science ®Google Scholar
• Krippendorff , K. 2004b . Content analysis: An introduction to its methodology. , 2nd ed. , Thousand Oaks, CA : Sage .
   Google Scholar
• Lombard , M. , Snyder‐Duch , J. and Bracken , C.C. 2002 . Content analysis in mass communication research: An assessment and reporting of intercoder reliability . Human Communication Research , 28 : 587 – 604 .
   Web of Science ®Google Scholar
• Marsh , H.W. and Overall , J.U. 1979 . Long‐term stability of students’ evaluations: A note on Feldman’s ‘consistency and variability among college students in rating their teachers and courses’ . Research in Higher Education , 10 : 139 – 47 .
   Google Scholar
• Marsh , H.W. and Roche , L.A. 1997 . Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility . American Psychologist , 52 : 1187 – 96 .
   Web of Science ®Google Scholar
• McGraw , K.O. and Wong , S.P. 1996 . Forming inferences about some intraclass correlation coefficients . Psychological Methods , 1 : 30 – 46 .
   Web of Science ®Google Scholar
• Nunnally , J.C. and Bernstein , I.H. 1994 . Psychometric theory. , 3rd ed. , New York : McGraw‐Hill .
   Google Scholar
• Rajaratnam , N. 1960 . Reliability formulas for independent decision data when reliability data are matched . Psychometrika , 25 : 261 – 71 .
   Web of Science ®Google Scholar
• Shockley‐Zalabak , P. 2006 . Reappointment, promotion and tenure (Policy No. 200‐001). http://web.uccs.edu/vcaf/uccspol.html (accessed April 19, 2006)
   Google Scholar
• Shrout , P.E. and Fleiss , J.L. 1979 . Intraclass correlations: Uses in assessing rater reliability . Psychological Bulletin , 86 : 420 – 8 .
   PubMed Web of Science ®Google Scholar
• Snedecor , G.W. 1946 . Statistical methods. , 4th ed. , Ames, IA : Iowa State College Press .
   Google Scholar
• SPSS . 1990 . SPSS reference guide , Chicago, IL : SPSS Inc. .
   Google Scholar
• Tinsley , H.E. and Brown , S.D. 2000 . “ Interrater reliability and agreement ” . In Handbook of applied multivariate statistics and mathematical modeling , Edited by: Tinsley , E.A. and Brown , S.D. 95 – 124 . San Diego, CA : Academic Press .
   Google Scholar
• Wachtel , H.K. 1998 . Student evaluation of college teaching effectiveness: A brief review . Assessment & Evaluation in Higher Education , 23 : 191 – 211 .
   Google Scholar
• Wilkinson , L. and APA Task Force on Statistical Inference . 1999 . Statistical methods in psychology journals: Guidelines and explanations . American Psychologist , 54 : 504 – 604 .
   Google Scholar