Determining Reliability of Clinical Assessment Scores in Real Time

REFERENCES

• Downing , S M . 2003 . Validity: On the meaningful interpretation of assessment data . Medical Education , 37 : 830 – 7 .
   PubMed Web of Science ®Google Scholar
• Cook , D A and Beckman , T J . 2006 . Current concepts in validity and reliability for psychometric instruments: Theory and application . American Journal of Medicine , 119 : 7 – 16 .
   PubMedGoogle Scholar
• Downing , S M and Haladyna , T M . 2004 . Validity threats: Overcoming interference with proposed interpretations of assessment data . Medical Education , 38 : 327 – 33 .
   PubMed Web of Science ®Google Scholar
• Beckman , T J , Cook , D A and Mandrekar , J N . 2006 . Factor instability of clinical teaching assessment scores among general internists and cardiologists . Medical Education , 40 : 1209 – 16 .
   PubMed Web of Science ®Google Scholar
• Messick , S . 1993 . “ Validity ” . In Educational measurement, , 3rd ed. , Edited by: Linn , R L . 13 – 103 . Phoenix, AZ : Oryx Press .
   Google Scholar
• American Education Research Association, American Psychological Association, National Council on Measurement in Education . 1999 . Standards for educational and psychological testing , Washington, DC : American Education Research Association .
   Google Scholar
• Smith , G T . 2005 . On construct validity: Issues of method and measurement . Psychological Assessment , 17 : 396 – 408 .
   PubMed Web of Science ®Google Scholar
• Clark , L A and Watson , D . 1995 . Constructing validity: Basic issues in objective scale development . Psychological Assessment , 7 : 309 – 19 .
   Web of Science ®Google Scholar
• Haynes , S N , Richard , D C and Kubany , E S . 1995 . Content validity in psychological assessment: A functional approach to concepts and methods . Psychological Assessment , 7 : 238 – 47 .
   Web of Science ®Google Scholar
• Beckman , T J , Ghosh , A K , Cook , D A , Erwin , P J and Mandrekar , J N . 2004 . How reliable are assessments of clinical teaching? A review of the published instruments . Journal of General Internal Medicine , 19 : 971 – 7 .
   PubMed Web of Science ®Google Scholar
• Beckman , T J , Cook , D A and Mandrekar , J N . 2005 . What is the validity evidence for assessments of clinical teaching? . Journal of General Internal Medicine , 20 : 1159 – 64 .
   PubMed Web of Science ®Google Scholar
• Beckman , T J , Lee , M C and Rohren , C H . 2003 . Evaluating an instrument for the peer review of inpatient teaching . Medical Teacher , 25 : 131 – 5 .
   PubMed Web of Science ®Google Scholar
• Beckman , T J and Mandrekar , J N . 2005 . The interpersonal, cognitive and efficiency domains of clinical teaching: Construct validity of a multidimensional scale . Medical Education , 39 : 1221 – 9 .
   PubMed Web of Science ®Google Scholar
• Beckman , T J , Lee , M C and Mandrekar , J N . 2004 . A comparison of clinical teaching evaluations by resident and peer physicians . Medical Teacher , 26 : 321 – 5 .
   PubMed Web of Science ®Google Scholar
• Beckman , T J . 2004 . Lessons learned from a peer review of bedside teaching . Academic Medicine , 79 : 343 – 6 .
   PubMed Web of Science ®Google Scholar
• Cohen , J . 1960 . A coefficient of agreement for nominal scales . Educational and Psychological Measurement , 20 : 37 – 46 .
   Web of Science ®Google Scholar
• Cicchetti , D V and Allison , T . 1971 . A new procedure for assessing reliability of scoring EEG sleep recordings . American Journal of EEG Technology , 11 : 101 – 9 .
   Google Scholar
• Fleiss , J L , Leven , B and Paik , M C . 2003 . Statistical methods for rates and proportions, , 3rd ed. , Hoboken, NJ : Wiley .
   Google Scholar
• Fleiss , J L and Cohen , J . 1973 . The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability . Educational and Psychological Measurement , 33 : 613 – 9 .
   Web of Science ®Google Scholar
• Fleiss , J L , Cohen , J and Everitt , B S . 1969 . Large sample standard errors of kappa and weighted kappa . Psychological Bulletin , 72 : 323 – 7 .
   Web of Science ®Google Scholar
• Landis , J R and Koch , G G . 1977 . The measure of observer agreement for categorical data . Biometrics , 33 : 159 – 74 .
   PubMed Web of Science ®Google Scholar
• Cronbach , L J . 1951 . Coefficient alpha and the internal structure of tests . Psychometrika , 16 : 297 – 333 .
   Web of Science ®Google Scholar
• Nunnally , J C and Berstein , I H . 1994 . Psychometric theory, , 3rd ed. , New York : McGraw-Hill .
   Google Scholar
• Kundel , H L and Polansky , M . 2003 . Measurement of observer agreement . Radiology , 228 : 303 – 8 .
   PubMed Web of Science ®Google Scholar
• DeVellis , R F . 1991 . Scale development: Theory and applications , London : Sage .
   Google Scholar
• Gorsuch , R L . 1983 . Factor analysis, , 2nd ed. , Hillsdale, NJ : Erlbaum .
   Google Scholar
• Messick , S . 1995 . Validity of psychological assessment. Validation of inferences from persons' responses and performances as scientific inquiry into score meaning . American Psychologist , 50 : 741 – 9 .
   Web of Science ®Google Scholar
• Downing , S M . 2004 . Reliability: On the reproducibility of assessment data . Medical Education , 38 : 1006 – 12 .
   PubMed Web of Science ®Google Scholar
• Copeland , H L and Hewson , M G . 2000 . Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center . Academic Medicine , 75 : 161 – 6 .
   PubMed Web of Science ®Google Scholar