Advanced search
Publication Cover
Medical Teacher Volume 35, 2013 - Issue 1
Submit an article Journal homepage
2,003
Views
23
CrossRef citations to date
0
Altmetric
Articles

Top five flashpoints in the assessment of teaching effectiveness

Ronald A. BerkJohns Hopkins University, USA; RONALD A. BERK, PhD, is Professor Emeritus, Biostatistics and Measurement, and former Assistant Dean for Teaching at the Johns Hopkins University, where he taught for 30 years. He has presented 400 keynotes/workshops and published 14 books, 165 journal articles, and 300 blogs. His professional motto is: “Go for the Bronze!”Correspondence[email protected]
Pages 15-26 | Published online: 26 Oct 2012

References

Medical/healthcare education

  • Ahmady S, Changiz T, Brommels M, Gaffney FA, Thor J, Masiello I. Contextual adaptation of the Personnel Evaluation Standards for assessing faculty evaluation systems in developing countries: The case of Iran. BMC Med Educ 2009; 9(18) DOI: 10.1186/1472-6920-9-18
  • Anderson HM, Cain J, Bird E. 2005. Online student course evaluations: Review of literature and a pilot study. Am J Pharm Educ 69(1):34–43. Available from http://web.njit.edu/~bieber/pub/Shen-AMCIS2004.pdf
  • Appling SE, Naumann PL, Berk RA. Using a faculty evaluation triad to achieve evidenced-based teaching. Nurs Health Care Perspect 2001; 22: 247–251
  • Barnett CW, Matthews HW. Teaching evaluation practices in colleges and schools of pharmacy. Am J Pharm Educ 2009; 73(6)
  • Barnett CW, Matthews HW, Jackson RA. A comparison between student ratings and faculty self-ratings of instructional effectiveness. J Pharm Educ 2003; 67(4)
  • Berk RA. Using the 360° multisource feedback model to evaluate teaching and professionalism. Med Teach 2009a; 31(12)1073–1080
  • Berk RA, Naumann PL, Appling SE. Beyond student ratings: Peer observation of classroom and clinical teaching. Int J Nurs Educ Scholarsh 2004; 1(1)1–26
  • Boerboom TBB, Mainhard T, Dolmans DHJM, Scherpbier AJJA, van Beukelen P, Jaarsma ADC. Evaluating clinical teachers with the Maastricht clinical teaching questionnaire: How much ‘teacher’ is in student ratings?. Med Teach 2012; 34(4)320–326
  • Chenot J-F, Kochen MM, Himmel W. Student evaluation of a primary care clerkship: Quality assurance and identification of potential for improvement. BMC Med Educ 2009; 9(17) DOI: 10.1186/1472-6920-9-17
  • DiVall M, Barr J, Gonyeau M, Matthews SJ, van Amburgh J, Qualters D, Trujillo J. Follow-up assessment of a faculty peer observation and evaluation program. Am J Pharm Educ 2012; 76(4)
  • Donnon T, Delver H, Beran T. Student and teaching characteristics related to ratings of instruction in medical sciences graduate programs. Med Teach 2010; 32(4)327–332
  • Elzubeir M, Rizk D. Evaluating the quality of teaching in medical education: Are we using the evidence for both formative and summative purposes?. Med Teach 2002; 24: 313–319
  • Hoeks TW, van Rossum HJ. The impact of student ratings on a new course: The general clerkship (ALCO). Med Educ 1988; 22(4)308–313
  • Jones RF, Froom JD. Faculty and administration views of problems in faculty evaluation. Acad Med 1994; 69(6)476–483
  • Kidd RS, Latif DA. Student evaluations: Are they valid measures of course effectiveness?. J Pharm Educ 2004; 68(3)
  • Maker VK, Lewis MJ, Donnelly MB. Ongoing faculty evaluations: Developmental gain or just more pain?. Curr Surg 2006; 63(1)80–84
  • Martens MJ, Duvivier RJ, van Dalen J, Verwijnen GM, Scherpbier AJ, van der Vleuten. Student views on the effective teaching of physical examination skills: A qualitative study. Med Educ 2009; 43(2)184–191
  • Mazor K, Clauser B, Cohen A, Alper E, Pugnaire M. The dependability of students’ rating of preceptors. Acad Med 1999; 74: 19–21
  • Pattison AT, Sherwood M, Lumsden CJ, Gale A, Markides M. Foundation observation of teaching project – A developmental model of peer observation of teaching. Med Teach 2012; 34(2)e36–e142
  • Pierre RB, Wierenga A, Barton M, Branday JM, Christie CD. Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica. BMC Med Educ 2004; 4(22) DOI: 10.1186/1472-6920-4-22
  • Schiekirka S, Reinhardt D, Heim S, Fabry G, Pukrop T, Anders S, Raupach T. Student perceptions of evaluation in undergraduate medical education: A qualitative study from one medical school. BMC Med Educ 2012; 12(45) DOI: 10.1186/1472-6920-12-45
  • Siddiqui ZS, Jonas-Dwyer D, Carr SE. Twelve tips for peer observation of teaching. Med Teach 2007; 29(4)297–300
  • Stalmeijer RE, Dolmans DH, Wolfhagen IH, Peters WG, van Coppenolle L, Scherpbier AJ. Combined student ratings and self-assessment provide useful feedback for clinical teachers. Adv Health Sci Educ Theory Pract 2010; 15(3)315–328
  • Stark P. Teaching and learning in the clinical setting: A qualitative study of the perceptions of students and teachers. Med Educ 2003; 37(11)975–982
  • Steinert Y. Student perceptions of effective small group teaching. Med Educ 2004; 38(3)286–293
  • Sullivan PB, Buckle A, Nicky G, Atkinson SH. Peer observation of teaching as a faculty development tool. BMC Med Educ 2012; 12(26) DOI: 10.1186/1472-6920-12-26
  • Turhan K, Yaris F, Nural E. Does instructor evaluation by students using a web-based questionnaire impact instructor performance?. Adv Health Sci Educ Theory Pract 2005; 10(1)5–13
  • Wellein MG, Ragucci KR, Lapointe M. A peer review process for classroom teaching. Am J Pharm Educ 2009; 73(5)

General higher education

  • Abrami PC. Improving judgments about teaching effectiveness using rating forms. The student ratings debate: Are they valid? How can we best use them? (New Directions for Institutional Research, No. 109), M Theall, PC Abrami, LA Mets. Jossey-Bass, San Francisco, CA 2001; 59–87
  • Addison WE, Stowell JR. 2012. Conducting research on student evaluations of teaching. In: Kite ME, editor. Effective evaluation of teaching: A guide for faculty and administrators. pp 1–12. E-book [Accessed 6 June 2012] Available from the Society for the Teaching of Psychology website http://teachpsych.org/ebooks/evals2012/index.php
  • AERA (American Educational Research Association), APA (American Psychological Association), NCME (National Council on Measurement in Education) Joint Committee on Standards. 1999. Standards for educational and psychological testing. Washington, DC: AERA.
  • Ali DL, Sell Y. Issues regarding the reliability, validity and utility of student ratings of instruction: A survey of research findings. University of Calgary APC Implementation Task Force on Student Ratings of Instruction, Calgary, AB 1998
  • Arreola RA. Developing a comprehensive faculty evaluation system: A handbook for college faculty and administrators on designing and operating a comprehensive faculty evaluation system3rd. Anker, Bolton, MA 2007
  • Avery RJ, Bryan WK, Mathios A, Kang H, Bell D. Electronic course evaluations: Does an online delivery system influence student evaluations?. J Econ Educ 2006; 37(1)21–37
  • Benton SL, Cashin WE. Student ratings of teaching: A summary of research and literature (IDEA Paper no 50). The IDEA Center., Manhattan, KS 2012, [Accessed 8 April 2012] Available from http://www.theideacenter.org/sites/default/files/idea-paper_50.pdf
  • Benton SL, Webster R, Gross A, Pallett W. An analysis of IDEA Student Ratings of Instruction in traditional versus online courses (IDEA Technical Report no 15). The IDEA Center, Manhattan, KS 2010a
  • Benton SL, Webster R, Gross A, Pallett W. An analysis of IDEA Student Ratings of Instruction using paper versus online survey methods (IDEA Technical Report no. 16). The IDEA Center, Manhattan, KS 2010b
  • Berk RA. The construction of rating instruments for faculty evaluation: A review of methodological issues. J Higher Educ 1979; 50: 650–669
  • Berk RA, 2005. Survey of 12 strategies to measure teaching effectiveness. Int J Teac Learn Higher Educ 17(1):48−62. Available from http://www.isetl.org/ijtlthe
  • Berk RA. Thirteen strategies to measure college teaching: A consumer's guide to rating scale construction, assessment, and decision making for faculty, administrators, and clinicians. Stylus, Sterling, VA 2006
  • Berk RA, 2009b. Beyond student ratings: “A whole new world, a new fantastic point of view.” Essays Teach Excellence 20(1). Available from http://podnetwork.org/publications/teachingexcellence.htm
  • Berk RA. The secret to the “best” ratings from any evaluation scale. J Faculty Dev 2010; 24(1)37–39
  • Braskamp LA, Ory JC. Assessing faculty work: Enhancing individual and institutional performance. Jossey-Bass, San Francisco, CA 1994
  • Calderon TG, Gabbin AL, Green BP. Report of the committee on promoting evaluating effective teaching. James Madison University, Harrisonburg, VA 1996
  • Carini RM, Hayek JC, Kuh GD, Ouimet JA. College student responses to web and paper surveys: Does mode matter?. Res Higher Educ 2003; 44(1)1–19
  • Carrier NA, Howard GS, Miller WG. Course evaluations: When?. J Educ Psychol 1974; 66: 609–613
  • Cashin WE. Evaluating college and university teaching: Reflections of a practitioner. Higher education: Handbook of theory and research, JC Smart. Kluwer Academic Publishers, Dordrechtthe Netherlands 2003; 531–593
  • Centra JA. Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. Jossey-Bass, San Francisco 1993
  • Cohen PA, McKeachie WJ. The role of colleagues in the evaluation of teaching. Improving College Univ Teach 1980; 28(4)147–154
  • Coren S. Are course evaluations a threat to academic freedom?. Academic freedom and the inclusive university, SE Kahn, D Pavlich. University of British Columbia Press, Vancouver, BC 2001; 104–117
  • d’Apollonia S, Abrami PC. Navigating student ratings of instruction. Am Psychol 1997a; 52: 1198–1208
  • d’Apollonia S, Abrami PC. Scaling the ivory tower, part 1: Collecting evidence of instructor effectiveness. Psychol Teach Rev 1997b; 6: 46–59
  • d’Apollonia S, Abrami PC. Scaling the ivory tower, part 2: Student ratings of instruction in North America. Psychol Teach Rev 1997c; 6: 60–76
  • deVellis RF. Scale development: Theory and applications3rd. Sage, Thousand Oaks, CA 2012
  • Dommeyer CJ, Baum P, Hanna RW, Chapman KS. Gathering faculty teaching evaluations by in-class and online surveys: Their effects on response rates and evaluations. Assess Eval Higher Educ 2004; 29(5)611–623
  • Donovan J, Mader CE, Shinsky J. Constructive student feedback: Online vs. traditional course evaluations. J Interact Online Learn 2006; 5: 283–296
  • Dunn-Rankin P, Knezek GA, Wallace S, Zhang S. Scaling methods. Erlbaum, Mahwah, NJ 2004
  • Franklin J. Interpreting the numbers: Using a narrative to help others read student evaluations of your teaching accurately. Techniques and strategies for interpreting student evaluations (Special issue) (New Directions for Teaching and Learning, No. 87), KG Lewis. Jossey-Bass, San Francisco, CA 2001; 85–100
  • Franklin J, Theall M. Communicating student ratings to decision makers: Design for good practice. Student ratings of instruction: Issues for improving practice (Special issue) (New Directions for Teaching and Learning, No. 43), M Theall, J Franklin. Jossey-Bass, San Francisco, CA 1990; 75–93
  • Frey PW. Validity of student instructional ratings as a function of their timing. J Higher Educ 1976; 47: 327–336
  • Freyd M. A graphic rating scale for teachers. J Educ Res 1923; 8(5)433–439
  • Gamliel E, Davidovitz L. Online versus traditional teaching evaluation: Mode can matter. Assess Eval Higher Educ 2005; 30(6)581–592
  • Gravestock P, Gregor-Greenleaf E. Student course evaluations: Research, models and trends. Higher Education Quality Council of Ontario., Toronto, ON 2008, E-book [Accessed 6 May 2012] Available from http://www.heqco.ca/en-CA/Research/Research%20Publications/Pages/Home.aspx
  • Green BP, Calderon TG, Reider BP. A content analysis of teaching evaluation instruments used in accounting departments. Issues Account Educ 1998; 13(1)15–30
  • Hardy N (2002) Perceptions of online evaluations: Fact and fiction. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, April 1–5, 2002, LA
  • Hardy N. Online ratings: Fact and fiction. Online student ratings of instruction (New Directions for Teaching and Learning, No. 96), DL Sorenson, TD Johnson. Jossey-Bass, San Francisco, CA 2003; 31–38
  • Heath N, Lawyer S, Rasmussen E. Web-based versus paper and pencil course evaluations. Teach Psychol 2007; 34(4)259–261
  • Hong PC. Evaluating teaching and learning from students’ perspectives in their classroom through easy-to-use online surveys. Int J Cyber Soc Educ 2008; 1(1)33–48
  • Hoyt DP, Pallett WH. Appraising teaching effectiveness: Beyond student ratings (IDEA Paper no 36). Kansas State University Center for Faculty Evaluation and Development, Manhattan, KS 1999
  • Johnson TD. Online student ratings: Research and possibilities. Invited plenary presented at the Online Assessment Conference, SeptemberChampaign, 2001, IL
  • Johnson TD. Online student ratings: Will students respond?. Online student ratings of instruction (New Directions for Teaching and Learning, no. 96), DL Sorenson, TD Johnson. Jossey-Bass, San Francisco, CA 2003; 49–60
  • Joint Committee on Standards for Educational Evaluation. The personnel evaluation standards: How to assess systems for evaluating educators2nd. Corwin Press, Thousand Oaks, CA 2009
  • Kite ME, editor. 2012. Effective evaluation of teaching: A guide for faculty and administrators. E-book [Accessed 6 June 2012] Available from the Society for the Teaching of Psychology website http://teachpsych.org/ebooks/evals2012/index.php
  • Knapper C, Cranton P, editors. 2001. Fresh approaches to the evaluation of teaching (New Directions for Teaching and Learning, no. 88). San Francisco, CA: Jossey-Bass. pp 19–29.
  • Layne BH, DeCristoforo JR, McGinty D. Electronic versus traditional student ratings of instruction. Res Higher Educ 1999; 40(2)221–232
  • Leung DYP, Kember D. Comparability of data gathered from evaluation questionnaires on paper through the Internet. Res Higher Educ 2005; 46: 571–591
  • Liu Y. A comparison of online versus traditional student evaluation of instruction. Int J Instr Technol Distance Learn 2006; 3(3)15–30
  • MarketTools. 2006. Zoomerang: Easiest way to ask, fastest way to know. [Accessed 17 July 2012] Available from http://info.zoomerang.com
  • Marsh HW. Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. The scholarship of teaching and learning in higher education: An evidence-based perspective, RP Perry, JC Smart. Springer, Dordrechtthe Netherlands 2007; 319–383
  • McGee DE, Lowell N. Psychometric properties of student ratings of instruction in online and on-campus courses. Online student ratings of instruction (New Directions for Teaching and Learning, no. 96), DL Sorenson, TD Johnson. Jossey-Bass, San Francisco, CA 2003; 39–48
  • Morrison R. A comparison of online versus traditional student end-of-course critiques in resident courses. Assess Eval Higher Educ 2011; 36(6)627–641
  • Netemeyer RG, Bearden WO, Sharma S. Scaling procedures. Sage, Thousand Oaks, CA 2003
  • Oliver RL, Sautter EP. Using course management systems to enhance the value of student evaluations of teaching. J Educ Bus 2005; 80(4)231–234
  • Ory JC. Faculty thoughts and concerns about student ratings. Techniques and strategies for interpreting student evaluations (Special issue) (New Directions for Teaching and Learning, No. 87), KG Lewis. Jossey-Bass, San Francisco, CA 2001; 3–15
  • Ory JC, Ryan K. How do student ratings measure up to a new validity framework?. The student ratings debate: Are they valid? How can we best use them? (Special issue) (New Directions for Institutional Research, 109), M Theall, PC Abrami, LA Mets. Jossey-Bass, San Francisco, CA 2001; 27–44
  • Peer E, Gamliel E. Too reliable to be true? Response bias as a potential source of inflation in paper and pencil questionnaire reliability. Practical Assess Res Eval 2011; 16(9)1–8, Available from http://pareonline.net/getvn.asp?v=16%n=9
  • Perrett JJ. Exploring graduate and undergraduate course evaluations administered on paper and online: A case study. Assess Eval Higher Educ 2011; 1–9 DOI: 10.1080/02602938.2011.604123
  • Ravelli B, 2000. Anonymous online teaching assessments: Preliminary findings. [Accessed 12 June 2012] Available from http://www.edrs.com/DocLibrary/0201/ED445069.pdf
  • Seldin P. Current practices – good and bad – nationally. Seldin P & Associates Changing practices in evaluating teaching: A practical guide to improved faculty performance and promotion/tenure decisions. 1999, Bolton, MA: Anker. 1–24
  • Seldin P. Building a successful evaluation program. Seldin P & Associates Evaluating faculty performance: A practical guide to assessing teaching, research, and service. 2006, Bolton, MA: Anker 1–19
  • Seldin P, Associates, editors. 2006. Evaluating faculty performance: A practical guide to assessing teaching, research, and service. Bolton, MA: Anker. pp 201–216
  • Sorenson DL, Johnson TD, editors. 2003. Online student ratings of instruction (New Directions for Teaching and Learning, no. 96). San Francisco, CA: Jossey-Bass.
  • Spooner F, Jordan L, Algozzine B, Spooner M. Student ratings of instruction in distance learning and on-campus classes. J Educ Res 1999; 92: 132–140
  • Stehle S, Spinath B, Kadmon M. Measuring teaching effectiveness: Correspondence between students’ evaluations of teaching and different measures of student learning. Res Higher Educ. 2012 DOI: 10.1007/s11162-012-9260-9
  • Stowell JR, Addison WE, Smith JL. Comparison of online and classroom-based student evaluations of instruction. Assess Eval Higher Educ 2012; 37(4)465–473
  • Strategy Group. 2011. National strategy for higher education to 2030 (Report of the Strategy Group). Dublin, Ireland: Department of Education and Skills, Government Publications Office. [Accessed 17 July 2012] Available from http://www.hea.ie/files/files/DES_Higher_Ed_Main_Report.pdf
  • Streiner DL, Norman GR. Health measurement scales: A practical guide to their development and use4th. Oxford University Press, New York 2008
  • Surgenor PWG. Obstacles and opportunities: Addressing the growing pains of summative student evaluation of teaching. Assess Eval Higher Educ 2011 DOI: 10.1080/02602938.2011.635247, 1–14, iFirst Article
  • Svinicki M, McKeachie WJ. McKeachie's teaching tips: Strategies, research, and theory for college and university teachers13th. Wadsworth, Belmont, CA 2011
  • Theall M, Feldman KA. Commentary and update on Feldman's (1997) “Identifying exemplary teachers and teaching: Evidence from student ratings”. The teaching and learning in higher education: An evidence-based perspective, RP Perry, JC Smart. Springer, Dordrechtthe Netherlands 2007; 130–143
  • Theall M, Franklin JL. Student ratings in the context of complex evaluation systems. Student ratings of instruction: Issues for improving practice (New Directions for Teaching and Learning, no. 43), M Theall, JL Franklin. Jossey-Bass, San Francisco, CA 1990; 17–34
  • Theall M, Franklin JL. Creating responsive student ratings systems to improve evaluation practice. Evaluating teaching in higher education: A vision for the future (Special issue) (New Directions for Teaching and Learning, no. 83), KE Ryan. Jossey-Bass, San Francisco, CA 2000; 95–107
  • Theall M, Franklin JL. Looking for bias in all the wrong places: A search for truth or a witch hunt in student ratings of instruction?. The student ratings debate: Are they valid? How can we best use them? (New Directions for Institutional Research, no. 109), M Theall, PC Abrami, LA Mets. Jossey-Bass, San Francisco, CA 2001; 45–56
  • Commission US Equal Employment Opportunity, (EEOC). 2010. Employment tests and selection procedures. [Accessed 20 August 2012] Available from http://www.eeoc.gov/policy/docs/factemployment_procedures.html
  • Venette S, Sellnow D, McIntire K. Charting new territory: Assessing the online frontier of student ratings of instruction. Assess Eval Higher Educ 2010; 35: 101–115
  • Waschull SB. The online delivery of psychology courses: Attrition, performance, and evaluation. Comput Teach 2001; 28: 143–147
  • Wright KB, 2005. Researching internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. J Comput Mediated Commun 10(3). Available from http://jcmc.indiana.edu/vol10/issue3/wright.html
  • Yarbrough DB, Shulha LM, Hopson RK, Caruthers FA. The program evaluation standards: A guide for evaluators and evaluation users3rd. Sage, Thousand Oaks, CA 2011

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Order Reprints Request Corporate Permissions

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

Request Academic Permissions

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.