Managing assessment during curriculum change: Ottawa Consensus Statement

References

• Amaral E, Norcini J. 2023. Quality assurance in health professions education: role of accreditation and licensure. Med Educ. 57(1):40–48. Epub 2022 Jul 29. PMID: 35851495. doi:10.1111/medu.14880.
   PubMed Web of Science ®Google Scholar
• Anderson LW, Krathwohl D. 2001. A revision of bloom”s taxonomy of educational objectives. New York: longman. Theory into Practice. 41(4):212–218. doi:10.1207/s15430421tip4104_2.
   Web of Science ®Google Scholar
• Antunes dos Santos R, Nunes MdPT 2019. Medical education in Brazil. Med Teach. 41(10):1106–1111. doi:10.1080/0142159X.2019.1636955.
   PubMed Web of Science ®Google Scholar
• Behar LS, Ornstein A. 1994. Domains of curriculum knowledge: an empirical analysis. The High School J. 77(4):322–329. http://www.jstor.org/stable/40364756. Accessed 19 Sept. 2023.
   Google Scholar
• Bezuidenhout MJ, Alt H. 2011 “Assessment drives learning”: do assessments promote high-level cognitive processing? Published Online:1 Jan 2011. [Accessed 10 April 2024]. https://hdl.handle.net/10520/EJC37738.
   Google Scholar
• Biggs J. 1996. Enhancing teaching through constructive alignment. High Educ. 32(3):347–364. doi:10.1007/BF00138871.
   Web of Science ®Google Scholar
• Boursicot K, Etheridge L, Setna Z, Sturrock A, Ker J, Smee S, Sambandam E. 2011. Performance in assessment: consensus statement and recommendations from the Ottawa conference. Med Teach. 33(5):370–383. doi:10.3109/0142159X.2011.565831.
   PubMedGoogle Scholar
• Buckley S, Coleman J, Davison I, Khan KS, Zamora J, Malick S, Morley D, Pollard D, Ashcroft T, Popovic C, et al. 2009. The educational effects of portfolios on undergraduate student learning: a best evidence medical education (BEME) systematic review. BEME Guide No. 11. Med Teach. 31(4):282–298. doi:10.1080/01421590902889897.
   PubMed Web of Science ®Google Scholar
• Burton JL, McDonald S. 2001. Curriculum or syllabus: which are we reforming? Med Teach. 23(2):187–191. doi:10.1080/01421590020031110.
   PubMed Web of Science ®Google Scholar
• Charon R. 2001. Narrative medicine: a model for empathy, reflection, profession, and trust. JAMA. 286(15):1897–1902. doi:10.1001/jama.286.15.1897.
   PubMed Web of Science ®Google Scholar
• Cintra KA, Borges MC, Panúncio-Pinto MP, de Almeida Troncon LE, Bollela VR. 2023. The impact and the challenges of implementing a faculty development program on health professions education in a Brazilian Medical School: a case study with mixed methods. BMC Med Educ. 23(1):784. doi:10.1186/s12909-023-04754-8.
   PubMed Web of Science ®Google Scholar
• Cleland JA, Milne A, Sinclair H, Lee AJ. 2008. Cohort study on predicting grades: is performance on early MBChB assessments predictive of later undergraduate grades? Med Educ. 42(7):676–683. Available doi:10.1111/j.1365-2923.2008.03037.x.
   PubMed Web of Science ®Google Scholar
• Cruess L, Cruess SR, Steinert Y. 2016. Amending Miller”s pyramid to include professional identity. Acad Med. 91(2):180–185. Available doi:10.1097/ACM.0000000000000913.
   PubMed Web of Science ®Google Scholar
• Dann R. 2014. Assessment as learning: blurring the boundaries of assessment and learning for theory, policy and practice. Assess Educ. 21(2):149–166. doi:10.1080/0969594X.2014.898128.
   Google Scholar
• Dave S, Chakravorty I, Menon G, Sidhu K, Bamrah J, Mehta R. 2020. Differential attainment in summative assessments within postgraduate medical education & training: 2020 thematic series on tackling differential attainment in healthcare professions. Sushruta J Health Policy Opinion.. 13(3) doi:10.38192/13.3.15.
   Google Scholar
• de Lisa JA, Lindenthal JJ. 2012. Reflections on diversity and inclusion in medical education. Acad Med. 87(11):1461–1463. doi:10.1097/ACM.0b013e31826b048c.
   PubMed Web of Science ®Google Scholar
• El BM, Anderson K, Finn GM. 2019. A narrative literature review considering the development and implementation of longitudinal integrated clerkships, including a practical guide for application. J Med Educ Curric Dev. 6:238212051984940.
   Web of Science ®Google Scholar
• Ford J, Ford L, Polin B. 2021. Leadership in the implementation of change: functions, sources, and requisite variety. Journal Change Manag. 21(1):87–119. [Accessed 27 February 2024 Available: 10.38192/13.3.15. doi:10.1080/14697017.2021.186169713(3):1–7.
   Web of Science ®Google Scholar
• Foulkes J, Bandaranayake R, Hays R, Phillips G, Rothman A, Southgate L. 1994. The Certification and Recertification of Doctors: Issues in the Assessment of Clinical Competence. Cambridge: Cambridge University Press.
   Google Scholar
• Fuller R, Goddard VCT, Nadarajah VD, Treasure-Jones T, Yeates P, Scott K, Webb A, Valter K, Pyorala E. 2022. Technology enhanced assessment: ottawa consensus statement and recommendations. Med Teach. 44(8):836–850. doi:10.1080/0142159X.2022.2083489.
   PubMed Web of Science ®Google Scholar
• Gelaidan HM, Al-Swidi A, Mabkhot HA. 2017. Leadership Behaviour for Successful Change Management. 2017. In: Farazmand A, Editors. Global Encyclopaedia of Public Administration, Public Policy and Governance. Cham: Springer International Publishing AG. doi:10.1007/978-3-319-31816-5_3138-1. Accessed 27 February 2024
   Google Scholar
• Harden RM, Lilley P. 2018. The eight roles of the medical teacher: the purpose and function of a teacher in the healthcare professions. London: Elsevier Health Sciences.
   Google Scholar
• Hays RB, Hamlin G, Crane L. 2015. 12 tips: increasing the defensibility of assessment. Med Teach. 37(5):433–436. doi:10.3109/0142159X.2014.943711.
   PubMed Web of Science ®Google Scholar
• Hays RB, Lawson M, Gray C. 2011. Problems presented by medical students seeking support: a possible intervention framework. Med Teach. 33(2):161–164. doi:10.3109/0142159X.2010.509415.
   PubMed Web of Science ®Google Scholar
• Hays RB, Ramani S, Hassell A. 2020. Healthcare systems and the sciences of health professional education. Adv Health Sci Educ Theory Pract. 25(5):1149–1162. doi:10.1007/s10459-020-10010-1.
   PubMed Web of Science ®Google Scholar
• Hays RB, Sen Gupta TK, Veitch J. 2008. The practical value of the standard error of measurement in borderline pass/fail decisions. Med Educ. 42(8):810–815. doi:10.1111/j.1365-2923.2008.03103.x.
   PubMed Web of Science ®Google Scholar
• Heeneman S, de Jong LH, Dawson LJ, Wilkinson TJ, Ryan A, Tait GR, Rice N, Torre D, Freeman A, van der Vleuten CPM. 2021. Ottawa 2020 consensus statement for programmatic assessment – 1. Agreement on the principles. Med Teach. 43(10):1139–1148. doi:10.1080/0142159X.2021.1957088.
   PubMed Web of Science ®Google Scholar
• Hong DZ, Lim AJS, Tan R, Ong YT, Pisupati A, Chong EJX, Quek CWN, Lim JY, Ting JJQ, Chiam M, et al. 2021. A systematic scoping review on portfolios of medical educators. J Med Educ Curric Dev. 8:23821205211000356. doi:10.1177/23821205211000356.
   Web of Science ®Google Scholar
• Jackson P. 1968. . Life in classrooms. Retrieved November 2005 from: http://www.kdpoorg/images/Jackson.jprg.
   Google Scholar
• Jürges H, Schneider K, Senkbeil M, Carstensen CH. 2012. Assessment drives learning: the effect of central exit exams on curricular knowledge and mathematical literacy. Econ Educ Rev. 31(1):56–65. doi:10.1016/j.econedurev.2011.08.007.
   Web of Science ®Google Scholar
• Karakitsiou DE, Markou A, Kyriakou P, Pieri M, Abuaita M, Bourousis E, Hido T, Tsatsaragkou A, Boukali A, de Burbure C, et al. 2012. The good student is more than a listener – The 12 + 1 roles of the medical student. [Accessed 10th April 2024]. Med Teach. 34(1):e1–e8. doi:10.3109/0142159X.2012.638006.
   PubMed Web of Science ®Google Scholar
• Kibbe DC, Kaluzny AD, McLaughlin CP. 1994. Integrating guidelines with continuous quality improvement: doing the right thing the right way to achieve the right goals. Jt Comm J Qual Improv. 20(4):181–191. Available doi:10.1016/S1070-3241(16)30060-8.
   PubMedGoogle Scholar
• Law MY. 2022. A review of curriculum change and innovation for higher education. JETS. 10(2):16–23. doi:10.11114/jets.v10i2.5448.
   Google Scholar
• Lefroy J, Watling C, Teunissen PW, Brand P. 2015. Guidelines: the do”s, don”ts and don”t knows of feedback for clinical education. Perspect Med Educ. 4(6):284–299. doi:10.1007/s40037-015-0231-7.
   PubMed Web of Science ®Google Scholar
• Lempp H, Seal C, Jackson PW. 2004. The hidden curriculumin undergraduate medical education: qualitative study of medical students” perceptions of teaching. BMJ. 329(7469):770–773. PMID: 15459051; PMCID: PMC520997. [Accessed 10th April 2024] doi:10.1136/bmj.329.7469.770.
   PubMed Web of Science ®Google Scholar
• Malau-Aduli BS, Hays RB, D”Souza K, Smith AM, Jones K, Turner R, Shires L, Smith J, Saad S, Richmond C, et al. 2021. Examiners” decision-making processes in observation-based clinical examinations. Med Educ. ; 55(3):344–353. doi:10.1111/medu.14357.
   PubMed Web of Science ®Google Scholar
• Meriam-Webster Online Dictionary 2024. Available: merriam-webster.com Accessed 27 February.
   Google Scholar
• Moonen-van Loon JMW, Overeem K, Donkers HHLM, van der Vleuten CPM, Driessen EW. 2013. Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ Theory Pract. 18(5):1087–1102. doi:10.1007/s10459-013-9450-z.
   PubMed Web of Science ®Google Scholar
• Nadarajah VD, Ramani S, Findyartini A, Sathivelu S, Nadkar AA. 2023. Inclusion in global health professions education communities through many lenses. Med Teach. 45(8):799–801. doi:10.1080/0142159X.2023.2186206.
   PubMed Web of Science ®Google Scholar
• Newble DI, Jaeger K. 1983. The effect of assessments and examinations on the learning of medical students. Med Educ. 17(3):165–171. Accessed 27 February 2024. doi:10.1111/j.1365-2923.1983.tb00657.x.
   PubMed Web of Science ®Google Scholar
• Norcini J, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, Galbraith R, Hays R, Kent A, Perrott V, et al. 2011. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach. 33(3):206–214. 10.3109/0142159X.2011.551559.21345060
   PubMed Web of Science ®Google Scholar
• Norcini JJ. 2003. Setting standards on educational tests. Med Educ. 37(5):464–469. Available doi:10.1046/j.1365-2923.2003.01495.x.
   PubMed Web of Science ®Google Scholar
• Norcini J, Anderson MB, Bollela V, Burch V, Costa MJ, Duvivier R, Hays R, Palacios Mackay MF, Roberts T, Swanson D. 2018. Consensus Framework for Good Assessment. Med Teach. 40(11):1102–1109. doi:10.1080/0142159X.2018.1500016.
   PubMed Web of Science ®Google Scholar
• Norcini J, Burch V. 2007. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach. 29(9):855–871. PMID: 18158655. doi:10.1080/01421590701775453.
   PubMed Web of Science ®Google Scholar
• Plochocki JH. 2019. Several ways generation Z may shape the medical school landscape. J Med Educ Curric Dev. 6:2382120519884325. PMID: 31701014; PMCID: PMC6823979. [Accessed 10th April 2024]. doi:10.1177/2382120519884325.
   Web of Science ®Google Scholar
• Rao K, Ok MW, Bryant BR. 2014. A review of research on universal design educational models. Remedial Spec Educ. 35(3):153–166. doi:10.1177/0741932513518980.
   Web of Science ®Google Scholar
• Raupach T, Brown J, Anders S, Hasenfuss G, Harendza S. 2013. Summative assessments are more powerful drivers of student learning than resource intensive teaching formats. BMC Med. 11(1):61. doi:10.1186/1741-7015-11-61.
   PubMedGoogle Scholar
• Raymond MR, Grande JP. 2019. A practical guide to test blueprinting. Med Teach. 41(8):854–861. doi:10.1080/0142159X.2019.1595556.
   PubMed Web of Science ®Google Scholar
• Reyes A, Galvan R, Jr, Navarro A, Velasquez M, Soriano DR, Cabuso AL, David JR, Lacson ML, Manansala NT, Tiongco RE. 2020. Across generations: defining pedagogical characteristics of generation X, Y, and Z allied health teachers using Q-methodology. Med Sci Educ. 30(4):1541–1549. PMID: 34457822; PMCID: PMC8368375. doi:10.1007/s40670-020-01043-7.
   PubMedGoogle Scholar
• Schauber SK, Hecht M. 2020. How sure can we be that a student really failed? On the measurement precision of individual pass-fail decisions from the perspective of Item Response Theory. Med Teach. 42(12):1374–1384. doi:10.1080/0142159X.2020.1811844.
   PubMed Web of Science ®Google Scholar
• Schuwirth LWT, Van Der Vleuten CPM. 2004. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 38(9):974–979. doi:10.1111/j.1365-2929.2004.01916.
   PubMed Web of Science ®Google Scholar
• Schuwirth LWT, Van der Vleuten CPM. 2011. Programmatic assessment: from assessment of learning to assessment for learning. Med Teach. 33(6):478–485. doi:10.3109/0142159X.2011.565828.
   PubMed Web of Science ®Google Scholar
• Sen Gupta T, Wong E, Doshi D, Hays R. 2021. Stability” of assessment: extending the utility equation [version. 1] MedEdPublish. 10:155. doi:10.15694/mep.2021.000155.1.
   PubMedGoogle Scholar
• Tai J, Ajjawi R, Bearman M, Boud D, Dawson P, Jorre de St Jorre T. 2023. Assessment for inclusion: rethinking contemporary strategies in assessment design. Higher Educ Res Dev. 42(2):483–497. doi:10.1080/07294360.2022.2057451.
   Web of Science ®Google Scholar
• Tan R, Qi Ting JJ, Zhihao Hong D, Sing Lim AJ, Ong YT, Pisupati A, Xin Chong EJ, Chiam M, Inn Lee AS, Shuen Tan LH, et al. 2022. Medical student portfolios: a systematic scoping review. J Med Educ Curric Dev. 9:23821205221076022. Accessed 10th April 2024 doi:10.1177/23821205221076022.
   Web of Science ®Google Scholar
• ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. 2015. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach. 37(11):983–1002. doi:10.3109/0142159X.2015.1060308.
   PubMed Web of Science ®Google Scholar
• Torre D, Rice NE, Ryan A, Bok H, Dawson LJ, Bierer B, Wilkinson TJ, Tait GR, Laughlin T, Veerapen K, et al. 2021. Ottawa 2020 consensus statements for programmatic assessment – 2. Implementation and practice. Med Teach. 43(10):1149–1160. doi:10.1080/0142159X.2021.1956681.
   PubMed Web of Science ®Google Scholar
• Twenge JM. 2009. Generational changes and their impact in the classroom: teaching generation Me. Med Educ. 43(5):398–405. doi:10.1111/j.1365-2923.2009.03310.x.
   PubMed Web of Science ®Google Scholar
• Van Der Vleuten CP, M. 1996. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ Theory Pract. 1(1):41–67. doi:10.1007/BF00596229.
   PubMed Web of Science ®Google Scholar
• van der Vleuten CPM, Schuwirth WT. 2005. The assessment of professional competence: developments, research and practical implications. Med Educ. 39(3):309–317. doi:10.1111/j.1365-2929.2005.02094.x.
   PubMed Web of Science ®Google Scholar
• van der Vleuten CPM, Schuwirth LWT, Driessen EW, Dijkstra J, Tigelaar D, Baartman LKJ, van Tartwijk J. 2012. A model for programmatic assessment fit for purpose. Med Teach. 34(3):205–214. doi:10.3109/0142159X.2012.652239.
   PubMed Web of Science ®Google Scholar
• Wald HS, Borkan JM, Taylor JS, Anthony D, Reis SP. 2012. Fostering and evaluating reflective capacity in medical education: developing the REFLECT rubric for assessing reflective writing. Acad Med. 87(1):41–50. Erratum in: Acad Med. 2012 Mar;87(3):355. PMID: 22104060. doi:10.1097/ACM.0b013e31823b55fa.
   PubMed Web of Science ®Google Scholar
• Wilkinson TJ, Wells JE, Bushnell JA. 2007. What is the educational impact of standards-based assessment in a medical degree? Med Educ. 41(6):565–572. doi:10.1111/j.1365-2923.2007.02766.x.
   PubMed Web of Science ®Google Scholar
• Wondimagegn D, Whitehead CR, Cartmill C, Rodrigues E, Correia A, Salessi Lins T, Costa MJ. 2023. Faster, higher, stronger - together? A bibliometric analysis of author distribution in top medical education journals. [BMJ Glob Health. 8(6):e011656. PMID: 37321659; PMCID: PMC10367082. doi:10.1136/bmjgh-2022-011656.
   PubMed Web of Science ®Google Scholar
• Worley P, Couper I, Strasser R, Graves L, Cummings B-A, Woodman R, Stagg P, Hirsh D., 2016. A typology of longitudinal integrated clerkships. Med Educ. 50(9):922–932. doi:10.1111/medu.13084.
   PubMed Web of Science ®Google Scholar
• Worley P, Silagy C, Prideaux D, Newble D, Jones A. 2001. The parallel rural community curriculum: an integrated clinical curriculum based in rural general practice. Med Educ. 34(7):558–565. doi:10.1046/j.1365-2923.2000.00668.x.
   Web of Science ®Google Scholar