Implementation of competence committees during the transition to CBME in Canada: A national fidelity-focused evaluation

References

• Acai A, Cupido N, Weavers A, Saperson K, Ladhani M, Cameron S, Sonnadara RR. 2021. Competence committees: the steep climb from concept to implementation. Med Educ. 55(9):1067–1077.
   PubMed Web of Science ®Google Scholar
• Andolsek K, Padmore J, Hauer K, Ekpenyong A, Edgar L, Holmboe E. 2020. Clinical competency committees - A guidebook for programs. 3rd ed. Chicago (IL): Accreditation Council for Graduate Medical Education.
   Google Scholar
• Battistone MJ, Milne C, Sande MA, Pangaro LN, Hemmer PA, Shomaker TS. 2002. The feasibility and acceptability of implementing formal evaluation sessions and using descriptive vocabulary to assess student performance on a clinical clerkship. Teach Learn Med. 14(1):5–10.
   PubMed Web of Science ®Google Scholar
• Beeson MS, Carter WA, Christopher TA, Heidt JW, Jones JH, Meyer LE, Promes SB, Rodgers KG, Shayne PH, Swing SR, et al. 2013. The development of the emergency medicine milestones. Acad Emerg Med. 20(7):724–729.
   PubMed Web of Science ®Google Scholar
• Bhanji F, Miller G, Cheung WJ, Puligandla PS, Winthrop A, Baird R, Davies D, Lopushinsky SR, Webber EM. 2020. The future is here! Pediatric surgery and the move to the royal college of physicians and surgeons of Canada’s competence by design. J Pediatr Surg. 55(5):796–799.
   PubMed Web of Science ®Google Scholar
• Chahine S, Cristancho S, Padgett J, Lingard L. 2017. How do small groups make decisions: a theoretical framework to inform the implementation and study of clinical competency committees Perspect Med Educ. 6(3):192–198.
   PubMed Web of Science ®Google Scholar
• Chan T, Oswald A, Hauer KE, Caretta-Weyer HA, Nousiainen MT, Cheung WJ, ICBME Collaborators. 2021. Diagnosing conflict: conflicting data, interpersonal conflict, and conflicts of interest in clinical competency committees. Med Teach. 43(7):765–773.
   PubMed Web of Science ®Google Scholar
• Chan T, Sebok-Syer S, Thoma B, Wise A, Sherbino J, Pusic M. 2018. Learning analytics in medical education assessment: the past, the present, and the future. Promes S, editor. AEM Educ Train. 2(2):178–187.
   PubMedGoogle Scholar
• Chan TM, Sebok ‐Syer SS, Cheung WJ, Pusic M, Stehman C, Gottlieb M. 2021. Workplace‐based assessment data in emergency medicine: a scoping review of the literature. AEM Educ Train. 5(3):1–12.
   Google Scholar
• Cheung WJ, Chan TM, Hauer KE, Woods RA, McEwen J, Martin LJ, Patocka C, Dong SL, Bhimani M, McColl T. 2020. CAEP 2019 academic symposium: got competence? Best practices in trainee progress decisions. CJEM. 22(2):187–193.
   PubMedGoogle Scholar
• Cook DA, Kuper A, Hatala R, Ginsburg S. 2016. When assessment data are words: validity evidence for qualitative educational assessments. Acad Med. 91(10):1359–1369.
   PubMed Web of Science ®Google Scholar
• Cruess SR, Cruess RL. 2004. Professionalism and medicine’s social contract with society. Clin Orthop Relat Res. 6(4):12–16.
   Google Scholar
• Dagnone JD, Buttemer S, Hall J, Berger L, Weersink K. 2020. Ten ways to get a grip on resident co-production within medical education change. Can Med Educ J. 11(1):e124–e129.
   PubMedGoogle Scholar
• Dennis AR. 1996. Information exchange and use in small group decision making. Small Gr Res. 27(4):532–550.
   Web of Science ®Google Scholar
• Dickey CC, Thomas C, Feroze U, Nakshabandi F, Cannon B. 2017. Cognitive demands and bias: challenges facing clinical competency committees. J Grad Med Educ. 9(2):162–164.
   PubMedGoogle Scholar
• Donato AA, Alweis R, Wenderoth S. 2016. Design of a clinical competency committee to maximize formative feedback. J Community Hosp Intern Med Perspect. 6(6):33533.
   PubMed Web of Science ®Google Scholar
• Doty CI, Roppolo LP, Asher S, Seamon JP, Bhat R, Taft S, Graham A, Willis J. 2015. How do emergency medicine residency programs structure their clinical competency committees? A survey. Acad Emerg Med. 22(11):1351–1354.
   PubMed Web of Science ®Google Scholar
• Ekpenyong A, Baker E, Harris I, Tekian A, Abrams R, Reddy S, Park YS. 2017. How do clinical competency committees use different sources of data to assess residents’ performance on the internal medicine milestones? A mixed methods pilot study. Med Teach. 39(10):1074–1083.
   PubMed Web of Science ®Google Scholar
• Ekpenyong A, Becker K. 2021. What resources do clinical competency committees (CCCs) require to do their work? A pilot study comparing CCCs across specialties. Med Teach. 43(1):86–92.
   PubMed Web of Science ®Google Scholar
• French JC, Dannefer EF, Colbert CY. 2014. A systematic approach toward building a fully operational clinical competency committee. J Surg Educ. 71(6):e22–e27.
   PubMedGoogle Scholar
• Friedman KA, Raimo J, Spielmann K, Chaudhry S. 2016. Resident dashboards: helping your clinical competency committee visualize trainees' key performance indicators. Med Educ Online. 21(1):29838.
   PubMed Web of Science ®Google Scholar
• van Ginkel WP, van Knippenberg D. 2009. Knowledge about the distribution of information and group decision making: When and why does it work? Organ Behav Hum Decis Process. 108(2):218–229.
   Web of Science ®Google Scholar
• Ginsburg S, McIlroy J, Oulanova O, Eva K, Regehr G. 2010. Toward authentic clinical evaluation: pitfalls in the pursuit of competency. Acad Med. 85(5):780–786.
   PubMed Web of Science ®Google Scholar
• Ginsburg S, van der Vleuten CPM, Eva KW. 2017. The hidden value of narrative comments for assessment: a quantitative reliability analysis of qualitative data. Acad Med. 92(11):1617–1621.
   PubMed Web of Science ®Google Scholar
• Govaerts M, van der Vleuten CP. 2013. Validity in work-based assessment: expanding our horizons. Med Educ. 47(12):1164–1174.
   PubMed Web of Science ®Google Scholar
• Hatala R, Sawatsky AP, Dudek N, Ginsburg S, Cook DA. 2017. Using In-Training Evaluation Report (ITER) qualitative comments to assess medical students and residents: a systematic review. Acad Med. 92(6):868–879.
   PubMed Web of Science ®Google Scholar
• Hauer KE, Cate OT, Boscardin CK, Iobst W, Holmboe ES, Chesluk B, Baron RB, O'Sullivan PS. 2016. Ensuring resident competence: a narrative review of the literature on group decision making to inform the work of clinical competency committees. J Grad Med Educ. 8(2):156–164.
   PubMedGoogle Scholar
• Hauer KE, Chesluk B, Iobst W, Holmboe E, Baron RB, Boscardin CK, Cate OT, O’Sullivan PS. 2015. Reviewing residents’ competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. 90(8):1084–1092. http://content.wkhealth.com/linkback/openurl?sid=WKPTLP
   PubMed Web of Science ®Google Scholar
• Hodges BD, Ginsburg S, Cruess R, Cruess S, Delport R, Hafferty F, Ho M-J, Holmboe E, Holtman M, Ohbu S, et al. 2011. Assessment of professionalism: recommendations from the Ottawa 2010 Conference. Med Teach. 33(5):354–363. http://www.ncbi.nlm.nih.gov/pubmed/21517683.
   PubMedGoogle Scholar
• Ketteler ER, Auyang ED, Beard KE, McBride EL, McKee R, Russell JC, Szoka NL, Nelson MT. 2014. Competency champions in the clinical competency committee: a successful strategy to implement milestone evaluations and competency coaching. J Surg Educ. 71(1):36–38.
   PubMed Web of Science ®Google Scholar
• Kinnear B, Warm EJ, Hauer KE. 2018. Twelve tips to maximize the value of a clinical competency  committee in postgraduate medical education. Med Teach. 40(11):1110–1115.
   PubMed Web of Science ®Google Scholar
• Klocke U. 2007. How to improve decision making in small groups. Small Gr Res. 38(3):437–468.
   Web of Science ®Google Scholar
• Lu L, Yuan YC, McLeod PL. 2012. Twenty-five years of hidden profiles in group decision making: a meta-analysis. Pers Soc Psychol Rev. 16(1):54–75.
   PubMed Web of Science ®Google Scholar
• Van Melle E, Hall A, Schumacher D, Kinnear B, Gruppen L, Thoma B, Caretta-Weyer H, Cooke L, Frank J. 2021. Capturing outcomes of competency-based medical education: The call and the challenge. Med Teach. 43(7):794–800.
   PubMed Web of Science ®Google Scholar
• Michaelsen LK, Watson WE, Black RH. 1989. A realistic test of individual versus group consensus decision making. J Appl Psychol. 74(5):834–839.
   Web of Science ®Google Scholar
• Odorizzi S, Cheung WJ, Sherbino J, Lee AC, Thurgur L, Frank JR. 2020. Signal through the noise: Do professionalism concerns impact the decision making of competence committees? Acad Med. 95(6):896-901.
   Google Scholar
• Pack R, Lingard L, Watling C, Cristancho S. 2020. Beyond summative decision making: illuminating the broader roles of competence committees. Med Educ. 54(6):517–527.
   PubMed Web of Science ®Google Scholar
• Pack R, Lingard L, Watling CJ, Chahine S, Cristancho SM. 2019. Some assembly required: tracing the interpretative work of Clinical Competency Committees. Med Educ. 53(7):723–734.
   PubMed Web of Science ®Google Scholar
• Promes SB, Wagner MJ. 2014. Starting a clinical competency committee. J Grad Med Educ. 6(1):163–164. http://www.jgme.org/doi/abs/10.4300/JGME-D-13-00444.1%5Cnpapers2://publication/doi/10.4300/JGME-D-13-00444.1.
   PubMedGoogle Scholar
• [RCPSC] Royal College of Physicians and Surgeons of Canada. 2017. Competence committees. [accessed 2017 Sep 13]. http://www.royalcollege.ca/rcsite/cbd/assessment/competence-committees-e.
   Google Scholar
• [RCPSC] Royal College of Physicians and Surgeons of Canada. 2018a. Competence committee guidelines - terms of reference. [accessed 2019 Jul 20]. https://www.royalcollege.ca/rcsite/documents/cbd/competence-committees-guidelines-for-terms-of-reference-e.
   Google Scholar
• [RCPSC] Royal College of Physicians and Surgeons of Canada. 2018b. Competence committee guideline: process and procedures in decision making. [accessed 2019 Jul 20]. https://www.royalcollege.ca/rcsite/documents/cbd/competence-committees-process-procedures-e.
   Google Scholar
• [RCPSC] Royal College of Physicians and Surgeons of Canada 2019. Competence by design (CBD): cost analysis. Ottawa (ON). https://www.royalcollege.ca/rcsite/documents/cbd/cbd-costing-analysis-executive-version-e.pdf.
   Google Scholar
• [RCPSC] Royal College of Physicians and Surgeons of Canada. 2020a. Competence by design technical guide 3: competence committees. [accessed 2020 Nov 23]. https://www.royalcollege.ca/rcsite/documents/cbd/cbd-technical-guide-3-comp-committees-e.pdf.
   Google Scholar
• [RCPSC] Royal College of Physicians and Surgeons of Canada. 2020b. Ready for launch? Checking off readiness factors for CBD implementation. [accessed 2021 Jan 10]. https://www.royalcollege.ca/rcsite/documents/cbd/readiness-to-implement-report-2020-e.pdf.
   Google Scholar
• [RCPSC] Royal College of Physicians and Surgeons of Canada. 2021. EPAs and CanMEDS milestones. [accessed 2021 Apr 13]. https://www.royalcollege.ca/rcsite/cbd/implementation/cbd-milestones-epas-e.
   Google Scholar
• Schumacher DJ, Kinnear B. 2020. Is the proof in the PUDding? Reflections on previously undocumented data (PUD) in clinical competency committees. Perspect Med Educ. 9(5):269–271.
   PubMed Web of Science ®Google Scholar
• Schuwirth LWT, van der Vleuten CPM. 2006. A plea for new psychometric models in educational assessment. Med Educ. 40(4):296–300.
   PubMed Web of Science ®Google Scholar
• Sherbino J, Bandiera G, Doyle K, Frank JR, Holroyd BR, Jones G, Norum J, Snider C, Magee K. 2020. The competency-based medical education evolution of Canadian emergency medicine specialist training. CJEM. 22(1):95–102.
   PubMedGoogle Scholar
• Silber CG, Nasca TJ, Paskin DL, Eiger G, Robeson M, Veloski JJ. 2004. Do global rating forms enable program directors to assess the ACGME competencies? Acad Med. 79(6):549–556.
   PubMed Web of Science ®Google Scholar
• Stasson MF, Kameda T, Davis JH. 1997. A model of agenda influences on group decisions. Gr Dyn Theory Res Pract. 1(4):316–323.
   Google Scholar
• Tam J, Wadhwa A, Martimianakis MA, Fernando O, Regehr G. 2020. The role of previously undocumented data in the assessment of medical trainees in clinical competency committees. Perspect Med Educ. 9(5):286–293.
   PubMed Web of Science ®Google Scholar
• Thoma B, Bandi V, Carey R, Mondal D, Woods R, Martin L, Chan T. 2020. Developing a dashboard to meet Competence Committee needs: a design-based research project. Can Med Educ J. 11(1):e16–e34.
   PubMedGoogle Scholar
• Thoma B, Caretta-Weyer H, Schumacher DJ, Warm E, Hall AK, Hamstra SJ, Cavalcanti R, Chan TM, ICBME Collaborators. 2021. Becoming a deliberately developmental organization: using competency based assessment data for organizational development. Med Teach. 43(7):801–809.
   PubMed Web of Science ®Google Scholar
• Thoma B, Ellaway RH, Chan TM. 2021. From Utopia through dystopia: charting a course for learning analytics in competency-based medical education. Acad Med. 96(7S):S89–S95.
   PubMed Web of Science ®Google Scholar
• Tindale RS, Kameda T. 2000. Social sharedness’ as a unifying theme for information processing in groups. Gr Process Intergr Relations. 3(2):123–140.
   Google Scholar
• Varpio L, Bell R, Hollingworth G, Jalali A, Haidet P, Levine R, Regehr G. 2012. Is transferring an educational innovation actually a process of transformation? Adv Health Sci Educ Theory Pract. 17(3):357–367.
   PubMed Web of Science ®Google Scholar
• Xiao L, Eastmure V. 2014. Information use in group decision making teams. Proc Am Soc Info Sci Tech. 51(1):1–4.
   Google Scholar
• Zhang R, Pakhomov S, Gladding S, Aylward M, Borman-Shoap E, Melton GB. 2012. Automated assessment of medical training evaluation text. AMIA Annu Symp Proc. 2012:1459–1468.
   PubMedGoogle Scholar