References
- Gruppen L , Frank JR , Lockyer J , Ross S , et al. Toward a research agenda for competency-based medical education. Med Teach . 2017;39(6):623–630. doi:https://doi.org/10.1080/0142159X.2017.1315065.
- Van Melle E , Frank JR , Holmboe ES , Dagnone D , Stockley D , Sherbino J. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med . 2019;94(7):1002–1009. doi:https://doi.org/10.1097/ACM.0000000000002743.
- Van Der Vleuten CP , Schuwirth LW , Driessen EW , Govaerts MJ , Heeneman S. Twelve tips for programmatic assessment. Med Teach . 2015;37(7):641–646. doi:https://doi.org/10.3109/0142159X.2014.973388.
- Bok HGJ , de Jong LH , O'Neill T , Maxey C , Hecker KG. Validity evidence for programmatic assessment in competency-based education. Perspect Med Educ . 2018;7(6):362–372. doi:https://doi.org/10.1007/s40037-018-0481-2.
- Van Melle E , Gruppen L , Holmboe ES , Flynn L , Oandasan I , Frank JR. Using contribution analysis to evaluate competency-based medical education programs: it's all about rigor in thinking. Acad Med . 2017;92(6):752–758. doi:https://doi.org/10.1097/ACM.0000000000001479.
- Li ST , Klein MD , Balmer DF , Gusic ME . Scholarly evaluation of curricula and educational programs: using a systematic approach to produce publishable scholarship. Acad Pediatr . 2020;20(8):1083–1093. doi:https://doi.org/10.1016/j.acap.2020.07.005.
- Bates J , Ellaway RH. Mapping the dark matter of context: a conceptual scoping review. Med Educ . 2016;50(8):807–816. doi:https://doi.org/10.1111/medu.13034.
- Kotter JP. Leading change: why transformation efforts fail. HBR. May–June, 1995:59–67.
- Heifetz RA , Laurie DL. The work of leadership. HBR. January–February, 1997:124–134.
- Dauphinee WD , Boulet JR , Norcini JJ. Considerations that will determine if competency-based assessment is a sustainable innovation. Adv Health Sci Educ Theory Pract . 2019;24(2):413–421. doi:https://doi.org/10.1007/s10459-018-9833-2.
- Dath D , Iobst W. The importance of faculty development in the transition to competency-based medical education. Med Teach . 2010;32(8):683–686. doi:https://doi.org/10.3109/0142159X.2010.500710.
- Holmboe ES , Ward DS , Reznick RK , et al. Faculty development in assessment: the missing link in competency-based medical education. Acad Med . 2011;86(4):460–467. doi:https://doi.org/10.1097/ACM.0b013e31820cb2a7.
- Lockyer J , Carraccio C , Chan M-K , et al. Core principles of assessment in competency-based medical education. Med Teach . 2017;39(6):609–616. doi:https://doi.org/10.1080/0142159X.2017.1315082.
- Stefan A , Hall JN , Sherbino J , Chan TM. Faculty development in the age of competency-based medical education: a needs assessment of Canadian emergency medicine faculty and senior trainees. Can J Emergency Med . 2019;21(4):527–534. doi:https://doi.org/10.1017/cem.2019.343.
- Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ . 2005;39(12):1176–1177. doi:https://doi.org/10.1111/j.1365-2929.2005.02341.x.
- Englander R , Flynn T , Call S , et al. Toward defining the foundation of the MD degree: core entrustable professional activities for entering residency. Acad Med . 2016;91(10):1352–1358. doi:https://doi.org/10.1097/ACM.0000000000001204.
- Favreau MA , Tewksbury L , Lupi C , Cutrer WB , Jokela J , Yarris LM. Constructing a shared mental model for faculty development for the core entrustable professional activities for entering residency. Acad Med . 2017;92(6):759–764. doi:https://doi.org/10.1097/ACM.0000000000001511.
- Lupi C , Ownby A , Jokela J , et al. Faculty development revisited: a systems-based view of stakeholder development to meet the demands of entrustable professional activity implementation. Acad Med . 2018;93(10):1472–1479. doi:https://doi.org/10.1097/ACM.0000000000002297.
- Shorey S , Lau TC , Lau ST , Ang E. Entrustable professional activities in health care education: a scoping review. Med Educ . 2019;53(8):766–777. doi:https://doi.org/10.1111/medu.13879.
- Bertram RM , Blase KA , Fixsen DL. Improving programs and outcomes: implementation frameworks and organization change. Res Soc Work Pract . 2015;25(4):477–487. doi:https://doi.org/10.1177/1049731514537687.
- Baldwin CD , Chandran L , Gusic ME. Building sustainable professional development programs: Applying strategies from implementation science to translate evidence into practice. J Contin Educ Health Prof . 2017;37(2):106–115. doi:https://doi.org/10.1097/CEH.0000000000000151.
- Kogan JR , Conforti LN , Bernabeo E , Iobst W , Holmboe E. How faculty members experience workplace-based assessment rater training: a qualitative study. Med Educ . 2015;49(7):692–708. doi:https://doi.org/10.1111/medu.12733.
- Kogan JR , Hatala R , Hauer KE , Holmboe E . Guidelines: the do's, don'ts and don't knows of direct observation of clinical skills in medical education. Perspect Med Educ . 2017;6(5):286–305. doi:https://doi.org/10.1007/s40037-017-0376-7.
- Steinert Y , Mann K , Anderson B , et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: a 10-year update: BEME Guide No. 40. Med Teach . 2016;38(8):769–786. doi:https://doi.org/10.1080/0142159X.2016.1181851.
- Ten Cate O , Chen HC , Hoff RG , Peters H , Bok H , van der Schaaf M. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE guide no. 99. Med Teach . 2015;37(11):983–1002. doi:https://doi.org/10.3109/0142159X.2015.1060308.
- Telio S , Ajjawi R , Regehr G . The “educational alliance” as a framework for reconceptualizing feedback in medical education. Acad Med . 2015;90(5):609–614. doi:https://doi.org/10.1097/ACM.0000000000000560.
- Lockyer J , Armson H , Könings KD , et al. In-the-moment feedback and coaching: Improving R2C2 for a new context. J Grad Med Educ . 2020;12(1):27–35. doi:https://doi.org/10.4300/JGME-D-19-00508.1.
- Ginsburg S , van der Vleuten CPM , Eva KW. The hidden value of narrative comments for assessment: A quantitative reliability analysis of qualitative data. Acad Med . 2017;92(11):1617–1621. doi:https://doi.org/10.1097/ACM.0000000000001669.
- Hatala R , Sawatsky AP , Dudek N , Ginsburg S , Cook DA. Using in-training evaluation report (ITER) qualitative comments to assess medical students and residents: a systematic review. Acad Med . 2017;92(6):868–879. doi:https://doi.org/10.1097/ACM.0000000000001506.
- Chen HC , van den Broek WS , ten Cate O. The case for use of entrustable professional activities in undergraduate medical education. Acad Med . 2015;90(4):431–436. doi:https://doi.org/10.1097/ACM.0000000000000586.
- Ten Cate O , Hart D , Ankel F , et al. Entrustment decision making in clinical training. Acad Med . 2016;91(2):191–198. doi:https://doi.org/10.1097/ACM.0000000000001044.
- Holzhausen Y , Maaz A , Cianciolo AT , ten Cate O , Peters H . Applying occupational and organizational psychology theory to entrustment decision-making about trainees in health care: a conceptual model. Perspect Med Educ . 2017;6(2):119–126. doi:https://doi.org/10.1007/s40037-017-0336-2.
- Tekian A , Ten Cate O , Holmboe E , Roberts T , Norcini J. Entrustment decisions: Implications for curriculum development and assessment. Med Teach . 2020;42(6):698–704. doi:https://doi.org/10.1080/0142159X.2020.1733506.
- Steinert Y. Faculty development: from workshops to communities of practice. Med Teach . 2010;32(5):425–428. doi:https://doi.org/10.3109/01421591003677897.
- Hauer KE , Holmboe ES , Kogan JR . Twelve tips for implementing tools for direct observation of medical trainees' clinical skills during patient encounters. Med Teach . 2011;33(1):27–33. doi:https://doi.org/10.3109/0142159X.2010.507710.
- Boillat M , Bethune C , Ohle E , Razack S , Steinert Y. Twelve tips for using the objective structured teaching exercise for faculty development. Med Teach . 2012;34(4):269–273. doi:https://doi.org/10.3109/0142159X.2011.599891.
- Henry D , West DC. The clinical learning environment and workplace-based assessment: frameworks, strategies, and implementation. Pediatr Clin North Am . 2019;66(4):839–854. doi:https://doi.org/10.1016/j.pcl.2019.03.010.
- Pereira AG , Woods M , Olson APJ , van den Hoogenhof S , Duffy BL , Englander R. Criterion-based assessment in a norm-based world: how can we move past grades? Acad Med . 2018;93(4):560–564. doi:https://doi.org/10.1097/ACM.0000000000001939.
- Ten Cate O , Schwartz A , Chen HC , et al. Assessing trainees and making entrustment decisions: On the nature and use of entrustment-supervision scales. Acad Med . 2020;95(11):1662–1669. doi:https://doi.org/10.1097/ACM.0000000000003427.
- de Jonge L , Timmerman AA , Govaerts MJB , et al. Stakeholder perspectives on workplace-based performance assessment: towards a better understanding of assessor behaviour. Adv Health Sci Educ Theory Pract . 2017;22(5):1213–1243. doi:https://doi.org/10.1007/s10459-017-9760-7.
- Sargeant J , Lockyer JM , Mann K , et al. The R2C2 model in residency education: How does it foster coaching and promote feedback use? Acad Med . 2018;93(7):1055–1063. doi:https://doi.org/10.1097/ACM.0000000000002131.
- Peters H , Holzhausen Y , Boscardin C , ten Cate O , Chen HC. Twelve tips for the implementation of EPAs for assessment and entrustment decisions. Med Teach . 2017;39(8):802–807. doi:https://doi.org/10.1080/0142159X.2017.1331031.
- Sargeant J , Lockyer J , Mann K , et al . Facilitated reflective performance feedback: developing an Evidence- and Theory-Based Model That Builds Relationship, Explores Reactions and Content, and Coaches for Performance Change (R2C2)). Acad Med . 2015;90(12):1698–1706. doi:https://doi.org/10.1097/ACM.0000000000000809.
- Sargeant J , Armson H , Chesluk B , et al. The processes and dimensions of informed self-assessment: a conceptual model. Acad Med . 2010;85(7):1212–1220. doi:https://doi.org/10.1097/ACM.0b013e3181d85a4e.
- Lockspeiser TM , Schmitter PA , Lane JL , Hanson JL , Rosenberg AAA. Validated rubric for scoring learning goals. MedEdPORTAL . 2013. https://doi.org/10.15766/mep_2374-8265.9369
- Lefroy J , Watling C , Teunissen PW , Brand P . Guidelines: the do's, don'ts and don't knows of feedback for clinical education. Perspect Med Educ . 2015;4(6):284–299. doi:https://doi.org/10.1007/s40037-015-0231-7.
- Sebok-Syer SS , Klinger DA , Sherbino J , Chan TM . Mixed messages or miscommunication? Investigating the relationship between Assessors' Workplace-Based Assessment Scores and Written Comments. Acad Med . 2017;92(12):1774–1779. doi:https://doi.org/10.1097/ACM.0000000000001743.
- Oller KL , Mai CT , Ledford RJ , O’Brien KE. Faculty development for the evaluation system: a dual agenda. Adv Med Educ Pract . 2017;8:205–210. doi:https://doi.org/10.2147/AMEP.S124004.
- Burm S , Sebok-Syer S , Lingard L , et al . “You want me to assess what?”: faculty perceptions of assessing residents from outside their specialty?. Acad Med . 2019;94(10):1478–1482. doi:https://doi.org/10.1097/ACM.0000000000002771.
- O'Sullivan PS , Irby DM. Reframing research on faculty development. Acad Med . 2011;86(4):421–428. doi:https://doi.org/10.1097/ACM.0b013e31820dc058.
- Li LC , Grimshaw JM , Nielsen C , Judd M , Coyte PC , Graham ID. Evolution of Wenger's concept of community of practice. Implement Sci . 2009;4(11). doi:https://doi.org/10.1186/1748-5908-4-11.
- Govaerts MJB , Van de Wiel MWJ , Schuwirth LWT , Van der Vleuten CPM , Muijtjens AMM . Workplace-based assessment: raters' performance theories and constructs. Adv Health Sci Educ Theory Pract . 2013;18(3):375–396. doi:https://doi.org/10.1007/s10459-012-9376-x.
- Yeates P , O'Neill P , Mann K , Eva K . Seeing the same thing differently: mechanisms that contribute to assessor differences in directly-observed performance assessments. Adv Health Sci Educ Theory Pract . 2013;18(3):325–341. doi:https://doi.org/10.1007/s10459-012-9372-1.
- Siegel S. Nonparametric Statistics for the Behavioral Sciences . New York, NY: McGraw-Hill; 1956.
- Tabachnick BG , Fidell LS. Using Multivariate Statistics . Boston, MA: Pearson/Allyn & Bacon. 2007.
- Ellaway RH , Pusic M , Yavner S , Kalet AL. Context matters: emergent variability in an effectiveness trial of online teaching modules. Med Educ . 2014;48(4):386–396. doi:https://doi.org/10.1111/medu.12389.
- Dijkstra J , Galbraith R , Hodges BD , et al. Expert validation of fit-for-purpose guidelines for designing programmes of assessment. BMC Med Educ . 2012;12:20. doi:https://doi.org/10.1186/1472-6920-12-20.
- Norcini J , Anderson MB , Bollela V , et al . 2018 consensus framework for good assessment. Med Teach . 2018;40(11):1102–1109. doi:https://doi.org/10.1080/0142159X.2018.1500016.
- Keeley MG , Gusic ME , Morgan HK , Aagaard EM , Santen SA. Moving toward summative competency assessment to individualize the postclerkship phase. Acad Med . 2019;94(12):1858–1864. doi:https://doi.org/10.1097/ACM.0000000000002830.
- Vaitsis C , Nilsson G , Zary N. Visual analytics in healthcare education: exploring novel ways to analyze and represent big data in undergraduate medical education. PeerJ . 2014;2:e683. doi:https://doi.org/10.7717/peerj.683.