Becoming a deliberately developmental organization: Using competency based assessment data for organizational development

References

• Ames SE, Ponce BA, Marsh JL, Hamstra SJ. 2020. Orthopaedic surgery residency milestones: initial formulation and future directions. J Am Acad Orthop Surg. 28(1):e1–e8.
   PubMed Web of Science ®Google Scholar
• Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. 2009. Evaluating obstetrical residency programs using patient outcomes. JAMA. 302(12):1277–1283.
   PubMed Web of Science ®Google Scholar
• Asch DA, Nicholson S, Srinivas SK, Herrin J, Epstein AJ. 2014. How do you deliver a good obstetrician? Outcome-based evaluation of medical education. Acad Med. 89(1):24–26.
   PubMed Web of Science ®Google Scholar
• Bansal N, Simmons KD, Epstein AJ, Morris JB, Kelz RR. 2016. Using patient outcomes to evaluate general surgery residency program performance. JAMA Surg. 151(2):111–119.
   PubMed Web of Science ®Google Scholar
• Beeson MS, Hamstra SJ, Barton MA, Yamazaki K, Counselman FL, Shayne PH, Holmboe ES, Muelleman RL, Reisdorff EJ. 2017. Straight line scoring by clinical competency committees using emergency medicine milestones. J Grad Med Educ. 9(6):716–720.
   PubMedGoogle Scholar
• Bloomberg J. 2020. Digitization, digitalization, and digital transformation: confuse them at your peril. Forbes; [accessed 2020 Mar 5]. https://www.forbes.com/sites/jasonbloomberg/2018/04/29/digitization-digitalization-and-digital-transformation-confuse-them-at-your-peril/#56350c972f2c.
   Google Scholar
• Bourgeois JA, Hategan A, Azzam A. 2015. Competency-based medical education and scholarship: creating an active academic culture during residency. Perspect Med Educ. 4(5):254–258.
   PubMed Web of Science ®Google Scholar
• Brennen S, Kreiss D. 2014. Digitalization and digitization. Culture Digitally; [accessed 2020 Mar 4]. http://culturedigitally.org/2014/09/digitalization-and-digitization/.
   Google Scholar
• Brennen JS, Kreiss D. 2016. Digitalization. In: International encyclopedia of communication theory and philosophy. [place unknown]: Wiley; [accessed 2020 Mar 4]; p. 1–11. http://onlinelibrary.wiley.com/doi/abs/10.1002/9781118766804.wbiect111.
   Google Scholar
• Carey R, Wilson G, Bandi V, Mondal D, Martin L, Woods R, Teresa C, Thoma B. 2020. Developing a dashboard to meet the needs of residents in a competency-based training program: a design-based research project. Can Med Educ J. 11(6):e21..
   PubMedGoogle Scholar
• Chan T, Sebok-Syer S, Thoma B, Wise A, Sherbino J, Pusic M. 2018. Learning analytics in medical education assessment: the past, the present, and the future. AEM Educ Train. 2(2):178–187.
   PubMedGoogle Scholar
• Chan TM, Luckett-Gatopoulos S, Thoma B. 2015. Commentary on competency-based medical education and scholarship: creating an active academic culture during residency. Perspect Med Educ. 4(5):214–217.
   PubMed Web of Science ®Google Scholar
• Chan TM, Paterson QS, Hall AK, Zaver F, Woods RA, Hamstra SJ, Stefan A, Ting DK, Thoma B. 2020. Outcomes in the age of competency-based medical education: recommendations for emergency medicine training in Canada from the 2019 symposium of academic emergency physicians. CJEM. 22(2):204–214.
   Google Scholar
• Chen C, Petterson S, Phillips R, Bazemore A, Mullan F. 2014. Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for Medicare beneficiaries. JAMA. 312(22):2385–2393.
   PubMed Web of Science ®Google Scholar
• Chickering AW. 2003. Reclaiming our soul: democracy and higher education. Change Mag High Learn. 35(1):38–44.
   Google Scholar
• Conforti LN, Yaghmour NA, Hamstra SJ, Holmboe ES, Kennedy B, Liu JJ, Waldo H, Selden NR. 2018. The effect and use of milestones in the assessment of neurological surgery residents and residency programs. J Surg Educ. 75(1):147–155.
   PubMed Web of Science ®Google Scholar
• Cooke LJ, Duncan D, Rivera L, Dowling SK, Symonds C, Armson H. 2018a. How do physicians behave when they participate in audit and feedback activities in a group with their peers? Implement Sci. 13(1):104.
   PubMed Web of Science ®Google Scholar
• Cooke LJ, Duncan D, Rivera L, Dowling SK, Symonds C, Armson H. 2018b. The Calgary Audit and Feedback Framework: a practical, evidence-informed approach for the design and implementation of socially constructed learning interventions using audit and group feedback. Implement Sci. 13(1):136.
   PubMed Web of Science ®Google Scholar
• Dagnone D, Stockley D, Flynn L, Egan R, van Wylick R, McEwan L, Walker R, Reznick R. 2019. Delivering on the promise of competency based medical education – an institutional approach. Can Med Educ J. 10(1):e28–e38.
   PubMedGoogle Scholar
• Dauphinee WD, Boulet JR, Norcini JJ. 2019. Considerations that will determine if competency-based assessment is a sustainable innovation. Adv Health Sci Educ. 24(2):413–421.
   PubMed Web of Science ®Google Scholar
• Deming WE. 2018. The new economics for industry, government, education. [place unknown]: MIT Press.
   Google Scholar
• Dudek NL, Marks MB, Regehr G. 2005. Failure to fail: the perspectives of clinical supervisors. Acad Med. 80(Suppl):S84–S87.
   PubMed Web of Science ®Google Scholar
• Dweck CS. 2008. Mindset: the new psychology of success. New York: Ballantine Books.
   Google Scholar
• Edgar L, Roberts S, Holmboe E. 2018. Milestones 2.0: a step forward. J Grad Med Educ. 10(3):367–369.
   PubMedGoogle Scholar
• Edmondson AC, Higgins M, Singer S, Weiner J. 2016. Understanding psychological safety in health care and education organizations: a comparative perspective. Res Hum Dev. 13(1):65–68.
   Web of Science ®Google Scholar
• Eva KW, Cunnington JPW, Reiter HI, Keane DR, Norman GR. 2004. How can I know what I don’t know? Poor self assessment in a well-defined domain. Adv Health Sci Educ Theory Pract. 9(3):211–224.
   PubMed Web of Science ®Google Scholar
• Eva KW, Regehr G. 2005. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 80(Suppl):S46–S54.
   PubMed Web of Science ®Google Scholar
• Eva KW, Regehr G. 2008. “I’ll never play professional football” and other fallacies of self-assessment. J Contin Educ Health Prof. 28(1):14–19.
   PubMed Web of Science ®Google Scholar
• Ferguson PC, Caverzagie KJ, Nousiainen MT, Snell L. 2017. Changing the culture of medical training: an important step toward the implementation of competency-based medical education. Med Teach. 39(6):599–602.
   PubMed Web of Science ®Google Scholar
• Govaerts M, van der Vleuten CP. 2013. Validity in work-based assessment: expanding our horizons. Med Educ. 47(12):1164–1174.
   PubMed Web of Science ®Google Scholar
• Guerrasio J, Furfari KA, Rosenthal LD, Nogar CL, Wray KW, Aagaard EM. 2014. Failure to fail: the institutional perspective. Med Teach. 36(9):799–803.
   PubMed Web of Science ®Google Scholar
• Hall AK, Rich J, Dagnone DJ, Weersink K, Caudle J, Sherbino J, Frank JRJ, Bandiera G, Vaan Melle E, Dagnone JD, et al. 2020. It’s a marathon, not a sprint: rapid evaluation of CBME program implementation. Acad Med. 95(5):786–793.
   PubMed Web of Science ®Google Scholar
• Hall AK, Schumacher DJ, Thoma B, Caretta-Weyer H, Gruppen L, Kinnear B, Cooke LJ, Frank JR, Van Melle E. 2021. Outcomes of competency-based medical education: a taxonomy for shared language. Med Teach. (In press.)
   Google Scholar
• Hamstra SJ, Yamazaki K, Barton MA, Santen SA, Beeson MS, Holmboe ES. 2019. A national study of longitudinal consistency in ACGME milestone ratings by clinical competency committees: exploring an aspect of validity in the assessment of residents’ competence. Acad Med. 94(10):1522–1531.
   PubMed Web of Science ®Google Scholar
• Hauer KE, Cate OT, Holmboe E, Boscardin C, Iobst W, Chesluk B, Baron RB, O’Sullivan PS. 2016. Ensuring resident competence: a narrative review of the literature on group decision-making to inform the work of clinical competency committees. J Grad Med Educ. 8(2):156–164.
   PubMedGoogle Scholar
• Hauer KE, Chesluk B, Iobst W, Holmboe E, Baron RB, Boscardin CK, Ten Cate O, O’Sullivan PS. 2015. Reviewing residents’ competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. 90(8):1084–1092.
   PubMed Web of Science ®Google Scholar
• Holmboe ES, Sherbino J, Englander R, Snell L, Frank JR. 2017. A call to action: the controversy of and rationale for competency-based medical education. Med Teach. 39(6):574–581.
   PubMed Web of Science ®Google Scholar
• Holmboe ES, Yamazaki K, Nasca TJ, Hamstra SJ. 2020. Using longitudinal milestones data and learning analytics to facilitate the professional development of residents: early lessons from three specialties. Acad Med. 95(1):97–103.
   PubMed Web of Science ®Google Scholar
• Kegan R, Lahey LL. 2016. An everyone culture: becoming a deliberately developmental organization. Boston: Harvard Business Review Press.
   Google Scholar
• Kopp M, Gröblinger O, Adams S. 2019. Five common assumptions that prevent digital transformation at higher education institutions. In: Proceedings of the 13th International Technology, Education and Development Conference; Mar. 11–13; Valencia, Spain; [accessed 2020 Mar 5]. p. 1448–1457. http://library.iated.org/view/KOPP2019FIV.
   Google Scholar
• Kruger J, Dunning D. 1999. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 77(6):1121–1134.
   PubMed Web of Science ®Google Scholar
• Larocque S, Luhanga FL. 2013. Exploring the issue of failure to fail in a nursing program. Int J Nurs Educ Scholarsh. 10(1):10:/j/ijnes. https://www.degruyter.com/view/j/ijnes.2013.10.issue-1/ijnes-2012-0037/ijnes-2012-0037.xml.
   Google Scholar
• Lockyer J, Carraccio C, Chan MK, Hart D, Smee S, Touchie C, Holmboe ES, Frank JR. 2017. Core principles of assessment in competency-based medical education. Med Teach. 39(6):609–616.
   PubMed Web of Science ®Google Scholar
• Lurie SJ, Mooney CJ, Lyness JM. 2011. Commentary: pitfalls in assessment of competency-based educational objectives. Acad Med. 86(4):412–414.
   PubMed Web of Science ®Google Scholar
• Mann K, van der Vleuten C, Eva K, Armson H, Chesluk B, Dornan T, Holmboe E, Lockyer J, Loney E, Sargeant J. 2011. Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med. 86(9):1120–1127.
   PubMed Web of Science ®Google Scholar
• McGaghie WC, Miller GE, Sajid AW, Telder TV. 1978. Competency-based curriculum development on medical education: an introduction. Public Health Pap. 68:11–91.
   PubMedGoogle Scholar
• Norman G, Norcini J, Bordage G. 2014. Competency-based education: milestones or millstones. J Grad Med Educ. 6(1):1–6.
   PubMedGoogle Scholar
• Nousiainen MT, Mironova P, Hynes M, Glover Takahashi S, Reznick R, Kraemer W, Alman B. 2018. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med Teach. 40(10):1042–1054.
   PubMed Web of Science ®Google Scholar
• Regehr G, Eva K. 2006. Self-assessment, self-direction, and the self-regulating professional. Clin Orthop Relat Res. 449:34–38.
   PubMed Web of Science ®Google Scholar
• Ricotta DN, Huang GC, Hale AJ, Freed JA, Smith CC. 2019. Mindset theory in medical education. Clin Teach. 16(2):159–161.
   PubMed Web of Science ®Google Scholar
• Schumacher DJ, Holmboe E, Carraccio C, Martini A, van der Vleuten C, Busari J, Sobolewski B, Byczkowski TL. 2020. Resident-sensitive quality measures in the pediatric emergency department: exploring relationships with supervisor entrustment and patient acuity and complexity. Acad Med. 95(8):1256–1264.
   PubMed Web of Science ®Google Scholar
• Schumacher DJ, Holmboe ES, van der Vleuten C, Busari JO, Carraccio C. 2018. Developing resident-sensitive quality measures: a model from pediatric emergency medicine. Acad Med. 93(7):1071–1078.
   PubMed Web of Science ®Google Scholar
• Schumacher DJ, Martini A, Holmboe E, Carraccio C, van der Vleuten C, Sobolewski B, Busari J, Byczkowski TL. 2020. Initial implementation of resident-sensitive quality measures in the pediatric emergency department: a wide range of performance. Acad Med. 95(8):1248–1255.
   PubMed Web of Science ®Google Scholar
• Sebok-Syer SS, Chahine S, Watling CJ, Goldszmidt M, Cristancho S, Lingard L. 2018. Considering the interdependence of clinical performance: implications for assessment and entrustment. Med Educ. 52(9):970–980.
   PubMed Web of Science ®Google Scholar
• Sebok-Syer SS, Pack R, Shepherd L, McConnell A, Dukelow AM, Sedran R, Lingard L. 2020. Elucidating system-level interdependence in electronic health record data: What are the ramifications for trainee assessment? Med Educ. 54(8):738–747. http://onlinelibrary.wiley.com/doi/abs/10.1111/medu.14147.
   PubMed Web of Science ®Google Scholar
• Smirnova A, Sebok-Syer SS, Chahine S, Kalet AL, Tamblyn R, Lombarts K, van der Vleuten CPM, Schumacher DJ. 2019. Defining and adopting clinical performance measures in graduate medical education: Where are we now and where are we going? Acad Med. 94(5):671–677.
   PubMed Web of Science ®Google Scholar
• Thoma B, Bandi V, Carey R, Mondal D, Woods R, Martin L, Chan T. 2020. Developing a dashboard to meet Competence Committee needs: a design-based research project. Can J Med Educ. 11(1):e16–e34.
   PubMedGoogle Scholar
• Thoma B, Warm E, Hamstra SJ, Cavalcanti R, Pusic M, Shaw T, Verma A, Frank JR, Hauer KE. 2020. Next steps in the implementation of learning analytics in medical education: consensus from an international cohort of medical educators. J Grad Med Educ. 12(3):303–311.
   PubMedGoogle Scholar
• Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. 2019. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 94(7):1002–1009.
   PubMed Web of Science ®Google Scholar
• Verina N, Titko J. 2019. Digital transformation: conceptual framework. In: Proceedings of the 6th International Scientific Conference on Contemporary Issues in Business, Management and Economics Engineering; May 9–10; Vilnius, Lithuania. Vilnius (Lithuania): Vilnius Gediminas Technical University; [accessed 2020 Mar 5]. http://cibmee.vgtu.lt/index.php/verslas/2019/paper/view/19.
   Google Scholar
• Watkins KE. 2005. What would be different if higher educational institutions were learning organizations? Adv Dev Hum Resour. 7(3):414–421.
   Google Scholar
• Whitehead CR, Kuper A. 2017. Faith-based medical education. Adv Health Sci Educ Theory Pract. 22(1):1–3.
   PubMed Web of Science ®Google Scholar
• Witteles RM, Verghese A. 2016. Accreditation Council for Graduate Medical Education (ACGME) milestones – time for a revolt? JAMA Intern Med. 176(11):1599–1600.
   PubMed Web of Science ®Google Scholar
• Yepes-Rios M, Dudek N, Duboyce R, Curtis J, Allard RJ, Varpio L. 2016. The failure to fail underperforming trainees in health professions education: a BEME systematic review. BEME Guide No. 42. Med Teach. 38(11):1092–1099.
   PubMed Web of Science ®Google Scholar