Constructing Approaches to Entrustable Professional Activity Development that Deliver Valid Descriptions of Professional Practice

References

• Nasca TJ. The CEO's First Column - the next step in the outcomes-Based Accreditation Project. ACGME Bulletin. 2008:2–4.
   Google Scholar
• Association of American Medical Colleges. Core entrustable professional activities for entering residency. May 2014. 1–114.
   Google Scholar
• The Royal College of Physicians and Surgeons of Canada. Competence by Design. Royal College of Physicians and Surgeons of Canada. http://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e. Published 2014. Accessed May 19, 2020.
   Google Scholar
• Touchie C, Boucher A, Allen T, et al. Entrustable professional activities for the transition from medical school to residency. Ottawa; 2016:1–25. https://afmc.ca/sites/default/files/documents/AFMC_Entrustable_Professional_Activities_EN_0.pdf.
   Google Scholar
• Leung W-C. Competency based medical training: review. BMJ. 2002;325(7366):693–696.
   PubMed Web of Science ®Google Scholar
• Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review. Acad Med. 2009;84(3):301–309. doi:10.1097/ACM.0b013e3181971f08.
   PubMed Web of Science ®Google Scholar
• Malone K, Supri S. A critical time for medical education: the perils of competence-based reform of the curriculum. Adv Health Sci Educ Theory Pract. 2012;17(2):241–246. doi:10.1007/s10459-010-9247-2.
   PubMed Web of Science ®Google Scholar
• Witteles RM, Verghese A. Accreditation Council for Graduate Medical Education (ACGME) milestones-time for a revolt? JAMA Intern Med. 2016;176(11):1599–1600. doi:10.1001/jamainternmed.2016.5552.
   PubMed Web of Science ®Google Scholar
• Holmboe ES, Call S, Ficalora RD. Milestones and competency-based medical education in internal medicine. JAMA Intern Med. 2016;176(11):1601–1602. doi:10.1001/jamainternmed.2016.5556.
   PubMed Web of Science ®Google Scholar
• Li S-T. The promise of milestones: are they living up to our expectations? J Grad Med Educ. 2017;9(1):54–57. doi:10.4300/JGME-D-16-00694.1.
   PubMedGoogle Scholar
• Taylor DR, Park YS, Smith CA, Karpinski J, Coke W, Tekian A. Creating entrustable professional activities to assess internal medicine residents in training: a mixed-methods approach. Ann Intern Med. 2018;168(10):724–729. doi:10.7326/M17-1680.
   PubMed Web of Science ®Google Scholar
• Kogan JR, Holmboe E. Realizing the promise and importance of performance-based assessment. Teach Learn Med. 2013;25(Suppl 1):S68–S74. doi:10.1080/10401334.2013.842912.
   PubMed Web of Science ®Google Scholar
• ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542–547. doi:10.1097/ACM.0b013e31805559c7.
   PubMed Web of Science ®Google Scholar
• Tekian A. Are all EPAs really EPAs? Med Teach. 2017;39(3):232–233. doi:10.1080/0142159X.2016.1230665.
   PubMed Web of Science ®Google Scholar
• Meyer EG, Chen HC, Uijtdehaage S, Durning SJ, Maggio LA. Scoping review of entrustable professional activities in undergraduate medical education. Acad Med. 2019;94(7):1040–1049. doi:10.1097/ACM.0000000000002735.
   PubMed Web of Science ®Google Scholar
• Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. Med Educ. 2019;53(8):766–777. doi:10.1111/medu.13879.
   PubMed Web of Science ®Google Scholar
• O'Dowd E, Lydon S, O'Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. Med Educ. 2019;53 (3):234–249. doi:10.1111/medu.13792.
   PubMed Web of Science ®Google Scholar
• Young JQ, Hasser C, Hung EK, et al. Developing end-of-training entrustable professional activities for psychiatry: results and methodological lessons. Acad Med. 2018;93(7):1048–1054. doi:10.1097/ACM.0000000000002058.
   PubMed Web of Science ®Google Scholar
• ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37(11):983–1002. doi:10.3109/0142159X.2015.1060308.
   PubMed Web of Science ®Google Scholar
• ten Cate O. Trusting graduates to enter residency: what does it take? J Grad Med Educ. 2014;6(1):7–10. doi:10.4300/JGME-D-13-00436.1.
   PubMedGoogle Scholar
• ten Cate O, Graafmans L, Posthumus I, Welink L, van Dijk M. The EPA-based Utrecht undergraduate clinical curriculum: development and implementation. Med Teach. 2018;40(5):506–513. doi:10.1080/0142159X.2018.1435856.
   PubMed Web of Science ®Google Scholar
• Wagner JP, Lewis CE, Tillou A, et al. Use of entrustable professional activities in the assessment of surgical resident competency. JAMA Surg. 2018;153(4):335–343. doi:10.1001/jamasurg.2017.4547.
   PubMed Web of Science ®Google Scholar
• Baker E, Sackett P, Bond L, et al. Standards for Educational and Psychological Testing. 6 ed. (Baker E, Sackett P, eds.). Washington DC: American Educational Research Association, American Psychological Association, and National Council on Measurement in Education; 1999.
   Google Scholar
• Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ. 2003;37(9):830–837. doi:10.1046/j.1365-2923.2003.01594.x.
   PubMed Web of Science ®Google Scholar
• Steil AV, Garcia CE. Work analysis in organizations - definition, uses, and methods. Psicol Estud. 2016;21(3):473–483. doi:10.4025/psicolestud.v21i3.30201.
   Google Scholar
• Jones AP, Main DS, Butler MC, Johnson LA. Narrative job descriptions as potential sources of job analysis ratings. Personnel Psychology. 1982;35(4):813–828. doi:10.1111/j.1744-6570.1982.tb02224.x.
   Web of Science ®Google Scholar
• McCormick EJ, Jeanneret PR, Gael S. Position Analysis Questionnaire (Jones EI, ed.). San Diego, California: Nay Personnel Research and Development Center. 1988. p. 78–86.
   Google Scholar
• Morgeson FP, Brannick MT, Levine EL. Job and Work Analysis. Los Angeles: SAGE. 2019.
   Google Scholar
• Marceau M, Gallagher F, Young M, St Onge C. Validity as a social imperative for assessment in health professions education: a concept analysis. Med Educ. 2018;52(6):641–653. doi:10.1111/medu.13574.
   PubMed Web of Science ®Google Scholar
• Hauer KE, Kohlwes J, Cornett P, et al. Identifying entrustable professional activities in internal medicine training. J Grad Med Educ. 2013;5(1):54–59. doi:10.4300/JGME-D-12-00060.1.
   PubMedGoogle Scholar
• Caverzagie KJ, Cooney TG, Hemmer PA, Berkowitz L. The development of entrustable professional activities for internal medicine residency training: a report from the Education Redesign Committee of the Alliance for Academic Internal Medicine. Acad Med. 2015;90(4):479–484. doi:10.1097/ACM.0000000000000564.
   PubMed Web of Science ®Google Scholar
• Chen HC, McNamara M, Teherani A, Cate OT, O’Sullivan P. Developing entrustable professional activities for entry into clerkship. Acad Med. 2016;91(2):247–255. doi:10.1097/ACM.0000000000000988.
   PubMed Web of Science ®Google Scholar
• Carraccio C, Englander R, Gilhooly J, et al. Building a framework of entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med. 2017;92(3):324–330. doi:10.1097/ACM.0000000000001141.
   PubMed Web of Science ®Google Scholar
• Messick S. Validity. In: Linn RL, ed. Educational Measurement. Washington DC: American Council on Education and National Council on Measurement in Education;1989:11–113.
   Google Scholar
• Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i–iv–1–88.
   Google Scholar
• Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74(9):979–983. doi:10.2105/ajph.74.9.979.
   PubMed Web of Science ®Google Scholar
• Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med Teach. 2017;39(1):14–19. doi:10.1080/0142159X.2017.1245856.
   PubMed Web of Science ®Google Scholar
• Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–380. doi:10.1136/bmj.311.7001.376.
   PubMed Web of Science ®Google Scholar
• Englander R, Flynn T, Call S, et al. Toward defining the foundation of the MD Degree: core entrustable professional activities for entering residency. Acad Med. 2016;91(10):1352–1358. doi:10.1097/ACM.0000000000001204.
   PubMed Web of Science ®Google Scholar
• Goodman CM. The Delphi technique: a critique. J Adv Nurs. 1987;12(6):729–734. doi:10.1111/j.1365-2648.1987.tb01376.x.
   PubMed Web of Science ®Google Scholar
• Green B, Jones M, Hughes D, Williams A. Applying the Delphi technique in a study of GPs' information requirements. Health Soc Care Community. 1999;7(3):198–205. doi:10.1046/j.1365-2524.1999.00176.x.
   PubMed Web of Science ®Google Scholar
• Jhangiani R, Tarry H. Working groups: performance and decision making. In: Stangor C, Jhangiani R, Tarry H, eds. Principles of Social Psychology: 1st International Edition. Victoria, BC: BCcampus; 2017.
   Google Scholar
• Downing SM, Yudkowsky R. Assessment in Health Professions Education. First. (Downing SM, Yudkowsky R, eds.). New York: Taylor and Francis; 2009.
   Google Scholar
• Post JA, Wittich CM, Thomas KG, et al. Rating the quality of entrustable professional activities: content validation and associations with the clinical context. J Gen Intern Med. 2016;31(5):518–523. doi:10.1007/s11606-016-3611-8.
   PubMed Web of Science ®Google Scholar
• Taylor DR, Park YS, Egan R, et al. EQual, a novel rubric to evaluate entrustable professional activities for quality and structure. Acad Med. 2017;92 (11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):S110–S117. doi:10.1097/ACM.0000000000001908.
   PubMedGoogle Scholar
• Waggoner J, Carline JD, Durning SJ. Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad Med. 2016;91(5):663–668. doi:10.1097/ACM.0000000000001092.
   PubMed Web of Science ®Google Scholar
• Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008–1015.
   PubMed Web of Science ®Google Scholar
• Raymond MR, Mee J, King A, Haist SA, Winward ML. What new residents do during their initial months of training. Acad Med. 2011;86(10 Suppl):S59–S62. doi:10.1097/ACM.0b013e31822a70ff.
   PubMed Web of Science ®Google Scholar
• Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, Gravel J. Entrustable professional activities in family medicine. J Grad Med Educ. 2013;5(1):112–118. doi:10.4300/JGME-D-12-00034.1.
   PubMedGoogle Scholar
• Hinsz VB. Cognitive and consensus processes in group recognition memory performance. - PsycNET. J Pers Soc Psychol. 1990;59(4):705–718. doi:10.1037/0022-3514.59.4.705.
   Web of Science ®Google Scholar
• Wittenbaum GM. Information sampling in decision-making groups: the impact of members’ task-relevant status. Small Group Research. 1998;29(1):57–84. doi:10.1177/1046496498291003.
   Web of Science ®Google Scholar
• Halcomb E, Davidson P, Hardaker L. Using the consensus development conference method in healthcare research. Nurse Res. 2008;16(1):56–71. doi:10.7748/nr2008.10.16.1.56.c6753.
   PubMedGoogle Scholar
• Delbecq AL, Van de Ven AH. A group process model for problem identification and program planning. The Journal of Applied Behavioral Science. 1971;7(4):466–492. doi:10.1177/002188637100700404.
   Web of Science ®Google Scholar
• Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract. 2012;18(2):188–194. doi:10.1111/j.1440-172X.2012.02017.x.
   PubMed Web of Science ®Google Scholar
• Touchie C, De Champlain A, Pugh D, Downing S, Bordage G. Supervising incoming first-year residents: faculty expectations versus residents’ experiences. Med Educ. 2014;48(9):921–929. doi:10.1111/medu.12503.
   PubMed Web of Science ®Google Scholar
• Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41(4):376–382. doi:10.1046/j.1365-2648.2003.02537.x.
   PubMed Web of Science ®Google Scholar
• Keeney S, Hasson F, McKenna H. The Delphi Technique in Nursing and Health Research. 1st ed. Oxford, UK: Wiley-Blackwell; 2011. doi:10.1002/9781444392029.
   Google Scholar
• Meyer EG, Taylor DR, Uijtdehaage S, Durning SJ. EQual rubric evaluation of the Association of American Medical Colleges’ core entrustable professional activities for entering residency. Acad Med. May 2020; doi:10.1097/ACM.0000000000003504.
   PubMed Web of Science ®Google Scholar
• Hart D, Franzen D, Beeson M, et al. Integration of entrustable professional activities with the milestones for emergency medicine residents. West J Emerg Med. 2019;20(1):35–42. doi:10.5811/westjem.2018.11.38912.
   PubMed Web of Science ®Google Scholar