References
- Albanese MA, Mejicano G, Anderson WM, Gruppen L. Building a competency-based curriculum: the agony and the ecstasy. Adv Health Sci Educ Theory Pract. 2010;15(3):439–454. doi:https://doi.org/10.1007/s10459-008-9118-2.
- Frank JR, Snell L, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638–645. doi:https://doi.org/10.3109/0142159X.2010.501190.
- Pusic MV, Kessler D, Szyld D, Kalet A, Pecaric M, Boutis K. Experience curves as an organizing framework for deliberate practice in emergency medicine learning. Acad Emerg Med. 2012;19(12):1476–1480. doi:https://doi.org/10.1111/acem.12043.
- Carraccio C, Englander R, Van Melle E, et al. Advancing competency-based medical education: a charter for clinician-educators. Acad Med. 2016;91(5):645–649. doi:https://doi.org/10.1097/ACM.0000000000001048.
- Holmboe ES, Sherbino J, Englander R, Snell L, Frank JR. A call to action: the controversy of and rationale for competency-based medical education. Med Teach. 2017;39(6):574–581. doi:https://doi.org/10.1080/0142159X.2017.1315067.
- Carraccio C, Englander R, Gilhooly J, et al. Building a framework of entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med. 2017;92(3):324–330. doi:https://doi.org/10.1097/ACM.0000000000001141.
- Nousiainen MT, Caverzagie KJ, Ferguson PC, Frank JR. Implementing competency-based medical education: what changes in curricular structure and processes are needed?Med Teach. 2017;39(6):594–598. doi:https://doi.org/10.1080/0142159X.2017.1315077.
- Lineberry M, Park YS, Cook DA, Yudkowsky R. Making the case for mastery learning assessments: key issues in validation and justification. Acad Med. 2015;90(11):1445–1450. doi:https://doi.org/10.1097/ACM.0000000000000860.
- Grant J. The incapacitating effects of competence: a critique. Adv Health Sci Educ Theory Pract. 1999;4(3):271–277. doi:https://doi.org/10.1023/A:1009845202352.
- Talbot M. Monkey see, monkey do: a critique of the competency model in graduate medical education. Med Educ. 2004;38(6):587–592. doi:https://doi.org/10.1046/j.1365-2923.2004.01794.x.
- Castro D, Yang J, Greer ML, et al. Competency based medical education—towards the development of a standardized pediatric radiology testing module. Acad Radiol. 2020;27(11):1622–1632. doi:https://doi.org/10.1016/j.acra.2019.12.005.
- Deitte LA, Gordon LL, Zimmerman BD, et al. Entrustable professional activities: ten things radiologists do. Acad Radiol. 2016;23(3):374–381. doi:https://doi.org/10.1016/j.acra.2015.11.010.
- Royal College of Physicians and Surgeons. Final in-Training Evalaution Report (FITER). http://www.royalcollege.ca/rcsite/documents/ibd/diagnostic_radiology_fiter_e.pdf. Accessed February 3, 2020.
- van Deven T, Wang S, Hibbert KM. Radiology Education: The Evaluation and Assessment of Clinical Competence. New York, NY: Springer; 2012:215.
- Chetlen AL, Mendiratta-Lala M, Probyn L, et al. Conventional medical education and the history of simulation in radiology. Acad Radiol. 2015;22(10):1252–1267. doi:https://doi.org/10.1016/j.acra.2015.07.003.
- Boutis K, Pecaric M, Pusic M. Selecting radiographs for teaching the interpretation of X-rays based on ratings by the target population. Med Educ. 2009;43(5):434–441. doi:https://doi.org/10.1111/j.1365-2923.2009.03311.x.
- Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(10):S46–S54. doi:https://doi.org/10.1097/00001888-200510001-00015.
- Hammond KS. The Essential Brunswik: Beginnings, Explications, Applications. Oxford, OA: Oxford University Press; 2001.
- Buijze GA, Guitton TG, van Dijk CN, Ring D. Training improves interobserver reliability for the diagnosis of scaphoid fracture displacement. Clin Orthop Relat Res. 2012;470(7):2029–2034. doi:https://doi.org/10.1007/s11999-012-2260-4.
- Østergaard ML, Rue Nielsen K, Albrecht-Beste E, Kjaer Ersbøll A, Konge L, Bachmann Nielsen M. Simulator training improves ultrasound scanning performance on patients: a randomized controlled trial. Eur Radiol. 2019;29(6):3210–3218. doi:https://doi.org/10.1007/s00330-018-5923-z.
- Soltanimehr E, Bahrampour E, Imani MM, Rahimi F, Almasi B, Moattari M. Effect of virtual versus traditional education on theoretical knowledge and reporting skills of dental students in radiographic interpretation of bony lesions of the jaw. BMC Med Educ. 2019;19(1):233. doi:https://doi.org/10.1186/s12909-019-1649-0.
- Todsen T, Melchiors J, Charabi B, et al. Competency-based assessment in surgeon-performed head and neck ultrasonography: a validity study. Laryngoscope. 2018;128(6):1346–1352. doi:https://doi.org/10.1002/lary.26841.
- Wong V, Smith AJ, Hawkins NJ, et al. Adaptive tutorials versus web-based resources in radiology: a mixed methods comparison of efficacy and student engagement. Acad Radiol. 2015;22(10):1299–1307. doi:https://doi.org/10.1016/j.acra.2015.07.002.
- Ericsson A, Pool R. The power of purposeful practice. In: Ericsson APR, ed. Peak: Secrets from the New Science of Expertise. Boston, MA: Houghton Mifflin Harcourt; Boston, MA, USA 2016:1–25.
- Ericsson KA. Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med. 2015;90(11):1471–1486. doi:https://doi.org/10.1097/ACM.0000000000000939.
- Lee MS, Pusic M, Carriere B, Dixon A, Stimec J, Boutis K. Building emergency medicine trainee competency in pediatric musculoskeletal radiograph interpretation: a multicenter prospective cohort study. Acad Emerg Med Educ Train. 2019;3(3):269–279.
- Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015;49(6):560–575. doi:https://doi.org/10.1111/medu.12678.
- Hambleton, RK, Pitoniak, M. Setting performance standards. In: Robert LB (ed.), Educational Measurement, 4th ed. Westport, CT: Praeger; 2014:433–470.
- Zieky M, Marianne P. A primer on setting cut scores on tests of educational achievement. https://www.ets.org/research/policy_research_reports/publications/publication/2006/dbkw.
- Mortaz Hejri S, Jalili M. Standard setting in medical education: fundamental concepts and emerging challenges. Med J Islam Repub Iran. 2014;19(28):34.
- Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychol Bull. 1955;52(4):281–302. doi:https://doi.org/10.1037/h0040957.
- Cizek GJ, Bunch MB. Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests. Thousand Oaks, California: SAGE Publications Inc.; 2006.
- Ebel RL. Essentials of Educational Measurement. 2nd ed. Upper Saddle River, New Jersey: Prentice Hall; 1972.
- Weiss DJ, Shanteau J. Empirical assessment of expertise. Hum Factors. 2003;45(1):104–114. doi:https://doi.org/10.1518/hfes.45.1.104.27233.
- Cook DA, Lineberry M. Consequences validity evidence: evaluating the impact of educational assessments. Acad Med. 2016;91(6):785–795. doi:https://doi.org/10.1097/ACM.0000000000001114.
- Yudkowsky R, Park YS, Lineberry M, Knox A, Ritter EM. Setting mastery learning standards. Acad Med. 2015;90(11):1495–1500. doi:https://doi.org/10.1097/ACM.0000000000000887.
- Dixon A. Pediatric fractures - an educational needs assessment of Canadian pediatric emergency medicine residents. Open Access Emerg Med. 2015;7:25–29. doi:https://doi.org/10.2147/OAEM.S82393.
- Reeder BM, Lyne ED, Patel DR, Cucos DR. Referral patterns to a pediatric orthopedic clinic: implications for education and practice. Pediatrics. 2004;113(3):e163–719. doi:https://doi.org/10.1542/peds.113.3.e163.
- Ryan LM, DePiero AD, Sadow KB, et al. Recognition and management of pediatric fractures by pediatric residents. Pediatrics. 2004;114(6):1530–1533. doi:https://doi.org/10.1542/peds.2004-0120.
- Taras HL, Nader PR. Ten years of graduates evaluate a pediatric residency program. Am J Dis Child. 1990;144(10):1102–1105. doi:https://doi.org/10.1001/archpedi.1990.02150340046021.
- Trainor JL, Krug SE. The training of pediatric residents in the care of acutely ill and injured children. Arch Pediatr Adolesc Med. 2000;154(11):1154–1159. doi:https://doi.org/10.1001/archpedi.154.11.1154.
- Al‐Sani F, Prasad S, Panwar J, et al. Adverse events from emergency physician pediatric extremity radiograph interpretations: a prospective cohort study. Acad Emerg Med. 2020;27(2):128–138. doi:https://doi.org/10.1111/acem.13884.
- van Merrienboer JJG, Kirschner PA, Kester L. Taking the load off a learner’s mind: instructional design for complex learning. Educ Psychol. 2010;38(1):5–13. doi:https://doi.org/10.1207/S15326985EP3801_2.
- Vandewaetere M, Manhaeve D, Aertgeerts B, Clarebout G, Van Merriënboer JJ, Roex A. 4C/ID in medical education: how to design an educational program based on whole-task learning: AMEE Guide No. 93. Med Teach. 2015;37(1):4–20. doi:https://doi.org/10.3109/0142159X.2014.928407.
- Wickens CD, Hutchins S, Carolan T, Cumming J. Effectiveness of part-task training and increasing-difficulty training strategies: a meta-analysis approach. Hum Factors. 2013;55(2):461–470. doi:https://doi.org/10.1177/0018720812451994.
- Pusic M, Pecaric M, Boutis K. How much practice is enough? Using learning curves to assess the deliberate practice of radiograph interpretation. Acad Med. 2011;86(6):731–736. doi:https://doi.org/10.1097/ACM.0b013e3182178c3c.
- Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med. 2006;18(1):50–57. doi:https://doi.org/10.1207/s15328015tlm1801_11.
- Hejri SM, Jalili M. Standard setting in medical education: fundamental concepts and emergening challenges. Med J Islamic Republic Iran. 2014;28(34):1–6.
- Hurtz GM, Hertz NR. How many raters should be used for establishing cut-off scores with the Agnoff method? A generalizability theory study. Educ Psychol Meas. 1999;59(6):885–897. doi:https://doi.org/10.1177/00131649921970233.
- Pusic MV, Andrews JS, Kessler DO, et al. Prevalence of abnormal cases in an image bank affects the learning of radiograph interpretation. Med Educ. 2012;46(3):289–298. doi:https://doi.org/10.1111/j.1365-2923.2011.04165.x.
- Downing SM. Item response theory: applications of modern test theory in medical education. Med Educ. 2003;37(8):739–745. doi:https://doi.org/10.1046/j.1365-2923.2003.01587.x.
- Boutis K, Howard A, Constantine E, Cuomo A, Somji F, Narayanan U. Evidence to practice: orthopedic management of common pediatric fractures. J Pediatr Orthop. 2015;35(1):18–23. doi:https://doi.org/10.1097/BPO.0000000000000223.
- Boutis K, Willan AR, Babyn P, Alman B, Narayanan U, Schuh S. A randomized, controlled trial of a removable brace versus casting in children with low-risk ankle fractures. Pediatrics. 2007;119(6):e1256–e1263. doi:https://doi.org/10.1542/peds.2006-2958.
- Flynn JM, Skaggs DL, Waters PM. Rockwood and Wilkins’ Fractures in Children. 7th ed. Lippincott-Raven Publishers; 2014:1252.
- Boutis K. Common pediatric fractures treated with minimal intervention. Pediatr Emerg Care. 2010;26(2):152–157. quiz 158-62. doi:https://doi.org/10.1097/PEC.0b013e3181ce310c.
- Sabhaney V, Boutis K, Yang G, et al. Bone fractures in children: is there an association with obesity?J Pediatr. 2014;165(2):313–318. doi:https://doi.org/10.1016/j.jpeds.2014.04.006.
- Cendan J, Wier D, Behrns K. A primer on standards setting as it applies to surgical education and credentialing. Surg Endosc. 2013;27(7):2631–2637. doi:https://doi.org/10.1007/s00464-012-2771-9.
- Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Health Sciences. 2nd ed. Lawrence Erlbaum Associates, Inc; 2007:234.
- Shepard LA. The centrality of test use and consequences for test validity. Educ Meas Issues Pract. 2005;16(2):5–24. doi:https://doi.org/10.1111/j.1745-3992.1997.tb00585.x.
- Kramer MS, Roberts-Bräuer R, Williams RL. Bias and ‘overcall’ in interpreting chest radiographs in young febrile children. Pediatrics. 1992;90(1):11–13.
- Zhao S, Wang S, Pan P, et al. Magnitude, risk factors, and factors associated with adenoma miss rate of tandem colonoscopy: a systematic review and meta-analysis. Gastroenterology. 2019;156(6):1661–1674.e11. doi:https://doi.org/10.1053/j.gastro.2019.01.260.
- Searle J. Defining competency - the role of standard setting. Med Educ. 2000;34(5):363–366. doi:https://doi.org/10.1046/j.1365-2923.2000.00690.x.
- Itri JN, Patel SH. Heuristics and cognitive error in medical imaging. AJR Am J Roentgenol. 2018;210(5):1097–1105. doi:https://doi.org/10.2214/AJR.17.18907.
- Taylor PM. A review of research into the development of radiologic expertise: implications for computer-based training. Acad Radiol. 2007;14(10):1252–1263. doi:https://doi.org/10.1016/j.acra.2007.06.016.
- Kwan C, Pusic M, Pecaric M, Weerdenburg K, Tessaro M, Boutis K. The variable journey in learning to interpret pediatric point-of-care ultrasound images: a multicenter prospective cohort study. Acad Emerg Med Educ Train. 2019;4(2):111–122.
- John-Matthews JS, Wallace MJ, Robinson L. The Delphi technique in radiography education research. Radiography. 2017;23(Suppl 1):S53–S57. doi:https://doi.org/10.1016/j.radi.2017.03.007.
- Wells GL, Windschitl PD. Stimulus sampling and social psychological experimentation. Pers Soc Psychol Bull. 1999;25(9):1115–1125. doi:https://doi.org/10.1177/01461672992512005.
- Maher BA. Stimulus sampling in clinical research: representative design reviewed. J Consult Clin Psychol. 1978;46(4):643–647. doi:https://doi.org/10.1037//0022-006x.46.4.643.
- Bourque J, Skinner H, Dupré J, et al. Performance of the Ebel standard-setting method in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. J Educ Eval Health Prof. 2020;17(12):12. doi:https://doi.org/10.3352/jeehp.2020.17.12.
- Mu EW, Pusic M, Coneybeare M, Meehan SA. Learning analytics to enhance dermatopathology education among dermatology residents. J Drugs Dermatol. 2019;18(12):1231–1236.
- Smith EV, Jr, Stone GE. Criterion Referenced Testing: Practice Analysis to Score Reporting Using Rasch Measurement. Maple Grove, Minn: JAM Press; 2009.
- Diaz FN, Ulla M. Validation of an informatics tool to assess resident’s progress in developing reporting skills. Insights Imaging. 2019;10(1):96. doi:https://doi.org/10.1186/s13244-019-0772-0.
- Kwan BYM, Mbanwi A, Cofie N, et al. Creating a competency-based medical education curriculum for Canadian Diagnostic Radiology Residency (Queen’s Fundamental Innovations in Residency Education)-Part 1: transition to discipline and foundation of discipline stages. Can Assoc Radiol J. 2020. doi:https://doi.org/10.1177/0846537119894723.
- Schmitt JE, Scanlon MH, Servaes S, Levin D, Cook TS. Milestones on a shoestring: a cost-effective, semi-automated implementation of the new ACGME requirements for radiology. Acad Radiol. 2015;22(10):1287–1293. doi:https://doi.org/10.1016/j.acra.2015.02.013.
- van der Laan MJ, Rose S. Targeted Learning in Data Science. New York, NY: Springer; 2018.
- Boutis K, Cano S, Pecaric M, et al. Interpretation difficulty of normal versus abnormal radiographs using a pediatric example. Can Med Ed J. 2016;7(1):e68-77–e77. doi:https://doi.org/10.36834/cmej.36654.