Implementing programmatic assessment transforms supervisor attitudes: An explanatory sequential mixed methods study

References

• Acai A, Li SA, Sherbino J, Chan T. 2019. Attending emergency physicians' perceptions of a programmatic workplace-based assessment system: the McMaster Modular Assessment Program (McMAP). Teach Learn Med. 31(4):434–444.
   PubMed Web of Science ®Google Scholar
• Bacon R, Holmes K, Palermo C. 2017. Exploring subjectivity in competency-based assessment judgements of assessors. Nutr Diet. 74(4):357–364.
   PubMed Web of Science ®Google Scholar
• Bakker A, Cai J, English L, Kaiser G, Mesa V, Van Dooren W. 2019. Beyond small, medium, or large: points of consideration when interpreting effect sizes. Educ Stud Math. 102(1):1–8.
   Web of Science ®Google Scholar
• Barbour R. 2007. Qualitative research kit: doing focus groups. 2nd ed. London: Sage.
   Google Scholar
• Begley A, Bird A, Palermo C. 2020. Developing national conceptual understanding to describe entry-to-practice dietetics competence. J Nutr Educ Behav. 52(4):351–358.
   PubMed Web of Science ®Google Scholar
• Benjamini Y, Hochberg Y. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc. 57(1):289–300.
   Google Scholar
• Berger R. 2015. Now I see it, now I don’t: researcher’s position and reflexivity in qualitative research. Qual Res. 15(2):219–234.
   Web of Science ®Google Scholar
• Cantillon P, Dornan T, De Grave W. 2019. Becoming a clinical teacher: identity formation in context. Acad Med. 94(10):1610–1618.
   PubMed Web of Science ®Google Scholar
• Carless SA, Robertson K, Willy J, Hart M, Chea S. 2012. Successful postgraduate placement experiences: what is the influence of job and supervisor characteristics? Aust Psychol. 47(3):156–164.
   Web of Science ®Google Scholar
• Castanelli DJ, Weller JM, Molloy E, Bearman M. 2020. Shadow systems in assessment: how supervisors make progress decisions in practice. Adv Health Sci Educ Theory Pract. 25(1):131–147.
   PubMed Web of Science ®Google Scholar
• Creswell J. 2014. Research design: qualitative, quantitative, and mixed methods approaches. 4th ed. Los Angeles (CA): Sage.
   Google Scholar
• Dalton M, Davidson M, Keating J. 2011. The Assessment of Physiotherapy Practice (APP) is a valid measure of professional competence of physiotherapy students: a cross-sectional study with Rasch analysis. J Physiother. 57(4):239–246.
   PubMed Web of Science ®Google Scholar
• de Jonge L, Timmerman AA, Govaerts MJB, Muris JWM, Muijtjens AMM, Kramer AWM, van der Vleuten CPM. 2017. Stakeholder perspectives on workplace-based performance assessment: towards a better understanding of assessor behaviour. Adv Health Sci Educ Theory Pract. 22(5):1213–1243.
   PubMed Web of Science ®Google Scholar
• Fritz CO, Morris PE, Richler JJ. 2012. Effect size estimates: current use, calculations, and interpretation. J Exp Psychol. 141(1):2–18.
   Google Scholar
• Gale N, Heath G, Cameron E, Rashid S, Redwood S. 2013. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 13(1):117–124.
   PubMed Web of Science ®Google Scholar
• Haji F, Morin MP, Parker K. 2013. Rethinking programme evaluation in health professions education: beyond 'did it work?'. Med Educ. 47(4):342–351.
   PubMed Web of Science ®Google Scholar
• Holmboe SE. 2015. Realizing the promise of competency-based medical education. Acad Med. 90(4):411–413.
   PubMed Web of Science ®Google Scholar
• Iobst WF, Holmboe ES. 2020. Programmatic assessment: the secret sauce of effective CBME implementation. J Grad Med Educ. 12(4):518–521.
   PubMedGoogle Scholar
• Jamieson J, Jenkins G, Beatty S, Palermo C. 2017. Designing programmes of assessment: a participatory approach. Med Teach. 39(11):1182–1188.
   PubMed Web of Science ®Google Scholar
• Kilminster SM, Jolly BC. 2000. Effective supervision in clinical practice settings: a literature review. Med Educ. 34(10):827–840.
   PubMed Web of Science ®Google Scholar
• Kraft MA. 2020. Interpreting effect sizes of education interventions. Educ Researcher. 49(4):241–253.
   Web of Science ®Google Scholar
• Lockyer J, Carraccio C, Chan M-K, Hart D, Smee S, Touchie C, Holmboe ES, Frank JR. 2017. Core principles of assessment in competency-based medical education. Med Teach. 39(6):609–616.
   PubMed Web of Science ®Google Scholar
• Malterud K, Siersma VD, Guassora AD. 2016. Sample size in qualitative interview studies: guided by Information Power. Qual Health Res. 26(13):1753–1760.
   PubMed Web of Science ®Google Scholar
• Massie J, Ali JM. 2016. Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings. Adv Health Sci Educ Theory Pract. 21(2):455–473.
   PubMed Web of Science ®Google Scholar
• Meeuwissen SNE, Stalmeijer RE, Govaerts M. 2019. Multiple-role mentoring: mentors' conceptualisations, enactments and role conflicts. Med Educ. 53(6):605–615.
   PubMed Web of Science ®Google Scholar
• Nasser R, Morley C, Cook S, Coleman J, Berenbaum S. 2014. Dietitians' perceptions of precepting: knowledge, skills, attitudes, barriers, and training. Can J Diet Pract Res. 75(1):7–14.
   PubMed Web of Science ®Google Scholar
• O’Connor A, Cantillon P, Parker M, McCurtin A. 2019. Juggling roles and generating solutions; practice-based educators' perceptions of performance-based assessment of physiotherapy students. Physiotherapy. 105(4):446–452.
   PubMed Web of Science ®Google Scholar
• Palermo C, Conway J, Beck EJ, Dart J, Capra S, Ash S. 2016. Methodology for developing competency standards for dietitians in Australia. Nurs Health Sci. 18(1):130–137.
   PubMed Web of Science ®Google Scholar
• Pitkänen S, Kääriäinen M, Oikarainen A, Tuomikoski A-M, Elo S, Ruotsalainen H, Saarikoski M, Kärsämänoja T, Mikkonen K. 2018. Healthcare students' evaluation of the clinical learning environment and supervision – a cross-sectional study. Nurse Educ Today. 62:143–149.
   PubMed Web of Science ®Google Scholar
• Rattray J, Jones MC. 2007. Essential elements of questionnaire design and development. J Clin Nurs. 16(2):234–243.
   PubMed Web of Science ®Google Scholar
• Rich JV, Fostaty Young S, Donnelly C, Hall AK, Dagnone JD, Weersink K, Caudle J, Van Melle E, Klinger DA. 2020. Competency-based education calls for programmatic assessment: but what does this look like in practice? J Eval Clin Pract. 26(4):1087–1095.
   PubMed Web of Science ®Google Scholar
• Sargeant J, Armson H, Chesluk B, Dornan T, Eva K, Holmboe E, Lockyer J, Loney E, Mann K, van der Vleuten C. 2010. The processes and dimensions of informed self-assessment: a conceptual model. Acad Med. 85(7):1212–1220.
   PubMed Web of Science ®Google Scholar
• Schut S, Driessen E, Tartwijk J, van der Vleuten C, Heeneman S. 2018. Stakes in the eye of the beholder: an international study of learners' perceptions within programmatic assessment. Med Educ. 52(6):654–663.
   PubMed Web of Science ®Google Scholar
• Schut S, Heeneman S, Bierer B, Driessen E, Tartwijk J, van der Vleuten C. 2020. Between trust and control: teachers' assessment conceptualisations within programmatic assessment. Med Educ. 54(6):528–537.
   PubMed Web of Science ®Google Scholar
• Schut S, Maggio LA, Heeneman S, van Tartwijk J, van der Vleuten C, Driessen E. 2021. Where the rubber meets the road – an integrative review of programmatic assessment in health care professions education. Perspect Med Educ. 10(1):6–13.
   PubMed Web of Science ®Google Scholar
• Schuwirth LWT, van der Vleuten CPM. 2019. Current assessment in medical education: programmatic assessment. J Appl Test Tech. 20(S2):2–10.
   Google Scholar
• Spencer J. 2003. The clinical teaching context: a cause for concern. Med Educ. 37(3):182–183.
   PubMed Web of Science ®Google Scholar
• Torre DM, Schuwirth LWT, van der Vleuten CPM. 2020. Theoretical considerations on programmatic assessment. Med Teach. 42(2):213–220.
   PubMed Web of Science ®Google Scholar
• Trede F, Smith M. 2014. Workplace educators' interpretations of their assessment practices: a view through a critical practice lens. Assess Eval High Educ. 39(2):154–167.
   Web of Science ®Google Scholar
• van der Vleuten C, Schuwirth L, Driessen E, Dijkstra J, Tigelaar D, Baartman L, van Tartwijk J. 2012. A model for programmatic assessment fit for purpose. Med Teach. 34(3):205–214.
   PubMed Web of Science ®Google Scholar
• Wilkinson T, Tweed M. 2018. Deconstructing programmatic assessment. Adv Med Educ Pract. 9:191–197.
   PubMed Web of Science ®Google Scholar