Implementing programmatic assessment transforms supervisor attitudes: An explanatory sequential mixed methods study

Abstract

Introduction

Programmatic assessment (PA) is an increasingly popular approach to competency-based assessment (CBA), yet evaluation evidence is limited. This study aimed to identify and explore supervisor attitudes before and after implementing a novel PA using a sequential explanatory mixed methods design. In phase one, a survey was used to identify supervisor perspectives on work-based placements, PA and CBA. Survey results were then applied to develop focus group questions to further explore supervisor attitudes.

Results

PA was found to improve supervisor-student relationships by removing high-stakes assessment decisions and creating greater capacity for feedback and teaching, leading to a productive learning environment. Assessment was perceived as an important role and supervisors wanted to feel valued and heard within PA. Trust was conceptualised as a triad between supervisor, student and university, and enabled supervisors to engage with PA which was important for success. Supervisor learning of PA was experiential and often supported by students, highlighting the need for hands-on training.

Conclusion

Participants reported a high level of agreement with PA and CBA principles which may have made them amenable to educational change. Further research is needed to explore the experience of all stakeholders and to understand how worldviews and culture influence assessment initiatives.

Keywords:

• Roles of teacher
• other
• assessment
• best evidence medical education
• student support

Acknowledgements

The authors wish to acknowledge Dr Jonny Lo for statistical analysis guidance.

Disclosure statement

The authors report no declarations of interest. The authors alone are responsible for the content and writing of the article.

Glossary

Information power: Is a concept researchers can apply to determine adequate sampling for qualitative studies. Sufficiency of sample size is considered across five parameters: (i) the scope of the research aim; (ii) the specificity of participants; (iii) the application of established theoretical framework; (iv) the strength and quality of dialogue between researcher and participants; and (v) the analysis strategy. Researchers should systematically and iteratively consider all five parameters throughout research design and execution to determine adequate sample size. Information power is an alternative to the concept of saturation which aligns with Grounded Theory (GT) analysis and is inconsistently applied in qualitative research.

Malterud K, Siersma VD, Guassora AD. 2016. Sample size in qualitative interview studies: guided by Information Power. Qual Health Res. 26(13):1753–1760.

Additional information

Notes on contributors

Janica Jamieson

Janica Jamieson, Grad Dip Diet (Hons), is a lecturer in Nutrition and Dietetics at Edith Cowan University and a doctoral student at Monash University.

Margaret Hay

Margaret Hay, BBSc (hons), Grad Dip Epi, GCHPE, PhD, MAPS, is the Head of Monash Centre for Professional Development and Monash Online Education in the Portfolio of the Deputy Vice-Chancellor (Education) at Monash University.

Simone Gibson

Simone Gibson, Grad Dip Diet, Grad Cert Health Prof Ed, PhD, is the Director of Education for the School of Clinical Sciences.

Claire Palermo

Claire Palermo, BSci, MNutrDiet, MPH, Grad Cert Health Prof Ed, PhD, is the Director of the Monash Centre for Scholarship in Health Education and Associate Professor in the Department of Nutrition, Dietetics and Food in the Faculty of Medicine, Nursing and Health Sciences.