Validating written feedback in clinical formative assessment

Abstract

Formative assessment is widely accepted as being crucial to promoting student learning and, since 2010, the UK General Medical Council has mandated its use in workplace-based clinical training for all new doctors. As a result, the Royal College of Radiologists (RCR) instituted a range of formative workplace-based assessments including the Radiology Direct Observation of Procedural Skills (Rad-DOPS), in which supervisors appraise trainees’ performance in carrying out clinical procedures. This paper reports on the quality of the written feedback in 2,500 Rad-DOPS online feedback forms in addressing the aims of the new assessment approach. Random samples of 500 were selected from the first three years of the new assessment implementation, 2010–13, and from 2016 to 17. Using an appropriate coding frame, the feedback was analysed across the samples against key trainee attributes including stage of training and level of adjudged competence. Criteria for identifying high quality feedback were derived from the literature and a simplified form of qualitative comparative analysis was used to identify the conditions associated with high quality feedback. An average of 97% of the assessments contained written feedback but the number of instances of high quality feedback was found to be exceedingly small at around 5%. The paper offers suggestions for making the feedback process more purposeful in achieving the aims of formative assessment.

Keywords:

• Formative assessment
• written feedback
• medical education
• workplace-based assessments

Acknowledgements

The authors wish to thank the Royal College of Radiologists for their permission to use the anonymised Rad-DOPS data and also the anonymous reviewers for their very helpful comments and suggestions.