An investigation into the use of multi-source feedback (MSF) as a work-based assessment tool

Abstract

Introduction: This study compared Specialist Trainees' (STs) hand-selected multi-source feedback (MSF) scores with those made by their clinical supervisors and explored perceptions of both those being assessed and those assessing.

Methods: Participating STs were asked to hand a mini-PAT questionnaire to a clinical colleague of their choice and also to their Clinical Supervisor. Statistical analysis was carried out on submitted paired assessments to determine any differences in responses between clinical supervisors and hand-chosen assessors. Semi-structured interviews were held with seven nurses, seven Consultants and six postgraduate doctors.

Results: Forty pairs of mini-PAT questionnaires were analysed. Hand-chosen assessors’ ratings were significantly higher than those for clinical supervisors with respect to: “good clinical care” (p < 0.01), “good medical practice” (p < 0.05), “teaching and training” (p < 0.01), “relationship with patients” (p < 0.05) as well as for overall impression of the trainee (p < 0.05). Five themes were identified from interviews: validity of selecting assessors; anonymity of assessors; usefulness of feedback; the value of multi-professional assessors; and grading.

Discussions: There is a systematic difference in the assessment scores for trainees in MSF between clinical supervisors and hand-chosen assessors, the former scoring trainees more harshly. Grading was open to interpretation. This raised questions, especially from nurse interviewees regarding appropriate benchmarking.

Acknowledgements

The authors would like to thank all the Specialist Trainees and MSF assessors who took part in this study.

Declaration of interest: Mersey Deanery funded this project. The authors report no declarations of interest. This study received University, Strategic Health Authority and NHS Research Ethics Committee and local NHS Trust Research & Development approval.