Three methods of multi-source feedback compared: A plea for narrative comments and coworkers’ perspectives

Abstract

Background: Doctor performance assessments based on multi-source feedback (MSF) are increasingly central in professional self-regulation. Research has shown that simple MSF is often unproductive. It has been suggested that MSF should be delivered by a facilitator and combined with a portfolio.

Aims: To compare three methods of MSF for consultants in the Netherlands and evaluate the feasibility, topics addressed and perceived impact upon clinical practice.

Method: In 2007, 38 facilitators and 109 consultants participated in the study. The performance assessment system was composed of (i) one of the three MSF methods, namely, Violato's Physician Achievement Review (PAR), the method developed by Ramsey et al. for the American Board of Internal Medicine (ABIM), or the Dutch Appraisal and Assessment Instrument (AAI), (ii) portfolio, (iii) assessment interview with a facilitator and (iv) personal development plan. The evaluation consisted of a postal survey for facilitators and consultants. Generalized estimating equations were used to assess the association between MSF method used and perceived impact.

Results: It takes on average 8 hours to conduct one assessment. The CanMEDS roles ‘collaborator’, ‘communicator’ and ‘manager’ were discussed in, respectively, 79, 74 and 71% of the assessment interviews. The ‘health advocate role’ was the subject of conversation in 35% of the interviews. Consultants are more satisfied with feedback that contains narrative comments. The perceived impact of MSF that includes coworkers’ perspectives significantly exceeds the perceived impact of methods not including this perspective.

Conclusions: Performance assessments based on MSF combined with a portfolio and a facilitator-led interview seem to be feasible in hospital settings. The perceived impact of MSF increases when it contains coworkers’ perspectives.

Additional information

Notes on contributors

K. Overeem

KARLIJN OVEREEM, MD, is a PhD student at the scientific institute for Quality of Healthcare, IQ healthcare, Radboud University Nijmegen Medical Centre.

M.J.M.H. Lombarts

KIKI LOMBARTS, PhD is researcher at the Department of Quality and Process Innovation, Academic Medical Centre, University of Amsterdam.

O.A. Arah

ONYEBUCHI ARAH, MD, PhD is Associate Professor, Department of Epidemiology, University of California, Los Angeles (UCLA), School of Public Health, Los Angeles, California.

N.S. Klazinga

NIEK KLAZINGA, MD, PhD, is Professor of Social Medicine at the Academic Medical Centre, University of Amsterdam.

R.P.T.M. Grol

RICHARD GROL, MD, PhD is Professor of Quality of Care at IQ healthcare, Radboud University Nijmegen Medical Centre.

H.C. Wollersheim

HUB WOLLERSHEIM, MD, PhD, is internist and Associate Professor at IQ healthcare, Radboud University Nijmegen Medical Centre.