A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design

References

• Alimo-Metcalfe B. 360° feedback and leadership development. International Journal of Selection and Assessment 1998; 6(1)35–44
   Web of Science ®Google Scholar
• Archer JC, Norcini J, Davies H. Use of SPRAT for peer review of paediatricians in training. British Medical Journal 2005; 330: 1251–1253
   PubMed Web of Science ®Google Scholar
• Atwater L, Waldman D. Accountability in 360° feedback. Human Resources Magazine 1998; 43(6)96–102
   Google Scholar
• Barclay J, Harland L. Peer performance appraisals: the impact of rater competence, rater location and rating correctability on fairness perceptions. Group and Organization Management 1995; 20(1)39–60
   Web of Science ®Google Scholar
• Becker T, Klimoski R. A field study of the relationship between the organizational feedback environment and performance. Personnel Psychology 1989; 42: 343–358
   Web of Science ®Google Scholar
• Bernardin H, Dahmus S, Redmon G. Attitudes of first-line supervisors toward subordinate appraisals. Human Resource Management 1993; 32(2–3)315–324
   Web of Science ®Google Scholar
• Bettenhausen K, Fedor D. Peer and upward appraisals: a comparison of their benefits and problems. Group and Organization Management 1997; 22(2)236–263
   Web of Science ®Google Scholar
• Bilsbury N. Academy of Human Resource Development Conference Proceedings. A case study: employee perceptions of the efficacy of 360 degree feedback at a large Midwestern utility. Academy of Human Resources Development, Oak Brook, IL 1998
   Google Scholar
• Bracken D, Summers L, Fleenor J. High-tech 360°. Training and Development 1998; 52(8)42–46
   Google Scholar
• Butterfield OS, Pearson JA. Nurses in resident evaluation: a qualitative study of the participants’ perspectives. Evaluation and the Health Professions 1990; 13: 453–473
   PubMed Web of Science ®Google Scholar
• Butterfield PS, Mazzaferri EL. A new rating form for use by nurses in assessing residents’ humanistic behaviour. Journal of General Internal Medicine 1991; 6: 155–161
   PubMed Web of Science ®Google Scholar
• Cardy R, Dobbins G. Performance Appraisal: Alternative Perspectives. South-Western Publishing, Cincinnati, OH 1994
   Google Scholar
• Carline JD, Wenrick M, Ramsey PG. Characteristics of ratings of physician competence by professional associates. Evaluation and the Health Professions 1989; 12: 409–423
   PubMed Web of Science ®Google Scholar
• Church A, Bracken D. Advancing the state-of-the-art of 360° feedback: guest editors’ comments on the research and practice of multi-rater assessment methods. Group and Organization Management 1997; 22(2)149–161
   Web of Science ®Google Scholar
• Church AH. Do higher performing managers actually receive better ratings? a validation of multirater assessment methodology. Consulting Psychology Journal: Practice and Research 2000; 52: 99–116
   Google Scholar
• Crossley T, Taylor I. Developing competitive advantage through 360° feedback. American Journal of Management Development 1995; 1(1)6–17
   Google Scholar
• Davidge AM, Hull A.L. A system for the evaluation of medical students’ clinical competence. Journal of Medical Education 1980; 55: 65–67
   PubMedGoogle Scholar
• Dawson-Saunders B, Paiva REA. The validity of clerkship performance evaluations. Medical Education 1986; 20: 240–245
   PubMed Web of Science ®Google Scholar
• DeSimone R. Establishing the link: relating a 360 degree management assessment and development process to the bottom line. Health Care Supervisor 1998; 17: 31–36
   PubMedGoogle Scholar
• Dielman TE, Hull AL, Davis WK. Psychometric properties of clinical performance ratings. Evaluation and the Health Professions 1980; 3: 103–117
   Google Scholar
• Dominick P, Reilly R, McGourty J. The effects of peer feedback on team member behaviour. Group and Organization Management 1997; 22(4)508–520
   Web of Science ®Google Scholar
• Druskatt V, Wolff S. Effects and timing of developmental peer appraisals in self-managing work groups. Journal of Applied Psychology 1999; 84(1)58–74
   PubMed Web of Science ®Google Scholar
• Edwards M, Ewen A. 360°. Feedback: The Powerful New Model for Employee Assessment and Performance Improvement. AMACOM, New York 1996
   Google Scholar
• Evans R, Elwyn G, Edwards A. Review of instruments for peer assessment of physicians. British Medical Journal 2004; 328: 1240–1243
   PubMed Web of Science ®Google Scholar
• Facteau CL, Facteau CD, Schoel LC, Russell EA, Poteet ML. Reactions of leaders to 360-degree feedback from subordinates and peers. Leadership Quarterly 1998; 9(4)427–448
   Web of Science ®Google Scholar
• Farh J, Dobbins G. Effects of self-esteem on leniency bias in self-reports of performance: a structural equation model analysis. Personnel Psychology 1989; 42: 835–850
   Web of Science ®Google Scholar
• Fedor D, Bettenhausen K. The impact of purpose, participant preconceptions and rating level on the acceptance of peer evaluations. Group and Organization Studies 1989; 14(2)182–197
   Web of Science ®Google Scholar
• Fletcher C. The implications of research on gender differences in self-assessment and 360 degree appraisal. Human Resource Management 1999; 9(1)39–46
   Google Scholar
• Folger R, Cropanzano R. Organizational justice and performance evaluation: test and trial measures. Organizational Justice and Human Resource Management. Sage Publications, Beverly Hills, CA 1998
   Google Scholar
• Garavan T, Morley M, Flynn M. 360-degree feedback: its role in employee development. Journal of Management Development 1997; 13(2–3)134–148
   Google Scholar
• General Medical Council. Good Medical Practice. GMC, London 2001
   Google Scholar
• Ghorpade J. Managing five paradoxes of 360-degree feedback. Academy of Management Executive 2000; 14: 140–150
   Google Scholar
• Handy L, Devine M, Heath L. 360° Feedback: Unguided Missile or Powerful Weapon?. Ashridge Management Research Group, Berkhamsted 1996
   Google Scholar
• Hautaluoma J, Jobe L, Visser S, Donkersgoed W. Employee reactions to different upward feedback methods. 1992, paper presented to the 7th Annual Meeting of the Society for Industrial and Organizational Psychology, Montreal
   Google Scholar
• Hazucha J, Hezlett S, Schneider R. The impact of 360-degree feedback on management skills development. Human Resource Management 1993; 32(2–3)325–351
   Web of Science ®Google Scholar
• Hoffman R. Ten reasons why you should be using 360-degree feedback. Human Resources Magazine 1995; 40: 82–86
   Google Scholar
• Ilott I, Bunch G. Competencies of basic surgical trainees. Annals of the Royal College of Surgeons of England 1980; 1(Suppl.)14–16
   Google Scholar
• Kanouse D. Why multi-rater feedback systems fail?. Human Resources Focus 1998; 75: 3–4
   Google Scholar
• Kaplan R. 360° feedback PLUS: boosting the power of co-worker ratings for executives. Human Resource Management 1993; 32(2–3)299–314
   Web of Science ®Google Scholar
• Keck JW, et al. Efficacy of cognitive/non cognitive measures in predicting resident physician performance. Journal of Medical Education 1979; 54: 759–765
   PubMedGoogle Scholar
• Keeping L, Levy P, Brown D. Examining Self-Appraisal Formality and Expectations on Appraisal Reactions. GA, Society for Industrial and Organizational Psychology, Atlanta 1999
   Google Scholar
• Lepsinger R, Lucia A. Creating champions for 360° feedback. Training and Development 1998; 52(2)49–53
   Google Scholar
• Lindsey E, Holmes V, McCall M. Key Events in Executives’ Lives. Centre for Creative Leadership, Greensboro, NC 1997
   Google Scholar
• Linn BS, Arostegui M, Zeppa R. Performance self assessment. British Journal of Medical Education 1975; 9: 98–101
   PubMedGoogle Scholar
• London M, Beatty R. 360° feedback as a competitive advantage. Human Resource Management 1993; 32(2–3)353–372
   Web of Science ®Google Scholar
• London M, Larsen H, Thisted L. Relationships between feedback and self-development. Group and Organization Management 1999; 24(1)5–27
   Web of Science ®Google Scholar
• London M, Smither J. Can multi-source feedback change perceptions of goal accomplishment, self-evaluations and performance-related outcomes? Theory-based applications and directions for research. Personnel Psychology 1995; 48(4)803–839
   Web of Science ®Google Scholar
• London M, Wohlers A, Gallagher P. 360° feedback surveys: a source of feedback to guide management development. Journal of Management Development 1990; 9: 17–31
   Google Scholar
• Mabe P, West S. Validity of self-evaluation of ability: a review and meta-analysis. Applied Psychology: An International Review 1982; 67: 280–296
   Web of Science ®Google Scholar
• Martocchio J, Judge T. Relationship between conscientiousness and learning in employee training: mediating influences of self-deception and self-efficacy. Journal of Applied Psychology 1997; 82(5)764–773
   PubMed Web of Science ®Google Scholar
• Maurer T, Tarulli B. Acceptance of peer/upward performance appraisal systems: role of work context factors and beliefs about managers’ development capability. Human Resource Management 1996; 35(2)217–241
   Web of Science ®Google Scholar
• Maxim BR, Dielman TE. Dimensionality, internal consistency and interrrater reliability of clinical performance ratings. Medical Education 1987; 21: 130–137
   PubMed Web of Science ®Google Scholar
• McCarthy A, Garavan T. Developing self-awareness in the managerial career development process: the value of 360° feedback and the MBTI. Journal of European Industrial Training 1999; 23(9)437–445
   Google Scholar
• McCarthy AM, Garavan TN. 360 Degree feedback processes: performance improvement and employee career development. Journal of European Industrial Training 2001; 25(1)3–32
   Google Scholar
• McEvoy G, Buller P. User acceptance of peer appraisals in an industrial setting. Personnel Psychology 1987; 40: 785–797
   Web of Science ®Google Scholar
• Murphy K, Cleveland J. Performance Appraisal: An Organizational Perspective. Allyn & Bacon, Boston, MA 1991
   Google Scholar
• Newble D, Paget N, McLaren B. Revalidation in Australia and New Zealand: approach of Royal Australasian College of Physicians. British Medical Journal 1999; 319(7218)1185–1188, (Intl edn)
   PubMed Web of Science ®Google Scholar
• O’Reilly B. 360° feedback can change your life. Fortune 1994; 130(8)93–7
   Google Scholar
• Payne T. Editorial: 360 degree assessment and feedback. International Journal of Selection and Assessment 1998; 6(1)16–18
   Web of Science ®Google Scholar
• Penny JA. Exploring differential item functioning in a 360-degree assessment: rater source and method of delivery. Organisational Research Methods 2003; 6(1)61–79
   Web of Science ®Google Scholar
• Ramsey PG, Carline JD, Blank LL, Wenrich MD. Feasibility of hospital-based use of peer ratings to evaluate the performances of practicing physicians. Academic Medicine 1996; 71(4)364–370
   PubMed Web of Science ®Google Scholar
• Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson B, LoGerfo JP. Use of peer ratings to evaluate physician performance. Journal of the American Medical Association 1993; 269: 1655–1660
   PubMed Web of Science ®Google Scholar
• Riggio R, Cole E. Agreement between subordinate and superior ratings of supervisory performance and effects on self and subordinate job satisfaction. Journal of Occupational and Organizational Psychology 1992; 65: 151–158
   Web of Science ®Google Scholar
• Risucci DA, Tortolania AJ, Ward RJ. Ratings of surgical residents by Self supervisors and peers. Surgery, Gynaecology + Obstetrics 1989; 169: 519–526
   PubMedGoogle Scholar
• Saavedra R, Kwun S. Peer evaluation in self-managing work groups. Journal of Applied Psychology 1993; 78(3)450–462
   Web of Science ®Google Scholar
• Southgate L, Hays RB, Norcini J, Mulholland H, Ayers B, Woolliscroft J, Cusimano M, McAvoy P, Ainsworth M, Haist S, Campbell M. Setting performance standards for medical practice: a theoretical framework. Medical Education 2001; 35(5)474–481
   PubMed Web of Science ®Google Scholar
• Steensma C, Gould L, Moseley C. Using a group-based 360 degree feedback process to facilitate the merger of four marketing units at Disney Networks. US Human Resource Planning 1998; 2(4)11–15
   Google Scholar
• Tornow W. Perceptions or reality: is multi-perspective measurement a means to an end?. Human Resource Management 1993; 32(2/3)221–229
   Web of Science ®Google Scholar
• Tornow W, London M. Maximizing the Value of 360-degree Feedback: A Process for Successful Individual and Organizational Development. Jossey-Bass, San Francisco, CA 1998
   Google Scholar
• Towers-Perrin. 360° Feedback: The Global Perspective. Towers Perrin, London 1998
   Google Scholar
• Van der Heijden BI, Nijhof AH. The value of subjectivity: problems and prospects for 360 degree appraisal systems. International Journal of Resource Management 2004; 15(3)493–511
   Web of Science ®Google Scholar
• Waldman D. Predictors of employee preferences for multi-rater and group-based performance appraisal. Group and Organization Management 1997; 22(2)264–287
   Web of Science ®Google Scholar
• Waldman D, Bowen D. The acceptability of 360° appraisals: a customer-supplier relationship perspective. Human Resource Management 1998; 37(2)117–129
   Web of Science ®Google Scholar
• Wenrich MD, Carline JD, Giles LM, Ramsay P. Ratings of the performances of practising internists by hospital based registered nurses. Academic Medicine 1993; 68: 680–687
   PubMed Web of Science ®Google Scholar
• Whitehouse A, Hassell A, Wood L, Wall D, Walzman M, Campbell I. Development and reliability testing of a new form for 360 degree assessment of senior house officers’ professional behaviour, as specified by the General Medical Council. Medical Teacher 2005; 27: 252–258
   PubMed Web of Science ®Google Scholar
• Whitehouse A, Wall D, Walzman M. Pilot study of 360 degree assessment of personal skills of senior house officers. Hospital Medicine 2002; 63(3)172–175
   PubMed Web of Science ®Google Scholar
• Wimer S, Nowack K. Thirteen common mistakes using 360-degree feedback. Training and Development 1998; 52(5)69–79
   Google Scholar
• Wood L, Campbell I. 360 degree assessment: encouraging results of a 6 year study. Annual Meeting of the Association for the Study of Medical Education. Liverpool, UK 2004
   Google Scholar
• Woolliscroft JO, Howell JD, Patel BP, Swanson DB. Resident–patient interactions: the humanistic qualities of internal medicine residents assessed by patients, attending physicians, program supervisors, and nurses. Academic Medicine 1994; 69(3)216–224
   PubMed Web of Science ®Google Scholar
• Yukl G, Lepsinger R. How to get the most out of 360° feedback. Training and Development 1995; 32(12)45–50
   Google Scholar
• Zhang Z, Luk W, Arthur D, Wong T. Nursing competencies: personal characteristics contributing to effective nursing performance. Advances in Nursing 2001; 33(4)467–474
   Web of Science ®Google Scholar