Realizing the Promise and Importance of Performance-Based Assessment

REFERENCES

• Epstein , R M . 2007 . Assessment in medical education . New England Journal of Medicine , 356 : 387 – 96 .
   PubMed Web of Science ®Google Scholar
• Miller , G E . 1990 . The assessment of clinical skills/competence/performance . Academic Medicine , 65 : S63 – 7 .
   PubMed Web of Science ®Google Scholar
• Swanwick , T and Chana , N . 2009 . Workplace-based assessment . British Journal of Hospital Medicine , 70 : 290 – 3 . (London)
   Web of Science ®Google Scholar
• Van der vleuten , C and Verhoeven , B . 2013 . In-training assessment developments in postgraduate education in Europe . ANZ Journal of Surgery , doi: :10.1111/ans.12190
   Google Scholar
• Batalden , P , Leach , D , Swing , S , Dreyfus , H and Dreyfus , S . 2002 . General competencies and accreditation in graduate medical education . Health Affairs , 21 : 103 – 11 .
   PubMed Web of Science ®Google Scholar
• CanMEDS . http://www.royalcollege.ca/portal/page/portal/rc/canmeds
   Google Scholar
• General Medical Council's Good Medical Practice . http://www.gmc-uk.org/guidance/good_medical_practice.asp
   Google Scholar
• Carraccio , C , Wolfsthal , S D , Englander , R , Ferentz , K and Martin , C . 2002 . Shifting paradigms: From Flexner to competencies . Academic Medicine , 77 : 361 – 7 .
   PubMed Web of Science ®Google Scholar
• Carraccio , C L and Englander , R . 2013 . From Flexner to competencies: Reflections on a decade and the journey ahead . Academic Medicine , 88
   Web of Science ®Google Scholar
• Govaerts , M JB , van der Vleuten , C PM , Schuwirth , L WT and Muijtjens , A MM . 2007 . Broadening perspectives on clinical performance assessment: Rethinking the nature of in-training assessment . Advances in Health Sciences Education: Theory and Practice , 12 : 239 – 60 .
   PubMed Web of Science ®Google Scholar
• Rethans , J , Sturmans , F , Drop , R , van der Vleuten , C and Hobus , P . 1991 . Does competence of general practitioners predict their performance? Comparison between examination setting and actual practice . British Medical Journal , 303 : 1377 – 80 .
   PubMed Web of Science ®Google Scholar
• Kopelow , M , Schnabl , G , Hassard , T , Klass , D , Beazley , G Hechter , F . 1992 . Assessing practicing physicians in two settings using standardized patients . Academic Medicine , 67 ( Suppl 10 ) : S19 – 21 .
   Google Scholar
• Frenk , J , Chen , L , Bhutta , Z A , Cohen , J , Crisp , N Evans , T . 2010 . Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world . Lancet , 376 : 1923 – 58 .
   PubMed Web of Science ®Google Scholar
• December 2008 . Institute of Medicine. Resident duty hours: Enhancing sleep, supervision, and safety . http://www.iom.edu/Reports/2008/Resident-Duty-Hours-Enhancing-Sleep-Supervision-and-Safety.aspx. Accessed July 8, 2013.
   Google Scholar
• March 2001 . Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century . http://www.iom.edu/~/media/Files/Report%20Files/2001/Crossing-the-Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdf. Accessed July 8, 2013
   Google Scholar
• Kennedy , T J , Lingard , L , Baker , G R , Kitchen , L and Regehr , G . 2007 . Clinical oversight: Conceptualizing the relationship between supervision and safety . Journal of General Internal Medicine , 22 : 1080 – 5 .
   PubMed Web of Science ®Google Scholar
• Ten cate , O , Snell , L and Carraccio , C . 2010 . Medical competence: The interplay between individual ability and the healthcare environment . Medical Teacher , 32 : 669 – 75 .
   PubMed Web of Science ®Google Scholar
• Ten Cate , O . 2006 . Trust, competence, and the supervisor's role in postgraduate training . British Medical Journal , 333 : 748 – 51 .
   PubMed Web of Science ®Google Scholar
• Kroboth , F J , Kapoor , W , Brown , F H , Karpf , M and Levey , G S . 1985 . A comparative trial of the clinical evaluation exercise . Archives of Internal Medicine , 145 : 1121 – 3 .
   PubMed Web of Science ®Google Scholar
• Schuwirth , L W and van der vleuten , C PM . 2011 . General overview of the theories used in assessment: AMEE Guide No. 57 . Medical Teacher , 33 : 783 – 97 .
   PubMed Web of Science ®Google Scholar
• Martinez , M E and Lipson , J L . 1989 . Assessment for learning . Education Leader , 47 : 73 – 5 .
   Google Scholar
• Norcini , J , Anderson , B , Bollela , V , Burch , V , Costa , M J Duvivier , R . 2011 . Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference . Medical Teacher , 33 : 206 – 14 .
   PubMed Web of Science ®Google Scholar
• Van der vleuten , C PM . 1996 . The assessment of professional competence: Developments, research, and practical implications . Advances in Health Sciences Education , 1 : 41 – 67 .
   PubMed Web of Science ®Google Scholar
• Norcini , J and Burch , V . 2007 . Workplace-based assessment as an educational tool: AMEE Guide No 31 . Medical Teacher , 29 : 855 – 71 .
   PubMed Web of Science ®Google Scholar
• Kogan , J R , Conforti , L N , Iobst , W and Holmboe , E . 2013 . Rater cognition as a patient care problem for medical education . Academic Medicine , In press
   Google Scholar
• Schuwirth , L W and van der vleuten , C PM . 2011 . Programmatic assessment: From assessment of learning to assessment for learning . Medical Teacher , 33 : 478 – 85 .
   PubMed Web of Science ®Google Scholar
• Kogan , J R , Holmboe , E S and Hauer , K E . 2009 . Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review . Journal of the American Medical Association , 302 : 1316 – 26 .
   PubMed Web of Science ®Google Scholar
• Norcini , J J , Blank , L L , Arnold , G K and Kimball , H R . 1995 . The mini-CEX (clinical evaluation exercise): a preliminary investigation . Annals of Intern Medicine , 123 : 795 – 99 .
   PubMed Web of Science ®Google Scholar
• Norcini , J J , Blank , L L , Duffy , F D and Fortna , G S . 2003 . The mini-CEX: A method for assessing clinical skills . Annals of Internal Medicine , 138 : 476 – 81 .
   PubMed Web of Science ®Google Scholar
• Wilkinson , J R , Crossley , J G , Wragg , A , Mills , P , Cowan , G and Wade , W . 2008 . Implementing workplace-based assessment across the medical specialties in the United Kingdom . Medical Education , 42 : 364 – 73 .
   PubMed Web of Science ®Google Scholar
• Reznick , R , Regehr , G , MacRae , H , Martin , J and McCulloch , W . 1997 . Testing technical skill via an innovative “bench station” examination . American Journal of Surgery , 173 : 226 – 30 .
   PubMed Web of Science ®Google Scholar
• Cruess , R , McIlroy , J H , Cruess , S , Ginsburg , S and Steinert , Y . 2006 . The professionalism mini-evaluation exercise: A preliminary investigation . Academic Medicine , 81 ( 10 Suppl ) : S74 – 8 .
   PubMed Web of Science ®Google Scholar
• Miller , A and Archer , J . 2010 . Impact of workplace based assessment on doctors’ education and performance: a systematic review . British Medical Journal , 341 : c5064
   PubMed Web of Science ®Google Scholar
• Jennett , P A , Scott , S M and Atkinson , M A . 1995 . Patient charts and office management decisions: Chart audit and chart stimulated recall . Journal of Continuing Education in the Health Professions , 15 : 31 – 9 .
   Google Scholar
• Carr , S . 2006 . The Foundation Programme assessment tools: An opportunity to enhance feedback to trainees? . Postgraduate Medical Journal , 82 : 576 – 9 .
   PubMed Web of Science ®Google Scholar
• Davies , H , Archer , J , Southgate , L and Norcini , J . 2009 . Initial evaluation of the of the Foundation assessment programme . Medical Education , 43 : 73 – 81 .
   Google Scholar
• Benson , S , Munger , B S , Krome , R L , Maatsch , J C and Podgorny , G . 1982 . The certification examination in emergency medicine: An update . Annals of Emergency Medicine , 11 : 91 – 6 .
   PubMed Web of Science ®Google Scholar
• Jennett , P and Affleck , L . 1998 . Chart audit and chart stimulated recall as methods of needs assessment in continuing professional health education . Journal of Continuing Education in the Health Professions , 18 : 163 – 71 .
   Google Scholar
• Green , M L , Reddy , S G and Holmboe , E S . 2009 . Teaching and evaluating point of care learning with an internet-based clinical question portfolio . Journal of Continuing Education in the Health Professions , 29 : 209 – 19 .
   PubMed Web of Science ®Google Scholar
• Pelgrim , E AM , Kramer , A WM , Mokkink , H GA , van den Elsen , L , Grol , RPTM and van der Vleuten , C PM . 2011 . In-training assessment using direct observation of single-patient encounters: A literature review . Advances in Health Sciences Education: Theory and Practice , 16 : 131 – 42 .
   PubMed Web of Science ®Google Scholar
• Norcini , J J . 2005 . Current perspectives in assessment: The assessment of performance at work . Medical Education. , 39 : 880 – 9 .
   PubMed Web of Science ®Google Scholar
• Gingerich , A , Regehr , G and Eva , K W . 2011 . Rater-based assessments as social judgments: Rethinking the etiology of rater errors . Academic Medicine , 86 ( 10 Suppl ) : S1 – 7 .
   PubMed Web of Science ®Google Scholar
• Yeates , P , O’Neill , P , Mann , K and Eva , K . 2013 . Seeing the same thing differently: Mechanisms that contribute to assessor differences in directly-observed performance assessments . Advances in Health Sciences Education: Theory and Practice , 18 : 325 – 41 .
   PubMed Web of Science ®Google Scholar
• Crossley , J and Jolly , B . 2012 . Making sense of work-based assessment: Ask the right questions, in the right way, about the right things, of the right people . Medical Education , 46 : 28 – 37 .
   PubMed Web of Science ®Google Scholar
• Redfern , S , Norman , I , Calman , L , Watson , R and Murrells , T . 2002 . Assessing competence to practise in nursing: a review of the literature . Research Papers in Education , 17 : 51 – 77 .
   Google Scholar
• Lockyer , J . 2013 . Multi-source feedback: Can it meet criteria for good assessment? . Journal of Continuing Education in the Health Professions , 33 : 89 – 98 .
   PubMed Web of Science ®Google Scholar
• Lockyer , J and Clyman , S . 2008 . “ Multisource Feedback (360 degree evaluation) ” . In Practical guide to the evaluation of clinical competence , Edited by: Holmboe , E S and Hawkins , R E . Philadelphia , PA : Mosby-Elsevier .
   Google Scholar
• Baker , D P , Salas , E , King , H , Battles , J and Barach , P . 2005 . The role of teamwork in the professional education of physicians: Current status and assessment recommendations . Joint Commission Journal on Quality and Patient Safety , 31 : 185 – 202 .
   PubMedGoogle Scholar
• Interprofessional Education Collaborative. Core competencies for interprofessional collaborative practice . Available at: https://www.aamc.org/download/186750/data/. Accessed July 30, 2013
   Google Scholar
• Baker , D P , Day , R and Salas , E . 2006 . Teamwork as an essential component of high-reliability organizations . Health Services Research , 41 : 1576 – 98 .
   PubMed Web of Science ®Google Scholar
• Agency for Healthcare Research and Quality . TeamSTEPPS . Available at: http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/. Accessed July 30, 2013
   Google Scholar
• Chesluk , B , Bernabeo , E , Hess , B , Lynn , L , Reddy , S and Holmboe , E . 2012 . A new assessment tool is designed to give hospitalists feedback to improve interprofessional teamwork and produce better patient care . Health Affairs , 31 : 2485 – 92 .
   Web of Science ®Google Scholar
• Wood , L , Hassell , Whitehouse , A , Bullock , A and Wall , D . 2006 . A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design . Medical Teacher , 28 : e185 – 91 .
   PubMedGoogle Scholar
• Hess , B J , Lynn , L A , Holmboe , E S and Lipner , R S . 2009 . Toward better care coordination through improved communication with referring physicians . Academic Medicine , 84 ( Suppl 10 ) : S109 – 12 .
   Google Scholar
• Lipner , R S , Blank , L L , Leas , B F and Fortna , G S . 2002 . The value of patient and peer ratings in recertification . Academic Medicine , 77 ( 10 Suppl ) : S64 – 6 .
   Google Scholar
• Levinson , W , Lesser , C S and Epstein , R M . 2010 . Developing physician communication skills for patient centered care . Health Affairs , 29 : 1310 – 18 .
   PubMed Web of Science ®Google Scholar
• Agency for Healthcare Research and Quality. Consumer Assessment of Healthcare Providers and Systems (CAHPS) . Available at: http://cahps.ahrq.gov/.Accessed July 30, 2013
   Google Scholar
• Solomon , L S , Hays , R D , Zaslavsky , A M , Alan , M , Ding , L and Cleary , P D . 2005 . Psychometric properties of a group-level Consumer Assessment of Health Plans Study (CAHPS) instrument . Medical Care , 43 : 53 – 60 .
   PubMed Web of Science ®Google Scholar
• Hays , R D , Chong , K , Brown , J , Spitzer , K L and Horne , K . 2003 . Patient reports and ratings of individual physicians: An evaluation of the DoctorGuide and Consumer Assessment of Health Plans Study provider-level surveys . American Journal of Medical Quality , 18 : 190 – 6 .
   PubMed Web of Science ®Google Scholar
• Wright , C , Richards , S H , Hill , J J , Roberts , M J , Norman , G R Greco , M . 2012 . Multisource feedback in medical regulation: The example of the UK GMC Patient and Colleague Questionnaires . Academic Medicine , 87 : 1668 – 78 .
   PubMed Web of Science ®Google Scholar
• Physician Achievement Review . Available at: http://www.par-program.org/information/. Accessed July 30, 2013
   Google Scholar
• Gray , B , Weng , W and Holmboe , E S . 2012 . An assessment of patient based and practice infrastructure based measures of the patient centered medical home. Do we need to ask the patient? . Health Services Research , 47 ( 1 Pt 1 ) : 4 – 21 .
   PubMed Web of Science ®Google Scholar
• Concato , J and Feinstein , A R . 1997 . Asking patients what they like: Overlooked attributes of patient satisfaction with primary care . American Journal of Medicine , 102 : 399 – 406 .
   PubMed Web of Science ®Google Scholar
• Agency for Healthcare Research and Quality. National measures clearinghouse . Available at: http://www.qualitymeasures.ahrq.gov/. Accessed July 30, 2013
   Google Scholar
• American Medical Association. Physician Consortium for Performance Improvement . Available at http://www.ama-assn.org/ama/pub/physician-resources/physician-consortium-performance-improvement.page. Accessed July 30, 2013
   Google Scholar
• Berenson , R A , Pronovost , P J and Krumholz , H M . Achieving the potential of health care performance measures. Timely analysis of immediate health policy issues . Available at: http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/05/achieving-the-potential-of-health-care-performance-measures.html. Accessed July 30, 2013
   Google Scholar
• Ivers , N , Jamtvedt , G , Flottorp , S , Young , J M , Odgaard-Jensen , J French , S D . 2012 . Audit and feedback: Effects on professional practice and healthcare outcomes . Cochrane Database of Systematic Reviews , doi: CD000259. doi:10.1002/14651858.CD000259.pub3
   Google Scholar
• Willett , L L , Heudebert , G R , Palonen , K P , Massie , F S , Kiefe , C I Allison , J J . 2009 . The importance of measuring competency-based outcomes: Standard evaluation measures are not surrogates for clinical performance of internal medicine residents . Teaching and Learning in Medicine , 21 : 87 – 93 .
   PubMed Web of Science ®Google Scholar
• olmboe , E S , Hess , B J , Conforti , L N and Lynn , L A . 201 . Comparative trial of a web-based tool to improve the quality of care provided to older adults in residency clinics: Modest success and a tough road ahead . Academic Medicine , 87 : 627 – 34 .
   Google Scholar
• The Society for Thoracic Surgery. STS national database . Available at: http://www.sts.org/national-database. Accessed July 30, 2013
   Google Scholar
• American College of Cardiology. National cardiovascular data registry . Available at: https://www.ncdr.com/webncdr/. Accessed July 30, 2013
   Google Scholar
• American Board of Family Medicine. Initial certification/residency . Available at: https://www.theabfm.org/cert/index.aspx. Accessed July 30, 2013
   Google Scholar
• Fitzpatrick , R , Bowling , A , Gibbons , E , Haywood , K , Jenkinson , C Mackintosh , A . A structured review of patient-reported measures in relation to selected chronic conditions, perceptions of quality of care and career impact . Available at: http://phi.uhce.ox.ac.uk/. Accessed July 30, 2013
   Google Scholar
• National Health Service (UK). Health and Social Care Information Center. Patient reported outcome measures . Available at: http://www.hscic.gov.uk/proms. Accessed July 30, 2013
   Google Scholar
• Buckley , S , Coleman , J , Davison , I , Khan , K S , Zamora , J Malick , S . 2009 . The educational effects of portfolios on undergraduate student learning; a Best Evidence Medical Education (BEME) systematic review. BEME Guide 11 . Medical Teacher , 31 : 282 – 98 .
   PubMed Web of Science ®Google Scholar
• Holmboe , E S , Ward , D S , Reznick , R K , Katsufrakis , P J , Leslie , K M Patel , V L . 2011 . Faculty development in assessment: The missing link in competency-based medical education . Academic Medicine , 86 : 460 – 7 .
   PubMed Web of Science ®Google Scholar
• Kogan , J R , Conforti , L , Berbaneo , E , Iobst , W and Holmboe , E . 2011 . Opening the black box of clinical skills assessment via observation: A conceptual model . Medical Education , 45 : 1048 – 60 .
   PubMed Web of Science ®Google Scholar
• Dijkstra , J , Galbraith , R , Hodges , B D , McAvoy , P A , McCrorie , P Southgate , L J . 2012 . Expert validation of fit-for-purpose guidelines for designing programmes of assessment . BMC Medical Education , 12 : 20 doi: 10.1186/1472-6920-12-20
   PubMed Web of Science ®Google Scholar
• Dijkstra , J , van der vleuten , C MP and Schuwirth , L WT . 2009 . A new framework for designing programmes of assessment . Advances in Health Sciences Education: Theory and Practice , 15 : 379 – 93 .
   PubMed Web of Science ®Google Scholar
• Van der Vleuten , C PM , Schuwirth , L WT , Scheele , F , Driessen , E W and Hodges , B . 2010 . The assessment of professional competence: Building blocks for theory development . Best Practice & Research Clinical Obstetrics and Gynaecology , 24 : 703 – 19 .
   PubMed Web of Science ®Google Scholar
• Saedon , H , Salleh , S , Balakrishnan , A , Imray , C HE and Saedon , M . 2012 . The role of feedback in improving the effectiveness of workplace based assessments: A systematic review . BMC Medical Education , 12 : 25 doi: 10.1186/1472-6920-12–25
   PubMed Web of Science ®Google Scholar