204
Views
4
CrossRef citations to date
0
Altmetric
Review

Clinical performance feedback and quality improvement opportunities for perioperative physicians

, &
Pages 115-123 | Published online: 03 May 2014

Abstract

Clinical performance feedback is an important component of the ongoing development and education of health care practitioners. For physicians, feedback about their clinical practice and outcomes is central to developing both confidence and competence at all stages of their medical careers. Cultural and financial infrastructures need to be in place, and the concept of feedback needs to be readily embraced and encouraged by clinical leadership and other stakeholders. The “buy-in” includes the expectation and view that feedback occurs on a routine basis, and those engaged in the process are both encouraged to participate and held accountable. Feedback must be part of an overarching quality improvement and physician education agenda; it is not meant to be an isolated, fragmented initiative that is typically undermined by lack of resources or systemic barriers to gaining improvement within programs. Effective feedback should be an integral part of clinical practice. Anesthesiologists and other perioperative physicians are identifying specialty-specific indicators that can be used when creating a broader quality improvement agenda. Placing a more immediate formal feedback strategy that focuses on goal-oriented behavior is rapidly becoming a mainstay. Physicians may use their individual feedback reports for reflection and designing personal development plans as lifelong learners and leaders in improving patient care.

Introduction

The paradigm of the 21st century health care system in the USA has shifted, focusing now on value-based driven care with an emphasis on high-quality delivery at lower costs. Health care organizations are tasked with improving the quality of patient care delivery, alongside the more conventional financial aspects of service performance.Citation1 In light of such demands, many health care systems aim to assess the professional performance of their health care workers, with a particular focus on physicians, who remain the primary decision-makers in the care of patients.Citation2 Recently, policy-makers have focused on how various clinical specialties measure and report on the quality of care delivered to patients. Although literature on the subject continues to be produced at an impressive rate, the role of performance feedback on physician improvement and strategies to further involve physicians remains elusive. Furthermore, there are few studies demonstrating positive outcomes, including increased patient satisfaction, improved health outcomes, and decreased costs.Citation3 Examining lessons learned and best practices among current feedback and assessment tools in other fields, however, allows stakeholders to have a better understanding of the importance and strategies of performance feedback.

Hospitals and practices striving to implement performance feedback must consider meaningful and ongoing physician involvement as a crucial cornerstone of care; the physician’s role in redesigning cost-effective quality systems is essential. Consequently, in shaping future delivery systems and payment models for perioperative care, it is imperative that perioperative physicians, including anesthesiologists, are fully engaged in the process.Citation4 Anesthesiologists understand the nuances behind perioperative care and are the health care leaders most appropriate and best positioned to identify quality indicators and their use in conjunction with other performance indicators.

As the USA and other countries move towards an even more data-driven health care medical system, it is critical to learn how best to use such data in the context of improving patient health care.Citation2 Similarly, the many metrics collected during the immediate postoperative period (eg, patient temperature, postoperative pain, nausea) and patient-reported quality of recovery are useful outcome measures, yet are not routinely reported back to clinicians.Citation5 Data collection is meaningless if not used in a constructive manner.

Performance measurement and feedback is most effective when integrated within a broader quality improvement agenda.Citation6,Citation7 Similarly, anesthesiologists and other perioperative physicians are not the only health care professionals who must change and adapt to new roles and expectations when provided with such feedback. For example, anesthesiologists work in the center of a care team and provide collaborative care through a shared decision-making process with surgeons, nurses, and ancillary professional staff members. All members of the care team should be receptive and participatory in creating a culture of transparency and feedback. In performance feedback, it is important to understand that the indirect, albeit most important, beneficiaries of physician improvement are the patients, who remain active participants in the feedback process. The multisource feedback methodCitation8 that includes evaluation from patients, peers, and nonphysician workers seems to be the most ideal format for performance assessment in anesthesiologists.

Central to patient satisfaction and the views of evidence-based quality health care, a more robust emphasis should be placed on establishing a formative evaluation process; in this evolution, such assessments should cater for the anesthesiologist, with eventual creation of a valid feedback instrument such as a physician’s individual “report card”. Such a report card should include constant, direct, and timely feedback from quality indicators that are provided to gauge an anesthesiologist’s progress and should ideally include patient-reported, peer-reported, and supervisor-reported measures to target areas for improvement. Embracing these concepts and implementing strategies that reflect the need to improve will aid anesthesiologists in creating both an economically sustainable and high-quality perioperative care system.

In this paper, we aim to provide a narrative overview of the relevant research and evidence supporting the use of quality indicators in medicine, with the focus on perioperative specialties such as anesthesiology. We delve into the concept of performance feedback, highlighting the evidence and identifying how it can be used in a broader quality improvement initiative and as an educational tool. We provide examples of how the principles outlined may be implemented in practice through a discussion of the concept of performance feedback initiatives in the anesthesiology department of a large tertiary care academic hospital. Using the anesthesiologist as an example, this article provides more context on the importance of feeding data back to physicians to improve care, establishes a framework for future research and implementation strategies on performance measurement and feedback for anesthesiologists and other perioperative physicians, and discusses the wider implications for such quality improvement initiatives in and beyond the specialty of anesthesiology.

Tale of medical specialty at the forefront of US health policy

The Institute of Medicine report entitled “To err is human” recognized anesthesia as one of the few health care disciplines that has taken effective action to reduce medical error and improve patient safety.Citation9,Citation10 Yet this was not always the case. The post-World War II era not only signaled economic expansion in the USA, but was also a period when the public began to examine patient mortality more closely. At the time, anesthesiologists took it upon themselves as a medical profession to examine this issue more closely. By the height of the 1970s, it was widely known to the public and the medical community that, as a medical specialty, anesthesiology was at the forefront of patient safety. Oft times, anesthesiology is cited as the medical specialty where patient safety is the central focus that ultimately led to a dramatic improvement in mortality and morbidity rates over the last two decades. While technological advances have certainly played a large role, active engagement of anesthesiologists combined with evolution of their role as perioperative physicians has proven to be of great value in promoting health quality.

The Institute of Medicine report “Crossing the quality chasm: a new health system for the 21st century”, a follow-up to its seminal treatise, further highlighted the importance of patient safety,Citation11 called for a change to the health care system processes to improve the level of quality,Citation12 and explored ways in which such change can be implemented in the health care system. These two significant reports concluded that safety is a quality problem. Furthermore, largely due to burgeoning health care costs that have proven economically unsustainable, the political paradigm and national discourse has shifted to improving quality outcomes while decreasing health care costs. Quality problems are best approached using performance measurements to promote and incentivize improvement.Citation4 Physician performance has emerged as yet another metric aimed at improving health care outcomes and if systematically conducted can help to constrain health care costs.Citation2,Citation3

It is worth noting that while anesthesiologists were once considered the leaders in patient safety, the specialty has largely been absent from the conversation on quality improvement. Perhaps equally as important, anesthesiologists have remained on the proverbial sidelines,Citation13 rather than leading efforts that will dramatically impact payment models and enable better health outcomes.

This process has started to change now that the predominant organization for anesthesiologists in the USA, the American Society of Anesthesiologists, has recognized the importance of engaging the profession at the forefront of such quality improvement initiatives. Through the Anesthesia Quality Institute and the National Anesthesia Clinical Outcomes Registry, the American Society of Anesthesiologists has created a framework for quality benchmarking and outcomes improvement in anesthesiology.Citation2 Assessment of performance is now a core component of anesthesiologist credentialing through the American Board of Anesthesiology.Citation2

In summary, effective quality control requires the specialty to not only develop reliable data collection mechanisms but also put in place systems and processes for effective feedback and use of the data to support quality improvement.Citation14

Quality metrics

Before discussing ways in which feeding back data can be used to improve quality of care, it is important to discuss how quality is defined and measured. Most experts agree with the Institute of Medicine’s definition of quality as “the degree to which health services for individuals and populations increases the likelihood of desired health outcome and are consistent with current professional knowledge”.Citation1

Modern day quality improvement initiatives benefit from the earlier work of the Donabedian structure–process–outcome conceptual model, which centers on the relationship and proximity of a variable to the desired performance result.Citation15 While a comprehensive primer and broader overview of the rationale and science of performance measurement is beyond the scope of the present paper and has been well reported elsewhere,Citation2 it is worth mentioning a few topics to provide the framework for this discussion.

The predominant model of data use within the quality improvement strategy is based on process measures. Process measures describe what is done to patients, eg, administering antibiotic prophylaxis within one hour of surgical incision.Citation2 Care processes are monitored continuously for process changes that are rapidly detectable.Citation7 Unlike outcome measures, process measures are directly actionable, do not require risk adjustments when the target population is appropriately selected, and do not measure quality for most activities.Citation2 From an intervention and education standpoint, it is generally easier to target process measures instead of outcome measures.

Next, there are structure measures that reflect the properties of the hospital setting, and typically include organizational characteristics, human resources, and technology, eg, adequacy of facilities, qualifications of medical staff, procedure volume, health information technology.Citation16,Citation17 Structure measures are relatively inexpensive and easy to collect, and procedure volume, especially with certain operations, has a strong correlation with outcome.Citation18 These structure measures do not assess the performance of individual physicians and while considered especially important by regulatory agencies and third-party payers, particularly as a way to incentivize adoption of organizational processes considered best practices, they will not be discussed further.Citation2,Citation9

Lastly, there are outcome measures, eg, postoperative nausea/vomiting and complications, that represent “the common product of all clinical activity and the cornerstone of performance measurement”.Citation2 Performance measurement is associated with improved outcomes. The type of data (less expensive, less accurate administrative data versus the more expensive, more accurate clinical data), data quality, risk adjustment, and sample size or frequency of the outcome are some of the key factors affecting how outcome measures can be used, particularly when measuring physician performance. There does seem to be a strong association between outcomes feedback and improved population outcomes. Outcomes did improve when physicians received feedback specifically on outcome data.Citation2

Concept of performance feedback

Before defining the key term of this paper, it is worth distinguishing feedback from commonly interchangeable yet very distinct terms, such as encouragement and evaluation. Encouragement (eg, “great job”) is purely supportive and does very little to improve an individual’s skills. Evaluation is often summative and represents the final judgment of an individual’s performance.Citation19 Perhaps the most distinguishing characteristic among these similar terms centers on the inherent design of feedback, ie, to improve future performance.

Feedback is often described as the act of providing knowledge of the results of behavior or performance to the individual.Citation20 Within health care, feedback is “any summary of clinical performance of health care over a specified period of time, given in a written, electronic or verbal format”.Citation7,Citation21 However, feedback is simply more than just providing information, and must incorporate some action to close the identified gap and promote improvement.Citation22 Another definition that reflects these points and is the one preferred for the purposes of this paper is “feedback is an informed, nonevaluative, objective appraisal of performance intended to improve clinical skills”.Citation23,Citation24

From an organizational standpoint, feedback over time can be an important mechanism for organizational learning resulting in both “incremental and large-scale modification to care systems and processes” over time.Citation2

Performance feedback in the medical community

There is a wide variety of stakeholders in the health care system focused on improving physician performance (eg, medical groups, professional medical societies, purchases/employers, private sector companies, health plans, and the federal government). The multiple efforts aimed at reporting performance data to individual physicians vary in terms of objectives. However, these stakeholders share the underlying goal of making physicians aware of their performance and encouraging improvement in specific aspects of care delivery, such as clinical quality, patient experience, patient safety, and resource use, when performance is lacking.Citation6

Physician-level performance measurement and feedback is a relatively new concept. The experiences that organizations or even specialties within medicine have had in measuring performance and providing feedback to individual physicians are fairly limited.Citation6 Important lessons learned in applied settings often remain unknown to others with an interest in using physician performance results to change physician behavior.

While performance measurement is an imperfect science, it is undoubtedly becoming a mainstay in the national quality improvement agenda. Future research is required to determine how best to harness the potential role it has in promoting and incentivizing improvement in the quality of health care.

Effectiveness of feedback performance

While feedback performance is reported to make a moderate difference in outcome measures depending on the overall quality improvement agenda incorporated, the evidence for its effectiveness is limited at best. Unfortunately, there are few randomized controlled studies beyond anecdotal examples that validate the use of this tool even though it has been extensively researched. Still, it warrants further discussion. Three research studies are examined to explore this topic.

Over 25 years ago, Tierney et al conducted a randomized controlled trial to compare the effects of supplying monthly feedback reports of compliance with preventive care protocols by 135 internal medicine house staff with the effects of providing specific reminders at the time of patient visits.Citation25 The study concluded that physician compliance with suggested preventive care protocols can be increased by both delayed feedback and immediate reminders. The data also strongly suggested that feedback reports are effective strategies to promote better and more responsible quality care.

The next study examined incorporated benchmarking to increase the overall effectiveness of performance feedback to clinicians. This 2001 study, which was part of the Ambulatory Care Quality Improvement Project in Alabama evaluated the effectiveness of using achievable benchmarks to enhance typical physician performance and improve care.Citation26 In this prospective cohort study, physicians were randomly assigned to receive a multimodal improvement intervention that included physician-specific feedback or an identical intervention plus achievable benchmark feedback.Citation27 The main outcome measures were changes in the proportion of patients receiving routine care (ie, a diabetic having a foot examination), vaccination (eg, influenza vaccine), and blood tests (eg, glucose control, and cholesterol and triglyceride levels) when compared between the two groups. Overall, the results demonstrated that use of achievable benchmarks significantly enhances the effectiveness of physician performance feedback in the setting of multimodal quality improvement intervention. It is important to recognize that audit and feedback methods, whereby clinicians receive reports of their performance are compared with the mean performance of a peer group, have been used and studied extensively but few of these studies have been in the form of randomized controlled trials. Still, clinicians care about their performance especially relative to their peers. The underlying theory is that viewing personal performance within the context of peer performance is a powerful motivator for change. However, there are only modest benefits with limited long-term sustainability. Hence, seeking a method to increase the effectiveness of using performance feedback for anesthesiologists may require an achievable benchmark method that can be developed from quality indicators, much like the ones currently being developed by the Anesthesia Quality Institute.Citation28

The concept of feedback performance is not unique to the field of medicine. From the aviation industry to Wall Street, feedback performance is an important part of quality improvement. A comprehensive literature search was done reviewing various types of feedback interventions since the early 1900s. Many interesting facts were noted, including the inconsistent results concerning the effectiveness of feedback. Some early experiments found that feedback improved performance for some indicators and impaired performance for others. In the 1950s, after an extensive review of the literature, Ammons, a pre-eminent scholar on feedback, concluded that feedback increased learning and motivation.Citation30,Citation31 His studies were not met without controversy, as it was later discovered that he failed to report the findings that contradicted his conclusions. Still, the positive effects of feedback on performance became one of the widely accepted principles in psychology. Next, over 131 papers were reviewed for meta-analysis and the results indicated a modest but positive effect of feedback on performance overall (less than one-half of one standard deviation improvement in performance), but 38% of feedback effects were negative, giving a very inconsistent picture of feedback effectiveness.Citation19 summarizes a few major papers addressing performance feedback.

Table 1 Summary of major findings for effectiveness of feedback and performance measurement

Barriers and quality indicators

Feeding back data in anesthesiology and other perioperative specialties is complex. It is difficult to ascertain which process or outcome measures are anesthesia-related. Optimal care of the surgical patient requires a multidisciplinary team approach consisting of the unique integration of anesthesiologists, surgeons, other proceduralists, and nurses. In a systematic review of the anesthesiology literature, 108 quality indicators were identified; 57% were outcome measures, 42% were process measures, and 1% involved structure of care.Citation4 Despite this, there is no validated algorithm to identify anesthesia-related outcomes.Citation2 Ideally, quality indicators should be transparent, reliable, evidenced-based, measurable, and improvable.Citation7 The quality of feedback is predicated on this notion.

Hesketh et al identified a number of barriers to giving effective feedback. These include feedback that is listed as too generalized and not related to specific facts or observations, inconsistent feedback from multiple sources, feedback that is not actionable (ie, not providing guidance on how to rectify behavior), and a lack of trust or respect for the source of feedback.Citation33 Physicians are more influenced by an authoritative, credible source.Citation34 Van der Veer et al identified lack of trust in data quality, lack of intensity of feedback, and lack of motivation.Citation24,Citation27

Perioperative outcomes in anesthesia

Mortality due to anesthesia is no longer the pervading outcome it once was over 40 years ago. Risks of perioperative mortality related to anesthesia are now less than an estimated 0.2% in healthy patients.Citation13,Citation36 Mortality itself is now considered a poor quality indicator because it is rare and often due to other factors beyond an anesthesiologist’s care.Citation7 Still, there are other routine outcome measures affected by variation in anesthesia care worth considering. Contemporary medicine has widely demonstrated that patient outcomes are multifactorial. Outcomes are not solely a result of the effectiveness of medical care but are also affected by a complex interplay between patient’s risk factors and random events.Citation37 Complications such as patient awareness under general anesthesia, neurological deficit after regional anesthesia, or even an epidural abscess after neuroaxial anesthesia may be more appropriately linked to anesthesia. Risk adjustment, the concept of not penalizing hospitals and physicians who treat high-risk patients,Citation38 is an important factor affecting outcome measures. When generating unbiased, constructive quality reports from the variable nature of patient outcomes, risk adjustment plays a major role.

Postoperative recovery and patient satisfaction

Variation in the quality of anesthesiology care may be more directly assessed in the postoperative period.Citation7 A patient’s recovery in the postoperative care unit is closely linked to the quality of anesthetic technique and medications given to minimize postoperative issues, such as nausea and vomiting and pain, two of the most important factors affecting patient’s experience, quality of recovery, and overall satisfaction with care.Citation8,Citation39 Postoperative nausea and vomiting and pain have a strong negative effect on patient satisfaction.Citation23 These two undesirable complications represent important quality indicators for the anesthesiologist. Pain can be measured using a variety of scales (eg, visual analog scale, numerical rating scale) that can be relayed back to the anesthesiologist. The nursing staff in the post anesthesia care unit monitors other quality measures that are elements of the patient’s experience. Data such as temperature upon arrival, time spent in the recovery room, and if a day surgery patient, the time to void and ambulate are quantifiable and can be used in the feedback process.Citation7

Patient satisfaction surveys involve quantification of a patient’s subjective perceptions. There are a number of surveys that have been developed and validated, and more importantly, can be used as part of the anesthesiologist’s performance profile. The American Society of Anesthesiologists is currently working to define valid, reliable, and useful clinical quality indicators for anesthesiologists. Identifying the appropriate process and outcome measures is the first requisite. Afterwards, the data can be turned into information that is both useful and actionable. Later, this can be implemented through a feedback mechanism.

There are various feedback strategies that yield a range of results. The evidence overall suggests that providing feedback results in small to moderate positive effects on clinical practice.Citation7 There are several possible factors that help with providing feedback. A study by Van der Veer et al found that success factors included timely dissemination of feedback from data collection and presenting information in a confidential and nonjudgmental manner.Citation24 Mugford et al showed that information feedback can influence clinical practice if the information was presented close to the time of decision-making and particularly when physicians had agreed beforehand to review their practice.Citation35 Jamtvedt et al identified source and duration of feedback as important characteristics. Parsloe et al identified that feedback must be given sensitively and appropriately.Citation40,Citation41 This study also commented on other aspects involving the person giving the feedback and the recipient, including differences in sex, age, and educational or cultural background.Citation27 These may not be viewed as obstacles, but can make feedback sessions strained and demotivating. De Vos et al established that feedback reports within an educational implementation strategy were most effective.Citation42 Barriers to quality improvement identified were lack of credible data, lack of supportive local management, and lack of hospital resources.Citation43 Effective feedback is continuous, timely, credible, nonpunitive, and supportive of remedial action.Citation44

Case example: monitoring and feedback in anesthesiology

Performance measurement and feedback has broader implications for the future of medicine. In particular, timely integration of feedback facilitates early and sustainable adoption of quality improvement initiatives. Moreover, integrating a systematic feedback process earlier on in medical training will likely encourage a culture where feedback is routine and part of the norm. While the direct educational merit of performance feedback within residency training programs is outside the scope of this paper, it is worth discussing, giving the example provided from a large tertiary academic medical center.

The anesthesiology residency program at our institution, Brigham and Women’s Hospital, comprises approximately 90 residents, 30 fellows, and 110 faculty members. In efforts to increase the evaluation process and provide residents with a more meaningful and comprehensive feedback tool, the department recently implemented a multisource feedback program for residents. Multiple international studies have used multisource feedback or 360 degree evaluation, which is an increasingly popular way of assessing multiple components of professional performance.Citation35 This rather reliable and valid feedback method uses external evaluation of physician performance and focuses on various tasks by involving: supervisors (such as faculty); peers with knowledge and experience of scope of practice; nonphysician coworkers (nurses, allied health care professionals, or administrative staff); and patients.Citation35,Citation36 Raters in each category who observed the physicians’ behavior are asked to answer questions about the physicians’ performance and this was compared with the physician’s self-evaluation to examine directions for change.Citation36

Although on the surface the decisions of key people on the higher rungs of the ladder in medicine make the more directed decisions of a person’s care, medicine has always been and is increasingly becoming team-based. For a large organization, such as a medical center or even a community hospital that sees multiple people from different departments playing a variety of different roles, the 360-degree performance appraisal may be beneficial and a reinforcement of this team model concept.

This multisource method gathers anonymous information from peer residents (and/or colleagues), ancillary staff, nurses, social workers, and anyone who has the opportunity to interact or observe the individual on a regular basis. Ideally, this should supplement and better inform a superior’s own observations. This should be a fair, consistent, and, most importantly, a private affair to ensure the integrity of the process. The supervisor (program director for residents, chair of the department, or medical director for attending) should directly ask the individual who is the source of the evaluation if clarifications of comments are needed, especially in situations where improvement is needed.

While it is both practical and logical to tie financial incentives to those for exemplary performance reviews, the constraints within an academic center may make this very difficult to achieve. Awards and recognition programs are a good way to not only boost department morale but also entice most if not all staff members to participate.Citation6

The future: integrating feedback platform into practice

Feedback is an important component of the ongoing development of practitioners in health care. For physicians in particular, feedback is central to developing both confidence and competence at all stages of their medical careers. Clinical practice, professional behaviors, and attitudes are routinely assessed using a variety of workplace instruments (eg, multisource feedback, case-based discussions, such as mortality and morbidity conferences). An example of this was discussed earlier in a case example from a residency training program in a tertiary center.

There are a few goals that need to be achieved when developing and implementing feedback initiatives, including solid cultural and financial infrastructures. Indeed, the concept of feedback needs to be readily embraced and encouraged by senior members of the clinical department and other stakeholders in the hospital at large who feel that this initiative is important. The “buy-in” includes the expectation and view that feedback occurs on a routine basis and those engaged in the process are both encouraged to participate and be held accountable. A warranted concern from those giving feedback is the potential negative consequences resulting from providing constructive feedback, fearing the relationship with others may be damaged.Citation38

Feedback must be part of an overarching quality improvement agenda; it is not meant to be an isolated, fragmented initiative that is typically undermined by lack of resources or systemic barriers to improvement within programs (). In addition to the financial investment and human resources required, the issue of inadequate training can be addressed by faculty development activities that allow participants to try out new skills, receive feedback on their performance, and network with peers to exchange ideas and best practices. Lack of training can also be addressed with self-study around the topic of feedback. Establishing feedback as a regular and recurring event focused on increasing patient satisfaction and outcomes may prevent defensive reactions on the part of opponents.

Table 2 Major premises of integrating feedback and establishing a culture for physician improvement

Effective feedback should be an integral part of clinical practice. While barriers to feedback for anesthesiologists and other perioperative physicians exist, these concerns can and will be addressed. The American Society of Anesthesiologists and other key stakeholders are identifying specialty specific indicators that can be used when creating a broader quality improvement agenda. Placing a more immediate, formal feedback strategy that focuses on goal-oriented behavior is rapidly becoming a mainstay. Ultimately, this will improve patient care. In the future, as a summative performance assessment report is created, it is envisioned that physicians may use their individual feedback information for reflection and designing personal development plans as lifelong learners and leaders in improving patient care.

Disclosure

The authors report no conflicts of interest in this work.

References

  • JencksSFWilenskyGRThe health care quality improvement initiative: a new approach to quality assurance in MedicareJAMA19922689009031640620
  • Moya-RuizCPeiróSMeneuREffectiveness of feedback to physicians in reducing inappropriate use of hospitalization: a study in a Spanish hospitalInt J Quality Health Care200214205312
  • OxmanADThomsonMADavisDAHaynesRBNo magic bullets: a systematic review of 102 trials of interventions to improve professional practiceCMAJ1995153142314317585368
  • GlanceLGFleisherLAAnesthesiology and health care reform: a call to action Manuscript draft
  • BennJArnoldGWeiIRileyCAlevaFUsing quality indicators in anaesthesia: feeding back data to improve careBr J Anaesth2012109809122661749
  • KiefeCIWoolleyTWAllisonJJBoxJBCraigASDetermining benchmarks: a data driven search for the best achievable performanceClin Perform Qual Health Care19942190194
  • KiefeCIAllisonJJWilliamsODImproving quality improvement using achievable benchmarks for physician feedbackJAMA20012852871287911401608
  • EndeJFeedback in clinical medical educationJAMA19832507777816876333
  • Institute of MedicineCrossing the quality chasm: a new health system for the twenty-first centuryWashington, DC, USANational Academy Press2001 Available from: http://www.iom.edu/∼/media/Files/Report%20Files/2001/Crossing-the-Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdfAccessed March 28, 2013
  • ChassinMRGalvinRWNational Roundtable on Health Care QualityThe urgent need to improve health care qualityJAMA1998280100010059749483
  • GlanceLGNeumanMMartinezEAPaukerKYDuttonRPPerformance measurement at a “tipping point”Anesth Analg201111295896621385976
  • TelekSSShawRDambergCLMcGlynnEAProviding performance feedback to individual physicians: current practice and emerging lessons: final reportSanta Monica, CA, USARand Corporation2006 Available from: http://www.rand.org/pubs/working_papers/WR381Accessed March 30, 2014
  • DonabedianACriteria, norms and standards of quality: what do they mean?Am J Public Health1981714094127468883
  • DonabedianAEvaluating the quality of medical care: 1966Milbank Q200511310041006
  • ShuteVJFocus on formative feedbackRev Educ Res200878153189
  • SteinerDLNormanGRHealth Measurement Scales: A Practical Guide to their Development and use4th edOxford, UKOxford University Press2008
  • FindlayGGoodwinAProtopapaKSmithNKnowing the Risk: A Review of the Perioperative Care of Surgical PatientsLondon, UKNational Confidential Enquiry into Patient Outcome and Death2011
  • HallerGStoelwinderJMylesPMcNeilJQuality and safety indicators in anesthesia: a systematic reviewAnesthesiology20091101158117519352148
  • HeskethEALaidlawJMDeveloping the teaching instinct. 1: FeedbackMed Teach20022424524812098410
  • FrankJRDanoffDThe CanMEDS initiative: implementing an outcomes-based framework of physician competenciesMed Teach20072964264718236250
  • HutchinsonAYoungTACooperKLTrends in healthcare incident reporting and relationship to safety and quality data in acute hospitals: results from the National Reporting and Learning SystemQual Saf Health Care20091851019204125
  • IezzoniLIReasons for risk adjustmentIezzoniLIRisk AdjustmentChicago, IL, USAHealth Administration Press2003
  • RamaprasadAOn the definition of feedbackBehav Sci198328413
  • van der VeerSde KeizerNRavelliATenkinkSJagerKImproving quality of care. A systematic review of how medical registries provide information feedback to health care providersInt J Med Inform20107930532320189451
  • TierneyWMHuiSLMcDonaldCJDelayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician complianceMed Care1986246596663736141
  • HayesRPBallardDJReview: feedback about practice patterns for measurable improvements in quality of care – a challenge for PROs under the Health Care Quality Improvement ProgramClin Perform Qual Health Care19953152210141395
  • AllisonJJKiefeCIWeissmanNWCan data-driven benchmarks be used to set the goals of Healthy People 2010?Am J Public Health19998961659987466
  • Anesthesia Quality Institute Available from: https://www.aqihq.org/Accessed April 14, 2014
  • OvereemKDoctor performance assessment: development and impact of a new systemPerspect Med Educ2012129810023316465
  • AmmonsRBEffects of knowledge of performance: a survey and tentative theoretical formulationJ General Psychology195654279299
  • DeNisiASAvrahamNKFeedback effectiveness: can 360-degree appraisals be improved?Academy of Management Perspectives200014129139
  • VeloskiJBoexJRGrasbergerMJEvansAWolfsonDBSystematic review of the literature on assessment feedback and physicians’ clinical performance: BEME Guide No 7Med Teach20062811712816707292
  • HeskethEAAndersonFBagnallGMDriverCPJohnstonDAMarshallDNeedhamGOrrGWalkerKUsing a 360 degrees diagnostic screening tool to provide an evidence trail of junior doctor performance throughout their first postgraduate yearMed Teach200527321923316011945
  • JamtvedtGYoungJMKristoffersenDTThomson O’BrienMAOxmanADAudit and feedback: effects on professional practice and health care outcomesCochrane Database Syst Rev20062CD00025916625533
  • MugfordMBanfieldPO’HanlonMEffects of feedback of information on clinical practice: a reviewBMJ19913033984021912809
  • IezonniLIThe risks of risk adjustmentJAMA1997278160016079370507
  • WhittyPShawIGoodwinDPatient satisfaction with general anaesthesiaAnaesthesia1996513273328686818
  • HeideggerTHusemannYNueblingMPatient satisfaction with anesthesia care: development of a psychometric questionnaire and benchmarking among six hospitals in Switzerland and AustriaBr J Anesthesia200289863872
  • GiganteJDellMSharkeyAGetting beyond “good job”: how to give effective feedbackPediatrics201112720520721242222
  • ParsloeEWrayMCoaching and MentoringLondon, UKKogan Page2000
  • ParsloeECoaching, Mentoring and AssessingLondon, UKKogan Page1995
  • De VosMGraafmansWKooistraMMeijboomBVan Der VoortPWestertGUsing quality indicators to improve hospital care: a review of the literatureInt J Qual Health Care20092111912919155288
  • BradleyEHHolmboeESMatteraJARoumanisSARadfordMJKrumholzHMData feedback efforts in quality improvement: lessons learned from US hospitalsQual Saf Health Care200413263114757796
  • OvereemKWollersheimHCArahOACruijsbergJKGrolRPLombartsKMEvaluation of physicians’ professional performance: an iterative development and validation study of multisource feedback instrumentsBMC Health Serv Res2012128022448816