Abstract
Background
Derived from multiple disciplines and established in industries outside of medicine, Implementation Science (IS) seeks to move evidence-based approaches into widespread use to enable improved outcomes to be realized as quickly as possible by as many as possible.
Methods
This review highlights selected IS theories and models, chosen based on the experience of the authors, that could be used to plan and deliver medical education activities to help learners better implement and sustain new knowledge and skills in their work settings.
Results
IS models, theories and approaches can help medical educators promote and determine their success in achieving desired learner outcomes. We discuss the importance of incorporating IS into the training of individuals, teams, and organizations, and employing IS across the medical education continuum. Challenges and specific strategies for the application of IS in educational settings are also discussed.
Conclusions
Utilizing IS in medical education can help us better achieve changes in competence, performance, and patient outcomes. IS should be incorporated into curricula across disciplines and across the continuum of medical education to facilitate implementation of learning. Educators should start by selecting, applying, and evaluating the teaching and patient care impact one or two IS strategies in their work.
1 July, 7:30 am. One of several newly-formed patient care teams is assembling to begin morning rounds. Dr. Lin, a newly-minted supervising physician, will be leading rounds for the first time. Dr. Rice, a second-year resident, and Dr. Martinez, a new intern, check the patient roster. Two medical students arrive. A new pharmacy student and a new nursing student join the team. The nursing student notices that not everyone washes their hands before starting rounds and thinks about how much trouble she would get in to if her supervisor saw her forget to do that.
The team members introduce themselves to each other. Dr. Lin mentions that the hospital was found to be out of compliance with surgical site infection rates. Teaching services have been targeted for their potential to impact a large number of patients. Dr. Lin has volunteered to serve on a committee to propose a plan for bringing the hospital into compliance. Dr. Rice asks: ‘What is the “differential diagnosis” of having too many infections in a hospital? How do you figure that out and fix it?’
Implementation science (IS) utilizes theories, models, principles, and methods derived from multiple disciplines and industries outside of medicine (e.g., organizational development, quality improvement, industrial engineering, business management) to ‘promote the systematic uptake of research findings and other evidence-based practices into routine practice … to improve the quality and effectiveness of health services’ (Citation1). By providing frameworks for translating and implementing new knowledge and skills in work settings, IS can help medical educators promote and measure behavior change in their learners. The integration of IS training across the spectrum of medical education can also provide assist in achieving milestones and competencies assessed by major accrediting agencies such as the Accreditation Council on Graduate Medical Education (ACGME) in their ‘Next Accreditation System (NAS) and Clinical Learning Environment Reviews (CLER)’ (Citation2) the Accreditation Council on Continuing Medical Education (ACCME) (Citation3), the American Board of Medical Specialties (ABMS) in their Maintenance of Certification (MOC) program, (Citation4) the Royal College of Physicians and Surgeons of Canada, and the General Medical Council in the United Kingdom (Citation5, Citation6).
This review highlights selected IS theories and models, chosen based on the experience of the authors, that could be used in planning and delivering medical education activities to help learners better implement and sustain effective education in their work settings. We discuss the necessity of using IS approaches for education at the level of the individual, team, and organization, as well as its integration across the medical education continuum from undergraduate and graduate training through continuing professional development. We also discuss the importance of determining whether IS enhances learning and leads to improved outcomes of care. We briefly identify potential barriers to incorporation of IS into medical training and identify potential next steps to implementation. Finally, we provide lists of Key Points and Strategies for moving this work forward.
Selected IS theories, models, and tools relevant to medical education
Most practicing physicians should be provided with tools and practical techniques to use in their work to identify and solve practice-based problems, work effectively with their teams, translate new knowledge and skills into practice, and implement continuous practice improvement in their organizational context. While we contend that medical educators will need firm foundational underpinnings in aspects of IS, most will not need to become ‘implementation scientists’ per se. However, medical educators need to understand and incorporate several different IS approaches in their educational design and delivery. This might mean working with others (quality improvement, clinical systems leadership) to identify and prioritize gaps in practice based on local contexts, anticipating or identifying barriers to implementation of learning (Citation7) and proactively addressing them as part of educational activities, and providing IS-based tools to facilitate learner practice change. While an exhaustive review of IS is beyond the scope of this paper, a few approaches pertinent to medical education at the level of the individual, team, and within organizational contexts are discussed.
IS to target change in individuals
Educational interventions are intended to improve the knowledge, competence, and/or performance of individual learners. highlights selected theories and models used in IS that are applicable at the level of the individual learner and gives examples of how they can be utilized in educational activities. Curriculum designers and educational researchers could collaborate to study the effect of incorporating these theories into educational activities on desired learner outcomes.
Table 1 Selected change theories models used in implementation science applicable to medical education at the level of the individual learner
IS targeting teams
The nursing student listens to the ward team's conversation. She doesn't understand all the terminology, but she does understand how things happen on the floor and sometimes is surprised by the way things are done or not done. Hand washing seems simple enough. Why doesn't everyone here do it? She has some ideas about how that could be changed, but is hesitant to raise the issue.
Whether in large organizations or small practices, health care is increasingly being delivered by interprofessional teams. Meaningful change (especially complex change) that results in improved health outcomes depends more on team improvement than changing individual physicians. As reflected by the opening vignette, our current health care system lacks both collaborative practice across health professions and a shared understanding of what this means. Team-based care is needed to meet competencies outlined in several frameworks: the Institute of Medicine (Citation16), CanMeds (Citation17), and the Scottish Doctor (Citation18). Because medical professionals learn primarily in silos, they are often ill equipped to work together to meet the needs of their patients and unprepared to function in teams. Reports by the Josiah Macy, Jr., Foundation (Citation19), the Interprofessional Education Collaborative Expert Panel (Citation20), and the European Interprofessional Education Network (Citation21) call for changes in medical education that emphasize interprofessional, team-based care, and community-based education for the entire continuum of medical education. Educators should develop educational interventions to help individuals function better as team members and help teams grow and improve together. Until recently, training on ‘how to be a team player’ has not been a routine part of medical education. Efforts are underway at the undergraduate health care level (Citation22) and in graduate medical education (Citation2) but are nascent for practicing professionals. The ACGME CLER visits specifically include assessments of faculty educational initiatives and faculty outcomes as a result of those new efforts (Citation23).
For individuals to function as a team, and for teams in general to be effective, several key competencies (Citation24, Citation25) should be developed (). A basic knowledge of team development (i.e., ‘Form, Storm, Norm, Perform’) (Citation26) could help educators tailor educational activities to the team's stage of function.
Table 2 Key team-based competencies for medical educators to target in their learners
IS targeting organizations
Dr. Lin considers the daunting task of providing appropriate oversight and teaching for her diverse team. She starts the morning by describing her training trajectory and disclosing that this is her first month in a new role. She asks the resident, the intern, the medical students, the nursing, and pharmacy students to describe their role, their expectations, their last rotation, a recent success, and their worst fear about this upcoming month. Everyone wonders if it is actually ‘safe’ to talk about their worst fears. Slowly, they begin to share with each other. The senior resident has never organized work rounds before. The intern feels ‘rusty’, having not done inpatient care for several months after less intense electives and vacation. The medical students need to hand in 10 patient write-ups that demonstrate applications of best evidence to patient management plans; they are not sure they know what that means. They also wonder who ‘all these people in different colored scrubs are’. The pharmacy student is not sure who they will be working with – the senior resident? The intern? The nurse? What is a nurse intern?
The first day is filled with patient care activities. That evening, Dr. Lin attends the first meeting of the committee that will work towards surgical site infection reduction. It feels like an unwieldy problem.
The commonly used educational planning sequence (problem identification/gap analysis, targeted needs assessment, determination of objectives, selection and delivery of appropriate educational formats, evaluation, and follow-up) (Citation27) align with several implementation strategies such as the Plan–Do–Study–Act cycle (Citation28), Rogers’ Diffusion of Innovations (Citation29), and Complex Adaptive Systems Theory (Citation30, Citation31). A previous publication (Citation32) describes how these models can be proactively used in developing educational interventions. In addition, Six Sigma (Citation33) and Lean Thinking (Citation34) can be applied to develop educational interventions to improve patient safety and promote efficiency in the delivery of health care. Examples exist utilizing a SOAP note format to guide clinicians accustomed to using this approach in patient care toward a broader, system-level view of needs for educational improvement (Citation35). shows how some of these constructs can be applied by the hospital to address the problem of surgical site infections in the vignette.
Table 3 Examples of implementation science models used within organizations
IS tools
IS uses tools from many fields and disciplines – a few that seem to hold promise for use in medical education include structured approaches to identifying and analyzing problems, methods of implementing evidence in the context of the team and organizational environment, and analytic techniques that evaluate and display changes over time (Citation36–Citation39). shows some of these tools, their application to medical education and examples of how they could be applied in the situation outlined in the vignette. Many of these tools can be used in mixed-methods evaluation of impact; qualitative methods in particular may be useful in identifying key factors that can assist in the translation of improvement between different contexts and settings.
Table 4 Examples of implementation science tools of potential use by medical educators
Incorporating IS along the continuum of medical education
Undergraduate and graduate medical training programs increasingly are required to develop curriculum and assessment metrics appropriate for the clinical learning environment (from academic medical center to community practice settings) in which trainees receive both the knowledge and experience of skill acquisition and meet specified milestones in these domains. ACGME CLER visits, for example, seek to specifically assess that environment on six axes (patient safety; health care quality and reduction in health care disparities; transitions in care; supervision; duty hours and fatigue management and mitigation; and professionalism) that reflect the culture of learning and patient care (Citation23). Active learning and continuous application of IS can assist in the achievement of these competencies and outcomes, to help students and residents transition to real-world practice prepared to make meaningful contributions to improve the processes and outcomes of care delivery. IS strategies can also help faculty adjust to the more safety and quality-focused expectations of both accreditation and credentialing/MOC bodies.
Introducing this material at the start of the undergraduate curriculum requires flexibility of approach but as shown with other new curricula, can be accomplished successfully. For example, the inclusion of medical error scenarios into a series of patient cases in a problem-based learning curriculum resulted in demonstrable improvement in learner knowledge of medical error genesis, failure modes, and root cause analysis principles (Citation42). Medical educators have devised ways to integrate other important themes, such as communication skills, cultural competence, and ethics into existing curricula. IS presents a rich educational opportunity – learners who understand IS principles can begin to utilize those principles to address both educational and clinical challenges in a unique and reinforcing way. Similarly, thoughtful approaches are needed to incorporate IS into continuing medical education activities for practicing physicians, to better equip them to work with their teams to care for populations of patients with increasingly complex and multifaceted problems, while attending to and improving preventive services. Accrediting, licensing, and credentialing bodies are calling for the creation of explicit goals and objectives for educational programs across the continuum and the provision of explicit pathways to competence so that learners demonstrate the ability to work in interdisciplinary teams, employ evidence-based practices, apply quality improvement concepts, and use high-quality medical informatics (Citation43). suggests IS competencies that the literature and the authors’ experience suggest are relevant to different learners and learner groups from trainees to experienced practitioners and from the individual through the team to the organization.
Table 5 Implementation science milestones relevant to different learners and learner groups
Community-based teaching settings
Students, residents, and other healthcare professionals are and will be increasingly learning in non-hospital practice settings (Citation47–Citation50). These sites constitute the overwhelming bulk of the medical landscape (Citation51); trainees will experience variability in the adoption and implementation of evidence-based and quality/outcomes-based practices in these more dispersed and de-centralized ‘real-world’ settings. Practicing physicians are required to evaluate their own practice and demonstrate evidence of continuous learning and improvement, ensuring that they are appropriately implementing the latest scientific principles to deliver safe, quality care, and improve patient outcomes (Citation4). Similar to institutionally-based medical educators, most community faculty will not need to become broad IS subject matter experts. However, they will be required to incorporate learnings about IS in their daily work, enabling them to critically evaluate and improve the care they provide, and using them in their MOC/Competence efforts. This will equip them to model and teach these approaches to their learners (Citation52).
At rounds the next morning, Dr. Lin describes the surgical site infection committee meeting. All agree that it will be a challenge to figure out why surgical site infection rates are high and how to lower them. The senior resident wonders how many people touch a patient each shift, how to study that, and whether he can use this as a research project.
Studying educational activities that incorporate IS
As described thus far, the theories and models of IS hold great promise for the evolution of medical education, but incorporating these approaches in our current system is no small undertaking. Educational and IS researchers must collaborate with educational activity planners to study whether adding IS approaches to education improves practitioner competence and performance, patient outcomes, and population health. While some of these changes can be assessed by self-report with commitment-to-change statements and barrier analysis assessed on immediate and follow-up education evaluation forms, we need additional outcomes-focused data to more fully assess educational outcomes.
Simulation-based training (including standardized patients) can be used to evaluate some competencies and individual and team performance changes in care processes after educational activities. Low-fidelity simulation (e.g., evolving patient scenarios) as well as high-fidelity simulation with mannequins and specialized venues may be useful, depending on the desired change and its complexity. McGaghie (Citation53) and others have shown that simulation strategies can result in improvements that translate from the simulation laboratory to the bedside. These educational interventions require input from multiple educational ‘team’ members as well as ‘care team’ member input in their design and execution (Citation53). However, simulation experiences (and application of other educational tools and constructs) under controlled and somewhat idealized circumstances may not always translate to the real world, thus underscoring the importance of follow-up assessment of effectiveness. Assessing the effect of implementation-focused, interprofessional team educational activities on care processes and outcomes and team functioning is also important, using or adapting existing team assessment tools or creating new validated instruments. Such tools could also identify gaps in team functioning that can be specifically addressed with follow-up professional development opportunities.
Barriers to incorporating IS into medical education
Across the spectrum (from medical school clinical experiences to GME to community practice settings), faculty and learners must be provided with timely, accurate, and regular data in order to identify areas for improvement and assess the progress toward their goals. User-friendly registries for data extraction from electronic health records, medical claims, and other data systems need to be accessible for use by teachers and learners in both institutional and community settings. IS must be supported and prioritized by faculty and institutional leaders, and achievement of IS competency by faculty and students must be made explicit and financially supported goals. Trainees must see IS applied in the real world of patient care or it will be devalued. This re-emphasizes the need to weave IS into curricula across the professional spectrum so that newly trained practitioners understand it, and practitioners can realistically experience its value. It also provides the opportunity to involve learners at all levels in solving real-world challenges together as they learn together.
Established faculty, physicians, and other health care givers have a strong tendency to maintain their current routines. Initiating an educational push for best practice implementation in undergraduate medical education and maintaining this in graduate medical education and clinical practice is necessary. Institutionalizing this new orientation in undergraduate and graduate medical education settings will require several steps. Finding a motivated cohort of early adopter educators who will be willing to champion change and equipping them with knowledge and skills to foster change across the continuum of medical education will be very important. These early adopters will help convince and teach other educators how to teach and incorporate IS into their work; thus, they will need to be located in the workplace where teaching occurs. Securing appropriate time for educator development and curricular time (in the face of competing demands) to incorporate IS competencies and securing funding and economic factors, including an understanding that implementation of best practices, will initially have a cost in time and dollars (Citation54, Citation55). Faculty development programs for physician educators must be developed, but this also means that these programs must be valued in terms of financial support and faculty promotion. Using IS strategies to help meet accreditation and credentialing requirements could help facilitate these changes in organizational culture.
Rounds are animated the next morning. Case presentations include some new questions: ‘How could this patient's problem have been prevented?’ ‘What are the risks to this patient during their hospitalization?’ ‘How can the team keep track of what everyone is doing?’ ‘How can rounds better take advantage of the skills and perspectives of different team members caring for these patients?’ ‘How many hands DO touch a patient each day?’
Conclusions and next steps
‘Knowledge is not enough – we must do’. – Goethe
The implicit goal in newer models of medical education that teach and incorporate IS is not just knowledge acquisition (which is necessary but not sufficient) but behavior change and improvement. One potential implication of these new benchmarks is the need to abandon potentially outmoded jargon. For example, while ‘learning objectives’ are supposed to be articulated in specific, actionable, behavioral, observable, implementable, and measurable terms, it is not uncommon for them to be teacher centric (‘what I am going to teach you’). Some now ask for ‘performance expectations’ instead of learning objectives, to explicitly focus attention on what we expect learners to do. Thinking beyond ‘what do we want learners to learn’ to ‘what's the problem we're trying to solve’ and ‘how will we know we've been successful in improving this problem’ will require a thoughtful integration of IS into medical education design and implementation. Imparting the knowledge, skills, and attitudes necessary to utilize IS into medical education holds promise for helping us better achieve changes in competence, performance, and patient outcomes; but we must be sure to thoroughly assess these changes to validate (or invalidate) these assumptions.
We believe that IS should be incorporated into curricula across the continuum (undergraduate, graduate, and continuous professional development level) to help provide tools and strategies for implementation of learning. All disciplines should ideally learn and use IS together, and the ability to use IS should be an explicit outcome of our teaching. We encourage educators, based on their local needs and priorities, to choose, apply, and evaluate one or two of the strategies outlined in this paper, working with others to assess their impact on teaching and on the outcomes of care.
20 May the following year.
Together, all members of the care team developed and refined a process and improved their adherence to a hand hygiene checklist. Hospital information technology placed this checklist into the electronic health record so that teams across the hospital can access and utilize it. Data is accumulating on the number of care teams accessing the checklist and their patient's rates of surgical site infections, and systems have been developed to regularly provide this data to the team. Individually, each team member has tried different types of reminders to improve their personal rates of hand washing. They also participated in hospital-sponsored seminars on teamwork and communication. They have gradually become more inclusive in their communication styles and have developed a comfort speaking up when problems arise. Hospital administration has reviewed the data and has noted that patients cared for by these individuals seem to have lower nosocomial infection rates.
The team members have a good sense of each other's strengths and weaknesses. Rounds have become more efficient as each team member knows what they are responsible for checking and reporting. As a team, they can often answer questions that might have waited for a different member to arrive, or they can quickly access the right member when specialized knowledge is necessary. When issues arise, they often use Plan–Do–Study–Act cycles to quickly make changes and determine their effectiveness. They have begun to think about solving problems in a fundamentally-different way.
The team is conducting pre-op bedside rounds on Mr. Davis, who is scheduled for a partial colon resection. The team proactively discusses ways to decrease Mr. Davis’ risk of post-op surgical site infection. Mr. Davis observes the collaborative nature of the discussion and is encouraged to ask questions. He enters his surgery impressed with and confident in the ability of his health care team.
Conflict of interest and funding
The authors have not received any funding or benefits from industry or elsewhere to produce this manuscript.
References
- Eccles M , Mittman BS . Welcome to implementation science. Implement Sci. 2006; 1: 1.
- Nasca TJ , Philibert I , Brigham T , Flynn TC . The next GME accreditation system – rationale and benefits. N Engl J Med. 2012; 366: 1051–6.
- Accreditation Council for Continuing Medical Education. Accreditation criteria. 2012. Available from: http://www.accme.org/requirements/accreditation-requirements-cme-providers/accreditation-criteria [cited 9 December 2014]..
- American Board of Medical Specialties. MOC competencies and criteria. 2014. Available from: http://www.abms.org/Maintenance_of_Certification/MOC_competencies.aspx [cited 9 December 2014]..
- Campbell CM , Parboosingh J . The Royal College experience and plans for the maintenance of certification program. J Contin Educ Health Prof. 2013; 33(Suppl 1): S36–47.
- Archer J , de Bere SR . The United Kingdom's experience with and future plans for revalidation. J Contin Educ Health Prof. 2013; 33(Suppl 1): S48–53.
- Price DW , Miller EK , Rahm AK , Brace NE , Larson RS . Assessment of barriers to changing practice as CME outcomes. J Contin Educ Health Prof. 2010; 30: 237–45.
- Ajzen I . The theory of planned behavior. Org Behav Hum Decis Process. 1991; 50: 179–211.
- Prochaska JO , DiClemente CC . Norcross JC , Goldfried MR . The transtheoretical approach. Handbook of psychotherapy integration. 2nd ed. 2005; New York: Oxford University Press. 147–51.
- Green LW , Kreuter MW . Health program planning: an educational and ecological approach. 4th ed. 2005; New York: McGraw-Hill Higher Education.
- Davis D , O'Brien MA , Freemantle N , Wolf FM , Mazmanian P , Taylor-Vaisey A . Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?. JAMA. 1999; 282: 867–74.
- Pathman DE , Konrad TR , Freed GL , Freeman VA , Koch GG . The awareness-to-adherence model of the steps to clinical guideline compliance. The case of pediatric vaccine recommendations. Med Care. 1996; 34: 873–89.
- Davis D , Evans M , Jadad A , Perrier L , Rath D , Ryan D , etal. The case for knowledge translation: shortening the journey from evidence to effect. BMJ. 2003; 327: 33–5.
- Regehr G , Norman GR . Issues in cognitive psychology: implications for professional education. Acad Med. 1996; 71: 988–1001.
- Lewin K . Defining the “field at a given time” Psychol Rev. 1943; 50: 292–310.
- Institute of Medicine. Health professions education: a bridge to quality. 2003; Washington, DC: National Academies Press.
- Frank JRE . The CanMEDS 2005 physician competency framework: better standards. Better physicians. 2005; Ottawa, ON: The Royal College of Physicians and Surgeons of Canada. Better care.
- The Scottish Doctor. Learning outcomes for the role of the doctor within the health service. 2011. Available from: http://www.scottishdoctor.org/node.asp?id=1100000000 [cited 9 December 2014]..
- Hager M , Russell S . Revisiting the medical school education mission at a time of expansion. 2009; New York: Josiah H. Macy, Jr. Foundation.
- Interprofessional Education Collaborative Expert Panel. Core competencies for interprofessional collaborative practice: report of an expert panel. 2011. Available from: https://www.aamc.org/download/186750/data/core_competencies.pdf [cited 9 October 2014]..
- Liaskos J , Frigas A , Antypas K , Zikos D , Diomidous M , Mantas J . Promoting interprofessional education in health sector within the European Interprofessional Education Network. Int J Med Inform. 2009; 78(Suppl 1): S43–7.
- Whitcomb ME , Association of American Medical Colleges . Educating doctors to provide high quality medical care: a vision for medical education in the United States; Report of the Ad Hoc Committee of Deans. Commissioned for the AAMC Institute for Improving Medical Education. 2004; Washington, DC: Association of American Medical Colleges.
- Weiss KB , Wagner R , Nasca TJ . Development, testing, and implementation of the ACGME Clinical Learning Environment Review (CLER) program. J Grad Med Educ. 2012; 4: 396–8.
- Baker DP , Salas E , King H , Battles J , Barach P . The role of teamwork in the professional education of physicians: current status and assessment recommendations. Jt Comm J Qual Patient Saf. 2005; 31: 185–202. [PubMed Abstract].
- Sargeant J , Loney E , Murphy G . Effective interprofessional teams: “contact is not enough” to build a team. J Contin Educ Health Prof. 2008; 28: 228–34.
- Tuckman BW , Jensen MAC . Stages of small group development revisited. Group Organ Stud. 1977; 2: 427.
- Kern DE , Thomas PA , Hughes MT . Curriculum development for medical education: a six-step approach. 2009; Baltimore, MD: The Johns Hopkins University Press. 2nd ed.
- Berwick DM . Developing and testing changes in delivery of care. Ann Intern Med. 1998; 128: 651–6.
- Rogers EM . Diffusion of innovations. 2003; New York: Free Press. 5th ed.
- Plsek PE , Greenhalgh T . Complexity science: the challenge of complexity in health care. BMJ. 2001; 323: 625–8.
- Sobo EJ , Bowman C , Gifford AL . Behind the scenes in health care improvement: the complex structures and emergent strategies of implementation science. Soc Sci Med. 2008; 67: 1530–40.
- Price D . Continuing medical education, quality improvement, and transfer of practice. Med Teach. 2005; 27: 259–68.
- Bandyopadhyay JK , Coppens K . Six Sigma approach to healthcare quality and productivity management. Int J Qual Prod Manag. 2005; 5: 1–13.
- Institute for Healthcare Improvement. Going lean in health care: IHI innovation series white paper. 2005. Available from: http://www.ihi.org/resources/Pages/IHIWhitePapers/GoingLeaninHealthCare.aspx [cited 9 December 2014]..
- Reznich CB , Wagner DP , Noel MM . A repurposed tool: the programme evaluation SOAP note. Med Educ. 2010; 44: 298–305.
- Lorenzi NM , Smith JB , Conner SR , Campion TR . The success factor profile for clinical computer innovation. Stud Health Technol Inform. 2004; 107(Pt 2): 1077–80. [PubMed Abstract].
- Anjard RP . Management and planning tools. Train Qual. 1995; 3: 34–7.
- White AA , Pichert JW , Bledsoe SH , Irwin C , Entman SS . Cause and effect analysis of closed claims in obstetrics and gynecology. Obstet Gynecol. 2005; 105(5 Pt 1): 1031–8.
- Chassin MR . Is health care ready for Six Sigma quality?. Milbank Q. 1998; 76: 565–91. [PubMed Abstract] [PubMed CentralFull Text] 510.
- Agency for Healthcare Research and Quality. TeamSTEPPS: national implementation. 2014. Available from: http://teamstepps.ahrq.gov [cited 9 December 2014]..
- Headrick LA , Baron RB , Pingleton SK , Skeff KM , Sklar DP , Varkey P , etal. Teaching for quality integrating quality improvement and patient safety across the continuum of medical education: report of an expert panel. 2013. Washington, DC: Association of American Medical Colleges.
- Wagner DP , Sparks HV , Fink G . Safety science meets basic science in problem-based learning. Med Educ. 2010; 44: 1125–6.
- Lucian Leape Institute. Unmet needs: teaching physicians to provide safe patient care. 2010; Boston, MA: Lucian Leape Institute. Report of the Lucian Leape Institute roundtable on reforming medical education.
- Barr H . Competent to collaborate: towards a competency-based model for interprofessional education. J Interprof Care. 1998; 12: 181–7.
- Sousa A , Wagner D , Rappley M . Michigan State University College of Human Medicine. Acad Med. 2010; 85(Suppl 9): S287–91.
- Accreditation Council for Graduate Medical Education. Common program requirements. 2014. Available from: https://www.acgme.org/acgmeweb/tabid/429/ProgramandInstitutionalAccreditation/CommonProgramRequirements.aspx [cited 9 December 2014]..
- Holmboe ES , Bowen JL , Green M , Gregg J , DiFrancesco L , Reynolds E , etal. Reforming internal medicine residency training. A report from the Society of General Internal Medicine's task force for residency reform. J Gen Intern Med. 2005; 20: 1165–72.
- Worley P , Esterman A , Prideaux D . Cohort study of examination performance of undergraduate medical students learning in community settings. BMJ. 2004; 328: 207–9.
- Kennedy C , Lilley P , Kiss L , Littvay L , Harden R . MEDINE2 Work Package 5: curriculum trends in medical education in Europe in the 21st century. 2013. Available from: http://medine2.com/Public/docs/outputs/wp5/DV5.18.1_Curriculum_Trends_Final_Report.pdf [cited 9 December 2014]..
- Colbert JA . Experiments in continuity – rethinking residency training in ambulatory care. N Engl J Med. 2013; 369: 504–5.
- Green LA , Fryer GE Jr. , Yawn BP , Lanier D , Dovey SM . The ecology of medical care revisited. N Engl J Med. 2001; 344: 2021–5.
- Steinert Y , Mann K , Centeno A , Dolmans D , Spencer J , Gelula M , etal. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach. 2006; 28: 497–526.
- McGaghie WC . Implementation science: addressing complexity in medical education. Med Teach. 2011; 33: 97–8.
- Fixsen DL , Naoom SF , Blase KA , Friedman RM , Wallace F . Implementation research: a synthesis of the literature. 2005; Tampa, FL: University of South Florida.
- Grol R , Wensing M . What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004; 180(Suppl 6): S57–60. [PubMed Abstract].