http://orcid.org/0000-0003-1528-546X
Abstract
Phenomenon: Rising healthcare expenditures threaten the accessibility and affordability of healthcare systems. Research has demonstrated that teaching (junior) physicians to deliver high-value, cost-conscious care can be effective when learning is situated in a supportive environment. This study aims to offer insight into how residents learn to provide high-value, cost-conscious care in the workplace and how the postgraduate training environment influences this learning. Approach: Six homogeneous focus groups were held between August 2015 and July 2016 with 36 residents from six residency programs (dermatology, n = 5; elderly care, n = 8; family medicine, n = 5; internal medicine, n = 6; orthopedic surgery, n = 6; surgery, n = 6). An iterative grounded theory approach was used to analyze the qualitative data. Findings: Influential factors in learning of high-value, cost-conscious care delivery operated on three levels: individual resident, training program, and the workplace. On the individual level, we discerned three types of beliefs regarding HV3C. At the training program level, perceived determinants of learning included resident–supervisor interactions, involvement in decision-making over time, and exposure to variation in care delivery. At the workplace level, learning depended on the availability of professional healthcare expertise and the presence of institutional policy. Insights: Residents struggle to seize high-value, cost-conscious care learning opportunities in the workplace setting. Both residency training programs and workplaces can contribute to creating these learning opportunities. An important starting point is being aware of the different personal beliefs of residents and the approaches to high-value, cost-conscious care on the level of the training program and workplace.
Introduction
Mounting concerns over rising expenditures in healthcare, jeopardizing its sustainability, affordability, and accessibility, have galvanized insurance companies, governments, and hospital boards into curbing healthcare expenditures. In this endeavor, eliminating waste is expected to yield significant gains, as an estimated 20% of healthcare delivery can be considered wasteful.Citation1–3 Physicians, who are involved in a large variety of healthcare decisions, are thought to control 80% of health expenditures,Citation4 making their role in achieving waste reduction a subject of keen interest.Citation5–7 One potential strategy to reduce waste is to train physicians to deliver high-value, cost-conscious care (HV3C). In the last 5 years, the medical education realm has therefore emphasized the use of educational interventions designed to achieve this purpose.Citation8–10 This focus has led to the launch of successful initiatives such as “Choosing Wisely,” the “Top Five” list,Citation11,Citation12 and the American College of Physicians’ “High-Value Care initiative”Citation13 aimed to train residents and physicians in order to simultaneously improve quality and eliminate healthcare waste. However, the effectiveness of these initiatives from an educational perspective has not been researched. A review of the “working ingredients” of educational interventions that aim to train physicians in the delivery of HV3C demonstrates that knowledge transmission, reflective practice, and a supportive environment are three important pillars of effective educational interventions.Citation14
Despite the intentions of directors to offer HV3C training, incorporating it in postgraduate medical training programs progresses slowlyCitation15,Citation16 due to the complexity of designing training aimed at effective knowledge transmission and reflective practice within an unsupportive environment.Citation14,Citation17,Citation18 A supportive environment is essential, as the hidden curriculum is known to hamper successful transfer of known skills into the workplace.Citation19 Informal education matters, as residents spend most of their training in clinical settings and messages in the hidden curriculum can be far more powerful than the lessons taught by formal curricula.Citation19,Citation20 Without the support of informal education, formal educational interventions might miss the opportunity to effect sustained/internalized behavioral change.Citation21 The purpose of this study is therefore to offer insight into (a) how residents learn to provide HV3C in the workplace and (b) how their workplace environment influences this learning. Our findings can complement existing literature about workplace-based learning and enhance learning of HV3C delivery during residency.
Method
Study design
Based on a literature review of educational interventions to promote HV3C delivery,Citation14 we developed a semistructured discussion guide designed to elicit the views of participants in focus groups (Appendix A). We invited participants to share their personal experiences and to respond to their peers’ views on HV3C delivery. This technique provided us with a rich understanding of the topic under discussion.Citation22–24 We first performed a dry run with eight residents in obstetrics and gynecology to test the order, content, and wording of the discussion guide, which needed no modifications. The Ethical Review Board of the Netherlands Association for Medical Education approved this study on June 18, 2015, under file number 547.
Participants and setting
We conducted our study in the Dutch postgraduate medical education setting, where at the time of our study, HV3C was not part of the formal training curriculum. and present a general overview of the Dutch healthcare system and the Dutch medicine program.
Table 1. Overview of general characteristics of the Dutch health care system
Table 2. Overview of general characteristics of Dutch medical education continuum
Participants were 36 residents recruited from four institutions and six training programs (dermatology, n = 5; elderly care, n = 8; family medicine, n = 5; internal medicine, n = 6; orthopedic surgery, n = 6; surgery, n = 6) to participate in mono-professional focus groups. These residency training programs were selected to represent a broad range covering both surgical and nonsurgical specialties including hospital-oriented and non-hospital-oriented training programs. We sent an e-mail invitation and one reminder to 74 residents. Date and location of the focus group were based on residents’ preferences and availability. Focus groups varied in size from five to eight members. The majority of participants had received more than 2 years of training at the time of data collection; describes the characteristics of participants.
Table 3. Characteristics of focus group participants
Data collection and analysis
We collected data between August 2015 and July 2016. During a 1.5-hour focus group session we discussed the residents’ views, learning experiences, and personal struggles related to HV3C delivery. Focus groups were moderated by IAS (n = 3) or LAS (n = 3) and observed by LAS (n = 3) or AOP (n = 3). A moderator led the groups to keep focus on the research question while an observer watched the interactions to provide feedback for future focus groups. Data were audio recorded and transcribed verbatim and analyzed anonymously. ATLAS.ti (version 8.2.0) was used to manage the data. To ensure methodological rigor, we took precautionary measures based on quality criteria such as credibility, transferability, dependability, and conformability.Citation25–27 As characteristic for constructivist grounded theory approach, data collection and data analysis were done in an iterative manner, meaning that data collection was alternated with analysis to guide and deepen both collection and analysis.Citation28 Two researchers (LAS and LJ) independently began analyzing the transcribed data using open coding in a line-by-line approach.Citation29 In this phase, LAS and LJ focused on descriptions, forbearing interpreting or analyzing the transcripts, and subsequently discussed their initial impressions and findings at several meetings. The next phase involved axial coding to identify causal conditions and residents’ strategies.Citation30 In addition, LAS and LJ wrote analytical notes and reflective memos through all phases of data analysis. Constant comparison with alternating composition of the research team led to a collaborative analysis of dataCitation29 and entailed comparing different transcripts; different codes, notes, and memos; and different perceptions of the data.Citation28 After six focus groups, we reached saturation (i.e., no new themes emerged from the data and a sufficient understanding of the themes was gained).Citation24 Table 4 presents an overview of the iterative process of collaborative transcript analysis (Appendix B).
Because the background of researchers has an influence on how data collection and analysis are executedCitation31 we deliberately composed a research team with members of different disciplinary backgrounds (nursing, medicine, education) and relation to postgraduate training (medical student, resident, head of residency training, curriculum design). More specifically, the first author (LAS) combines research with residency training. To contain any potential bias inferring with data analysis, the entire team reflected critically on LAS’s dual attachments, training experiences, and solidarity with fellow residents during the analytical process.
Results
Data analysis yielded factors that influence how residents learn to provide HV3C. These factors operated on three levels: the individual, the training program, and the workplace.
Individual factors: residents’ different beliefs regarding HV3C
First, residents shared many commonalities in the way perceive HV3C. They all felt a strong sense of responsibility for providing the best care for their patients. Loyalty to patients was strong, and money was considered “a dirty word” if a patient’s condition demanded a diagnostic test or treatment. Despite these similarities, we discovered three types of beliefs regarding HV3C held by residents.
I’m just a frugal person
To some residents, upholding HV3C goes to the heart of being an excellent physician. These residents were critical of themselves and of their environment in terms of the way healthcare was delivered. Residents embracing this attitude typically asked critical questions such as “Will these tests change our treatment plan?” during group meetings and “Would you be open to medication?” in patient encounters.
When asked why HV3C was so important to these residents, they explained it was a personal trait that manifested itself not only in their role as physician but also in their private life. For example as one resident stated,
I’m just frugal, for myself as well. We [the institution] are on a tight budget; I refuse to spend it all on medication, on a pill that costs 5 euro, when you can get one for 45 cents that has the same effect. I think that is wasteful. … You know, if nobody does it [be critical], it will never change. (R16)
We have a professional responsibility, but …
A second group of residents also valued the importance of HV3C, a topic that cropped up sometimes in interactions with patients, supervisors, and healthcare professionals. However, they did not always stick to the principle, as they found it hard, sometimes even undesirable, to translate into practice. Specifically, they struggled to explain HV3C considerations in their communication with patients, feeling that it would merely add to uncertainty. In addition, they were often unsure whether to prioritize HV3C delivery above time management and learning goals, for example. They also tended to be accommodating when dealing with articulate, demanding patients who mistook more care for better care. As Resident 21 said, “But it does run in the back of your mind, that I think … that I don’t see any medical grounds [for more care] but yeah, am I going to spend half an hour chewing the fat about this?”
In all, these residents sometimes let time pressure, demanding patients, concerns over supervisors potentially overruling them, their wish to develop or maintain a patient–resident relationship, and fears of claims make them lose their focus on HV3C delivery. Although they initially aimed to provide HV3C, under external pressure their pro-HV3C aspirations waned. As Resident 2 explained, “Some patients do not want to leave. They want another scan and I struggle with that. Sometimes you give in, sometimes you can talk them out of it.”
Costs are not on my mind
One of the residents who participated in the focus groups felt very strongly that it was incumbent upon residents to do everything in their power to help the patient, however costly. Moreover, this resident suggested that the only person able to judge the quality of care is the patient. Although this position startled the other focus-group participants, it was clearly distinguishable as a personal-held belief about HV3C, as reflected in the following quote: “Well, you have to, you have to help the patient and that patient is lying in the hospital because they want to be helped. … I think you should pull out all the stops and not think, right … ?” (R6).
Factors associated with the training program
The focus group discussions identified the factors associated with clinical educators and the training program that impacted the residents’ learning curve. They explored both fruitful and missed learning opportunities, and the group reflected on how the training program shaped their approach.
Resident–supervisor interactions
How residents learned HV3C delivery was contingent upon the availability and approaches of supervisors. Specifically, supervisors played a big role in reducing the anxiety that uncertainty and external pressures could prompt residents to start unnecessary procedures. Uncertainty was frequently the driver of overdiagnosis or overtreatment, so the residents found it useful to get feedback from supervisors and to reflect with them on the necessity of particular tests or the motives underlying the requests. However, there was little opportunity to do this, which sometimes led to unwanted situations. The impact of such adverse events could be profound. For example, one resident began ordering more tests in response to an unexpected death.
Um, I had an unexpected death of a young patient with chest pain, and I’d failed to request an ECG. After that, I ordered lots of ECGs, all in cases of chest pain, and I guess that’s just the way it is at the moment. (R28)
The residents often mentioned that they sought advice from supervisors whom they knew would concur with their own views. As one resident put it,
You start positioning yourself a bit, manipulate a little, if you know what I mean. You know what you want to achieve, you trust your own ability at a given moment. And you know how to come across to so-and-so supervisor. … And you know … yeah okay, the supervisor would never let you do something that’s not allowed, but if you want to do something, then you learn, yes—[how to] manipulate. Then you just go to the supervisor for their accord. (R3)
Involvement in decision-making
Residents said that being increasingly involved in decision-making promoted HV3C learning. Yet opportunities to do so were not always forthcoming. On one hand, internal obstacles could prevent them from participating in (multidisciplinary) discussions, as some residents felt it not yet their place, questioned their own abilities, or were uncertain as to when to introduce the topic of HV3C into discussions that were already inherently complex.
On the other hand, however, supervisors did not always afford residents the opportunity to codecide. Some supervisors were known to give residents little or no say in the development of a diagnostic or treatment plan, which residents perceived as a missed opportunity. Similarly, residents reported that certain staff members did not value their participation, causing them to feel frustrated about the delivered care. In cases where the supervisor dominated, or residents wanted to avoid discussion, residents adapted their diagnostic or treatment plan to their supervisors’ preferences. At the same time, some supervisors encouraged residents to determine their own course of action. This is greatly appreciated, as Resident 7 said.
Well, we sure do have supervisors who say, “Okay, so this is what you propose? I’d do it this way. But let’s do it your way and see what happens.” Those are the cases where I learn the most … in a safe environment of course.
A final reason why residents do not always reap the full benefit of codecision opportunities is the short-term duration of their residency. Although they have plenty of opportunities to interact with patients during the residency, interactions were often momentary and long-term relationships with either supervisors or patients were scarce. This scarcity of follow-up opportunities caused residents to not always consciously consider patients’ individual preferences and values, as the next quote illustrates: “So we literally pull out all the stops to offer someone the highest quality of care possible, but often without really, yeah, at least ostensibly without thinking about what would actually be best for that individual patient (R32).”
Exposure to variation in care delivery
Varying attitudes to HV3C delivery among residents, physicians, departments, and hospitals often gave an important impetus to the focus group discussions. Residents recognize that HV3C practices depend in part on the patient population, available resources, and organizational structure, as demonstrated by Resident 36’s quote: “You learn by spotting the differences. Like in the case of biopsies. They do them really differently in a nonacademic center. It makes you wonder why we do it like that.”
Variations in individual approaches of physicians also represented important learning cues for residents inspiring them to pose critical questions and start discussions with individual physicians or within professional teams. In doing so, especially the more senior residents learned to differentiate between the warranted and unwarranted approaches of physicians or organizations.
Factors associated with the workplace
Availability of resources and expertise
Residents were motivated to learn about HV3C and actively sought input in their workplace. The depth of their learning, however, depended heavily on the workplace learning opportunities, which took the form of mostly written information resources—such as guideline, protocols, and books—and professionals, serving as information sources and adequate role models of how to provide HV3C. Residents found it easier to tap into professional expertise in small- or medium-sized nonacademic centers. The smaller scale was conducive to familiarity, both inter- and intraprofessionally, and made it easier to cement the resident–expert bonds that create better opportunities for feedback and questions. Finally, residents noted that in certain workplaces, staff seemed to be more critical of HV3C. For example, in the following example on prescribing, residents were called out when staff felt care delivery did not match HV3C:
It [ordering something expensive] does draw attention. If it’s not the one prescribing the drug, then it will be the pharmacist who says, “Gee, that drug you prescribed.” (R18)
(Response) Yeah, somebody will call you. (R13)
The presence of institutional policy
Residents agreed that the presence of a clear institutional policy on HV3C helped them to determine the best course of action. An institutional policy would be defined as the presence of clear guidelines based on the standards of professional associations, healthcare institutions, or departments (e.g., protocols, treatment plans). These guidelines would provide residents with direction and focus when making HV3C-related decisions. Yet an institutional policy regarding HV3C was often absent, and dilemmas were usually resolved with the ad hoc aid of supervisors.
Take transfusions. My current ward observes cutoff values very strictly. We often order them, but refuse to administer above the cutoff point. We don’t transfuse if a patient is tired. That’s not enough [justification]. The indications are less clear-cut on some wards but where I’m working now, it’s no use trying. We don’t think: Someone’s tired, anemic, let’s try two bags, maybe it’ll help. Do you understand? There’s a real difference. (R8)
Adding to the complexity of learning to provide HV3C were the mixed messages that residents received at the workplace level regarding their role in HV3C. For instance, when management would stress in the first meeting that laboratory testing should be reduced while they would stress in the second meeting that quality and following of guidelines should be prioritized over reduction of laboratory tests. Some residents also observed that hospital boards purportedly upholding HV3C values merely focused on money, not quality. Lack of clarity confused the residents, leading to some even resisting HV3C delivery.
Discussion
Residencies are currently faced with the difficult task of producing physicians who are skilled in the delivering HV3C.Citation15,Citation32 This study has sought to understand how residents learn to provide HV3C and how the postgraduate training environment influences their learning. Dutch residents, much like residents in other countries, are challenged to provide high-value care and take the costs into account.Citation18,Citation33 Answering the first research question, we found that residents’ learning was driven by their personal beliefs regarding HV3C, which led to equally different responses to HV3C-related dilemmas in clinical practice. When designing curricula, it is important to acknowledge different beliefs of residents and how these beliefs influence their HV3C learning. With this knowledge in mind, residents might benefit from training tailored to their individual needs and personal learning curves, as advocated in the literature.Citation34 Workplace-based learning could be enhanced if residents were encouraged to discuss their own beliefs and see how it affects the prioritization of their responsibilities as a physician.
Another powerful determinant of learning HV3C that emerged in our study was the presence of appropriate role models. Role modeling, combined with autonomy and reflection, has long been recognized as the most important pillar of workplace-based learning.Citation7,Citation19,Citation21 In our study, effective role models were supervisors who valued residents’ involvement in complex decision-making and gave them the opportunity to pursue their preferred plans. Furthermore, this study demonstrated that the influence of uncertainty and anxiety of residents hindered the delivery of HV3C. This influence of uncertainty is not limited to residents and has been reported in previous research among physicians.Citation35 Openly discussing these feeling of uncertainty and anxiety during supervision could therefore be important. To foster uptake of HV3C delivery in the workplace, it is therefore imperative that role models be informed about their influence and be involved in workplace-based learning. Role modeling HV3C is known to be scarce in residency training, due to lack of personal knowledge and training.Citation18 Specific training may be required, teaching physicians to become effective role models of HV3C. It is not surprising that today’s academic faculty experience difficulties in teaching HV3C delivery, considering that they themselves were not trained in these competencies.Citation36 Finally, it may help to let residents observe and reflect on intersupervisor variations.
Learning environments are known to have a major impact on learning in medical training and on care delivery during residency,Citation17,Citation37,Citation38 and even throughout physicians’ future career.Citation38 Our second research question addressed the influence of the learning environment on the uptake of HV3C delivery. Previous research has demonstrated that physicians’ HV3C learning must be situated in a supportive environment,Citation7,Citation20,Citation33 and currently little emphasis is given to HV3C education in residency training.Citation15,Citation16,Citation32,Citation39–41 Once again, our results emphasize the importance of a supportive workplace to facilitate learning. This criterion was met when the workplace operated with clear policies and residents could tap easily into the professionals’ expertise. In this context, smaller healthcare organizations proved to be better breeding grounds for interdisciplinary networks with short communication lines facilitating access to expertise. This finding would make a case for residents rotating between healthcare organizations that vary in size were it not for the fact that this would interfere with the need highlighted by our participants to establish long-term relationships with patients. A final recommendation arising from this research, which has been reiterated on various occasions elsewhere,Citation42–44 is to create a safe learning environment: To enhance learning, residents must feel free to ask their supervisors critical questions and discuss HV3C-related dilemmas and residents’ different beliefs.
Strengths and limitations
A major strength of this research is the variety of specialties involved to meet triangulation and transferability quality requirements.Citation25–27 The participants not only came from six residency programs but also worked in various primary, secondary, and tertiary care institutions. This has led to rich data, enabling us to develop a broad theory, transcending the typical one hospital-based workplace. Furthermore, the wide range of expertise in the research team allowed us to discuss the data from several perspectives. Another strength is the rigor with which we performed our data analysis, owing to the use of constructivist grounded theory to address our research questions.Citation30 Yet a few limitations must be mentioned. First, participants may have been more aware of and favorable to HV3C because focus group participation was voluntary. Nevertheless, the residents did not seem particularly enthusiastic at the start of the focus groups and sometimes showed uncertainty as to what could be defined as HV3C. Second, caution should be exercised when extrapolating the results to areas outside the Netherlands, where this study was conducted, as the Dutch residency program, financial structure, and healthcare organization may differ from those existing in other regions of the world.
Conclusion
Residents struggle to seize high-value, cost-conscious care learning opportunities in the workplace setting. Both residency training programs and departmental policies can contribute to creating these learning opportunities. An important starting point is being aware of the different personal beliefs of residents and the approaches to HV3C on the level of the training program and workplace.
References
- Pate C DCM, Lutz S. The price of excess: identifying waste in health care spending.: Price Water house Coopers; 2008 [cited 2018 21-12-2018]. Available from: http://www.pwc.com.
- Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA 2012;307(14):1513–1516. doi: 10.1001/jama.2012.362.
- Pl Sr Y, Olsen LA. The Healthcare Imperative: lowering Costs and Improving Outcomes: workshop Series Summary. Washington D.C.: The National Academies Press (U.S); 2010.
- Crosson FJ. Change the microenvironment. Delivery System Reform Essential to Control Costs. Mod Healthc 2009;39(17):20–21.
- Shah N, Levy AE, Moriates C, Arora VM. Wisdom of the crowd: bright ideas and innovations from the teaching value and choosing wisely challenge. Acad Med. 2015;90(5):624–628. doi: 10.1097/ACM.0000000000000631.
- Owens DK, Qaseem A, Chou R, Shekelle P. Clinical Guidelines Committee of the American College of P. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011;154(3):174–180. doi: 10.7326/0003-4819-154-3-201102010-00007.
- Detsky AS, Verma AA. A new model for medical education: celebrating restraint. JAMA 2012;308(13):1329–1330. doi: 10.1001/2012.jama.11869.
- Weinberger SE. Educating trainees about appropriate and cost-conscious diagnostic testing. Acad Med. 2011;86(11):1352doi: 10.1097/ACM.0b013e3182308db7.
- Cooke M. Cost consciousness in patient care-what is medical education's responsibility? N Engl J Med. 2010;362(14):1253–1255. doi: 10.1056/NEJMp0911502.
- Weinberger SE. Providing high-value, cost-conscious care: a critical seventh general competency for physicians. Ann Intern Med. 2011;155(6):386–388. doi: 10.7326/0003-4819-155-6-201109200-00007.
- Choosing Wisely Foundation T. Choosing Wisely Foundation [Webpage]. 2012; [updated 2019; cited 2018 08-03-2018]. Available from: http://www.choosingwisely.org/.
- Brody H. Medicine's ethical responsibility for health care reform-the Top Five list. N Engl J Med. 2010;362(4):283–285. doi: 10.1056/NEJMp0911423.
- Williams High Value Care Curriculum (HVC) 4.0: ACP; 2012 [cited 2018 01-07-2018]. 4.0: Available from: https://www.acponline.org/clinical-information/high-value-care/medical-educators-resources/newly-revised-curriculum-for-educators-and-residents-version-40.
- Stammen LA, Stalmeijer RE, Paternotte E, et al. Training Physicians to Provide High-Value, Cost-Conscious Care: A Systematic Review. JAMA 2015;314(22):2384–2400. doi: 10.1001/jama.2015.16353.
- Patel MS, Reed DA, Loertscher L, McDonald FS, Arora VM. Teaching residents to provide cost-conscious care. Jama Intern Med. 2014;174(3):470–472. doi: 10.1001/jamainternmed.2013.13222.
- Ryskina KL, Holmboe ES, Shea JA, Kim E, Long JA. Physician Experiences With High Value Care in Internal Medicine Residency: Mixed-Methods Study of 2003-2013 Residency Graduates. Teach Learn Med. 2018;30(1):57–66. doi: 10.1080/10401334.2017.1335207.
- Gupta R, Moriates C. Swimming Upstream: Creating a Culture of High-Value Care. Acad Med. 2017;92(5):598–601. doi: 10.1097/ACM.0000000000001485.
- King BC, DiPace J, Naifeh M, Hammad H, Gerber LM, Abramson E. Pediatric Training Faculty and Resident Perceptions on Teaching High-Value, Cost-Conscious Care: A Multi-Institutional Study. Hosp Pediatr 2017;7(9):547–552.
- Glicken AD, Merenstein GB. Addressing the hidden curriculum: understanding educator professionalism. Med Teach. 2007;29(1):54–57. doi: 10.1080/01421590601182602.
- Hafferty FW, Franks R. The hidden curriculum, ethics teaching, and the structure of medical education. Acad Med. 1994;69(11):861–871.
- Coulehan J, Williams PC. Vanquishing virtue: the impact of medical education. Acad Med. 2001;76(6):598–605.
- Savin-Baden M. Qualitative Research - The Essential Guide to Theory and Practice. New York: Taylor & Francis group; 2013. 569 p.
- Parker ATJ. Focus group method and methodology: current practice and recent debate. International Journal of Research & Method in Education. 2006;29(1):23–37. doi: 10.1080/01406720500537304.
- Stalmeijer RE, McNaughton N, Van Mook WN. Using focus groups in medical education research: AMEE Guide No. 91. Med Teach. 2014;36(11):923–939. doi: 10.3109/0142159X.2014.917165.
- Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–357. doi: 10.1093/intqhc/mzm042.
- Frambach JM, van der Vleuten CP, Durning SJ. AM last page. Quality criteria in qualitative and quantitative research. Acad Med. 2013;88(4):552doi: 10.1097/ACM.0b013e31828abf7f.
- O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–1251. doi: 10.1097/ACM.0000000000000388.
- Watling CJ, Lingard L. Grounded theory in medical education research: AMEE Guide No. 70. Med Teach. 2012;34(10):850–861. doi: 10.3109/0142159X.2012.704439.
- Charmaz K. Constructing Grounded Theory: London Sage; 2006.
- Corbin J. Basics in Qualitative Research: Thousand Oaks; Sage; 2008.
- McMillan W. Researching Medical Education: New York; John Wileys & Sons; 2015.
- Smith CD. Alliance for Academic Internal Medicine-American College of Physicians High V, Cost-Conscious Care Curriculum Development C. Teaching high-value, cost-conscious care to residents: the Alliance for Academic Internal Medicine-American College of Physicians Curriculum. Ann Intern Med 2012;157(4):284–286.
- Sedrak MS, Patel MS, Ziemba JB, et al. Residents' self-report on why they order perceived unnecessary inpatient laboratory tests. J Hosp Med. 2016;11(12):869–872. doi: 10.1002/jhm.2645.
- Finlay L. Negotiating the Swamp: The Opportunity and Challange of Reflexivity in Research Practice. Qualitative Research. 2(2). London: Sage publications; 2002. p. 209–30. doi: 10.1177/146879410200200205.
- Colla CH, Kinsella EA, Morden NE, Meyers DJ, Rosenthal MB, Sequist TD. Physician perceptions of Choosing Wisely and drivers of overuse. Am J Manag Care. 2016;22(5):337–343.
- Wong BM, Holmboe ES. Transforming the Academic Faculty Perspective in Graduate Medical Education to Better Align Educational and Clinical Outcomes. Acad Med. 2016;91(4):473–479. doi: 10.1097/ACM.0000000000001035.
- Hoff TJ, Pohl H, Bartfield J. Creating a learning environment to produce competent residents: the roles of culture and context. Acad Med. 2004;79(6):532–539.
- Asch DA, Nicholson S, Srinivas S, Herrin J, Epstein AJ. Evaluating obstetrical residency programs using patient outcomes. JAMA 2009;302(12):1277–1283. doi: 10.1001/jama.2009.1356.
- Gould BE, O'Connell MT, Russell MT, Pipas CF, McCurdy FA. Teaching quality measurement and improvement, cost-effectiveness, and patient satisfaction in undergraduate medical education: the UME-21 experience. Fam Med 2004;36 (Suppl):S57–S62.
- Dine CJ, Miller J, Fuld A, Bellini LM, Iwashyna TJ. Educating Physicians-in-Training About Resource Utilization and Their Own Outcomes of Care in the Inpatient Setting. J Grad Med Educ. 2010;2(2):175–180. doi: 10.4300/JGME-D-10-00021.1.
- Ryskina KL, Halpern SD, Minyanou NS, Goold SD, Tilburt JC. The role of training environment care intensity in US physician cost consciousness. Mayo Clin Proc. 2015;90(3):313–320. doi: 10.1016/j.mayocp.2014.12.005.
- Lombarts KM, Heineman MJ, Scherpbier AJ, Arah OA. Effect of the learning climate of residency programs on faculty's teaching performance as evaluated by residents. PLoS One. 2014;9(1):e86512doi: 10.1371/journal.pone.0086512.
- Genn JM. AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med. Teach 2001;23(4):337–344. doi: 10.1080/01421590120063330.
- Genn JM. AMEE Medical Education Guide No. 23 (Part 2): Curriculum, environment, climate, quality and change in medical education - a unifying perspective. Med Teacher. 2001;23(5):445–454. doi: 10.3109/01421590109177942.
Appendix A
Focus group: Specialism, Date.
Before starting the focus group → participants sign informed consent form.
Welcome:.
• Moderator and observer proposals.
• Brief explanation of focus group.
Research goal: brief explanation of the framework and objective of the study.
“Thank you for taking part in this focus group. At Maastricht University, we are conducting research into HV3C and the role of education in this. I (IS/LS) will lead the group. We’re about to start but before we do, I’d like to say that our aim is to gain further insight into your experiences and opinions regarding the theme of HV3C, I’d like to emphasize that there are no right or wrong answers. Different experiences and opinions are all useful for our study. That’s also why we’d like you to respond to colleagues’ experiences and opinions. At the end of the session, we will briefly discuss what you have said and commented on in response to the research questions.”
Appendix B
COREQ (COnsolidated criteria for REporting Qualitative research) Checklist.
A checklist of items that should be included in reports of qualitative research. You must report the page number in your manuscript where you consider each of the items listed in this checklist. If you have not included this information, either revise your manuscript accordingly before submitting or note N/A.