Facilitators and barriers to a nationwide implementation of competency-based postgraduate medical curricula: A qualitative study

Figures & data

Table

Practice points

Despite the increasing attention to competency-based PGME curricula, research on the implementation process of these curricula is lacking. The implementation process of competency-based PGME is highly dynamic, non-linear, and influenced by many factors. The local/regional context determines the speed, quality, direction of the implementation process, and how specific factors affect this process. The implementation process is largely dependent on the quality of the initial design of the curriculum; we recommend designing the curriculum with the needs of the primary users (program directors, supervisors, and residents) in mind and align the benefits directly with the outcomes of health care processes, such as patient safety, to make it more attractive for the users to adopt the innovation(s). For implementing competency-based PGME, it is crucial to translate the national curriculum to a regional and local curriculum that specifies the CanMEDS roles, procedures for work-based learning, and assessment in the context of patient care. For organizing the implementation process, it is important to create coordination, establish goals and a timeframe, and monitor the results. This coordination needs to balance “letting it emerge” and “making it happen.” Organizational/educational support and using residents as change agents are both very helpful in accelerating the implementation process.

Table 1  Changes introduced into the Dutch PGME curricula

The RDMA introduced in 2004 the following changes into PGME in the Netherlands:

• The CanMEDS framework. The entire curriculum, including the learning goals, building blocks, assessment strategy and instruments, teaching activities, and materials, should be oriented to a set of seven roles for practicing specialists, derived from societal needs

– medical expert

– communicator

– health advocate

– collaborator

– manager

– scholar

– professional

• The mini clinical evaluation exercise (Mini-CEX); frequency – 10 observations each year

– method to assess competencies in real-life clinical practice (Norcini et al. 2003)

– consists of a short observation by a qualified medical specialist of a resident demonstrating clinical skills, using a pre-defined scoring format, followed by a structured feedback conversation (Pendleton et al. 2003)

• The portfolio, including periodical interviews between the program director and residents about the portflio; frequency – 1st year five meetings, 2nd and 3rd year – three meetings, and 4th and 5th year – two meetings

– in the portfolio, residents collect evidence documenting personal development according to the CanMEDS roles

– the results from the assessment instruments and evidence of acquired knowledge and skills are assembled, along with results of repeated reflection sessions on acquired competencies and a personal strengths–weaknesses analysis (Scheele et al. 2008)

• The critical appraised topic; this includes the systematic search of scientific evidence for a clinically relevant question; frequency – twice each year

• Knowledge progression tests; frequency – each year

Figure 1. Organization of the In VIVO implementation process.

Table 2  Sampling procedure and interview questions

	Goals	Respondents	Interview topics/questions
National level	• Gain overall view of the project • Gain insight into effects of national and regional implementation strategies and processes, and the factors influencing them • Investigate different implementation strategies and regional implementation teams	Interviews with members of national project team (N = 3) • project leader • educationalist • researcher	• What were the goals and results of the project • Was the project structure effective in achieving the goals and results • What factors positively and negatively influenced the implementation process • What were the differences between regions in terms of the implementation strategy
Regional level	• Gain insight into effects of national and regional implementation strategies and processes and the factors influencing them • Investigate whether some program directors and departments in the region were more innovative and active in the project	Interviews with members of two regional implementation teams (N = 10) • project leader in O&G • project leader in Pediatrics • two residents • educational specialist Regional implementation team A • Highly structured regional implementation process; regional implementation meetings were organized with all program directors in the region • Located in low-density population of people and hospitals Regional implementation team B • Weakly structured regional implementation process; no structural regional implementation activities were executed • Located in a dense population of people and hospitals	• What were the goals and results of the project • What was the implementation strategy in your region • What factors positively and negatively influenced the implementation process • What are the main differences between before and after the project • How has the project contributed to these differences • Were there differences between the departments in your region with regard to innovation and activity
Local level	• Gain insight into effects of regional and local implementation strategies and processes and the factors influencing them	Interviews with members of two departments in the two regions (departments A and B) (N = 12) • program director • supervisor • resident The two departments differed in their innovation and activity with regard to the regional implementation team and project	• What were the goals and results of the project • What was the implementation strategy in your department? • How do you perceive the new curriculum and assessment instruments • What are the main differences between before and after the project • How has the project contributed to these differences • What factors positively and negatively influenced the implementation process

Table 3  Main attributes found to be important for PGME innovation

	Facilitator to implementation (what and why it worked as a facilitator)	Barrier to implementation (what and why it worked as a barrier)
Attributes of the innovation and adopters
Relative advantage of:
CanMEDS roles^a	The adopters attached some advantages to the use of this innovation. More attention to generic competencies in the assessment of residents was expected to have positive impacts on the improvement of health care quality and processes, like patient safety, collaboration, and communication and organization. Furthermore, the use of the CanMEDS roles provided a language between program directors, supervisors, and residents to discuss the progress of the residents	The program directors, supervisors, and residents attached clear disadvantages to the use of this innovation: The CanMEDS roles proved difficult to operationalize, implement, and assess – especially the non-medical competencies – in workplace-based training. This was caused by the abstract nature of competencies, a tendency to focus on easily assessable competencies and lack of time, training and evidence
Behavioral changes supervisor: structured feedback^a	The program directors, supervisors, and residents attached clear advantages to the use of this innovation. More structured feedback led to more balanced (both positive and negative comments), efficient, and effective feedback encounters between supervisor and resident	The program directors, supervisors, and residents attached some disadvantages to the use of this innovation. Giving and receiving feedback was perceived as difficult: there was fear of damaging the relationship and fear of negative feedback. Also, the quality of feedback by supervisors was sometimes perceived as worrisome by residents
Behavioral changes resident: pro-active attitude^a	The adopters attached clear advantages to the use of this innovation. The residents became more aware of their learning objectives	The residents attached some disadvantages to the use of this innovation. The residents felt sometimes disappointed because their learning objectives could not be accommodated due to the complexity of work schedules. Some supervisors considered the learning goals far too specific
Assessment instruments^a	The adopters attached some advantages to the use of these innovations. The use of the Mini-CEX produced an improved, standardized, and more objective assessment of clinical skills that allowed more explicit, focused, and balanced reflection on the clinical behavior of residents in specific situations. The portfolio enhanced the opportunity of residents to reflect on their own development and to construct their own training	The program directors, supervisors, and residents attached some disadvantages to the use of this innovation. The accomplishment of a certain number of assessments did become a goal in itself, although it should be a means to facilitate feedback. Organizing a Mini-CEX took a considerable time and effort. The potential of the portfolio was limited due the subjective nature and the limitations of the residents’ ability to shape their own training
Changes educational program^a	The adopters attached some advantages to the use of this innovation. More continuity in the internships led to more opportunities of supervisors to observe residents and it led to steeper learning curves of the resident	The program directors, supervisors, and residents attached some disadvantages to the use of this innovation. Implementation was sometimes difficult due to understaffing of supervisors and residents
Simplicity and compatibility	The innovations “structured feedback by supervisor” and “pro-active attitude resident” were fairly simple to comprehend and they were compatible with the values, norms, and perceived needs of the adopters. These innovations were easily implemented because relatively few additional resources were required to implement them in the work-based training	The innovation's “assessment instruments,” “changes educational program,” and “CanMEDS roles” were fairly difficult to comprehend and they were not so compatible with the values, norms, and perceived needs. These innovations were less likely to be implemented because they required a lot of effort to organize and implement and they intervened with daily health care processes
Attributes of the implementation process
External developments	The forces that were outside of control of the national project were made to work for the implementation process, rather than allowing them to work against the process. The increased attention to educational quality allowed the national project team to start pilot projects introducing an educational quality care system into PGME	Allowing the forces that were outside of control of the national project to work against the implementation process. The introduction of a national medical training fund generated skepticism among the program directors, as it remained unclear how the quality was supposed to be measured. This had a negative impact on the implementation process, because adopters associated this process with the allocation of governmental funds and it consumed a lot of valuable time. The reduction in the number of pediatrics training positions during the project decreased motivation and increased the workloads of supervisors and residents, thus allowing less time for the implementation of the innovations
Appropriate change model	“Letting it happen” was facilitated by the national project team by stimulating regional implementation teams and program directors to entrepreneurship. Entrepreneurship caused regional implementation teams and program directors to take creative action and to let them feel responsible for the implementation process	Too little “making it happen” by the national project team. The program directors, supervisors, and residents perceived that they received too little guidance on the implementation process from the national project team in the beginning of the project. The adopters felt that they were turned loose and that is was unclear what was expected. This slowed down the process
Good project management	The flexibility in the project management by the national project team and the absence of objectives to be reached at certain points in time. The organic implementation process allowed for the incorporation of relevant developments. This increased the acceptance among the adopters because they perceived as having control over the process	The absence of explicit goals, a clear time frame, and monitoring of the results by the national project team. The adopters felt that they were turned loose and that it was unclear what was expected
National and regional implementation strategies and activities^a	Well-organized national and regional meetings/activities with clear goals and relatively high returns for their time investment. Such meetings and activities generated knowledge sharing and a mutual understanding of the process	The absence of or poorly organized national and regional meetings/activities. The absence of such meetings slowed down the implementation process because opportunities to share knowledge were missed. Poorly organized meetings left adopters frustrated because they experienced relatively low returns for their time investment
Educational support^a	The presence of well-organized educational support and competent educationalists. Educational support and educationalists significantly accelerated the implementation process by their specific expertise and knowledge, by providing concrete instructions for the use of the assessment instruments, helping to translate the national curriculum to workplace-based training, organizing the process, assisting in setting up regional meetings and activities, and exchanging best-practices with their colleagues in other hospitals and regions. This support and expertise complemented the expertise of the busy physicians	The absence of well-organized educational support and competent educationalists. The absence slowed down the implementation process. It meant that all support for the implementation process had to be delivered by busy physicians who are in general not trained in educational and change management issues
Human resource factors: training	The provision in the early phases of the implementation process of well-organized training preferably for all adopters in a department aimed at implementing the innovations. Training generated a more positive attitude toward the innovations, led to knowledge sharing, enhanced the educational knowledge and teaching behavior of the doctors, and caused improvements in the clinical learning climate. Entering the training into the early phases allowed the adopters to pick up the innovations early and more easily	The absence of training, the provision of training to some adopters in a department, or the provision of training in the later phases of the implementation process. The absence of training or the provision of the training in the later phases slowed down the process. Providing training to a subset of the adopters in a department caused a sub-optimal learning climate
Interpersonal influence	Facilitate knowledge sharing between program directors, effectively use social networks to spread the innovations and use residents as change agents. Knowledge sharing between program directors led to mutual understanding of the process and led to the exchange of best-practices. The effective use of residents as change agents enhanced their motivation and efforts of knowledge diffusion to their fellow residents	The absence of or suboptimal knowledge sharing and use of residents as change agents. Suboptimal knowledge sharing between program directors slowed down the process because opportunities to exchange best-practices were missed. Not using residents as change agents by their program directors caused these residents to feel left-out, it reduced their motivation and it reduced their efforts to diffuse knowledge of the innovation to their fellow residents, or in a worse case scenario, it caused them to diffuse negative opinions of the innovations to their fellow residents
Attributes of the organization
Leadership and management	The presence of good leaders and management at all levels of the projects. The implementation process was accelerated by the presence of good leaders (project leaders and program directors); they were perceived as role models, they inspired, were ambitious, experienced high intrinsic motivation, were entrepreneurs, and got things done	The absence of good leaders, the lack of “substitute” leaders, and the lack of investment in new leaders at all levels of the projects. The implementation process was slowed down by the absence of good leaders (project leaders and program directors): relatively few initiatives and creative action were undertaken and supervisors and residents lacked good role models to inspire them
Slack resources and support management	The support of management (on all management levels) in providing slack resources, especially educational support. Program directors, supervisors, and residents highly appreciated the positive reinforcement by management, such as the Board of Directors. The implementation process was significantly accelerated if the management was willing and able to provide resources, especially educational support	The absence of support of management (on all management levels) in providing slack resources, especially educational support. The implementation process was significantly slowed down if management was unwilling or unable to provide resources. The adopters were less likely to invest into the process without the positive reinforcement of management
Effective data capture and feedback systems	The application of questionnaires by the national project team to monitor the implementation process. These proved useful for providing feedback on the implementation process and provided a new push for the process	The questionnaires provided only quantitative data (mostly five-point scale). The lack of qualitative data made it hard to interpret the information acquired
Tension for change and balance between supporters and opponents	High value attached to medical education, in general, and high tension for urgency and need to implement the innovations more specifically, as perceived by preferably all adopters in a department. Adopters were more likely to implement the innovations if they attached high value to medical education. “True believers” in the new learning philosophy experienced high intrinsic motivation and functioned as role models for their fellow supervisors and residents. Residents experienced higher motivation and had more fun when program directors and supervisors also felt responsible and took initiative to execute the instruments	Low value attached to medical education in general, and low tension for urgency and need to implement the innovations more specifically, as perceived by all or some adopters in a department. If some or many supervisors and residents in the department considered education less important than research or medical care, adopters were less motivated to implement the innovation. The extent to which implementation was rewarded, supported and expected within the organizations was found to be a driver of adoption of the particular innovation
Difference between general and university hospitals^a	Small departments from general hospitals with cohesive communication and decision-making structures on the one hand and effective regional collaboration structures between general and university hospitals on the other hand. Communication and decision-making processes in the smaller departments were more efficient, and members felt more responsible for implementing the innovations than did their counterparts in the larger university hospital departments. University hospitals were perceived as being more in the lead in terms of the regular regional educational collaboration between program directors, the regional allocation of resident training positions, and educational expertise. The implementation process was accelerated by effective regional collaboration structures between program directors that consolidated the strengths of both types of hospitals	Large departments from university hospitals with fragmented communication and decision-making structures on the one hand and the absence of or ineffective regional collaboration structures between general and university hospitals on the other hand. Communication and decision-making processes in the bigger departments were less efficient. The implementation process was slowed down by the absence of or ineffective regional collaboration structures between program directors of general and university hospitals. Knowledge sharing and best-practices exchange opportunities were missed, which slowed down the process
Balance training and patient care^a	Departments that found a good balance between the personal learning objectives of the resident and the workload of patient care. Preferably, with the renewed curriculum, every resident should have received a tailor-made training according to the required learning objectives. This ambition was limited by the high workload of patient care and the size of the resident group. Furthermore, working under high pressure also provided learning opportunities. A good balance had a facilitating effect on the implementation process	Departments that had a misbalance balance between the personal learning objectives of the resident and the workload of patient care. In most cases, the misbalance is hanging toward doing patient care. Residents missed valuable learning opportunities and were not accomplishing their personal learning objectives. This slowed down the implementation process

Note: ^aSpecific feature of PGME innovation within the Greenhalgh framework, as described in the discussion section of this paper.

Table 4  Recommendations to tailor interventions in the context of PGME innovation

Level^a

The challenge of implementing the CanMEDS roles • Translate the national curriculum to a regional and local curriculum that specifies the CanMEDS roles, procedures for work-based learning, and assessment in the context of patient care • Design the curriculum with the needs of the primary users (program directors, supervisors, and residents) in mind to make it more attractive for them to adopt the innovation • Align the benefits of the renewed curricula directly with the outcomes of health care processes, such as patient safety and team collaboration and communication Regional implementation strategies and educational support • Create coordination of the implementation process (by specialty) • Establish goals, establish a timeframe, and monitor the results in concordance with the professional • Balance “letting it emerge” and “making it happen,” depending on implementation progression • Use training as an implementation tool (to raise awareness among users and to learn the required skills and knowledge) • Facilitate knowledge sharing through meetings and social networks • Use residents as change agents • Identify and reward good leaders and good teams; ensure continuity • Organize logistic organizational and educational support Need for regional inter-organizational networks of hospitals • Build strong regional collaboration networks between university and general hospitals Balance between training and patient care • Achieve a careful balance between education (work-based learning) and patient care

N, R, L N, R, L N, R, L N, R, L N, R, L N, R, L R, L N, R, L R, L N, R, L N, R, L R, L L

Notes: ^alevel at which it is important to apply the practice points: N, national; R, regional; and L, local.

Norcini J, Blank L, Duffy FD, Fortna G. The mini-CEX: A method for assessing clinical skills. Ann Intern Med 2003; 138(6)476–481

 PubMed Web of Science ®Google Scholar

Pendleton D, Schofield D, Tate P, Havelock P. The new consultation: Developing doctor-patient communication. Oxford University Press, Oxford 2003

 Google Scholar

Scheele F, Teunissen P, Van Luijk S, Heineman E, Fluit L, Mulder H, Meininger A, Wijnen-Meijer M, Glas G, Sluiter H, et al. Introducing competency-based postgraduate medical education in the Netherlands. Med Teach 2008; 30(3)248–253

 PubMed Web of Science ®Google Scholar