Implementation of a competency-based residency curriculum: experiences from a resource-limited environment in the Caribbean

Abstract

Background: The introduction of competency-based curricula in institutions situated in resource-limited environments is likely to pose new challenges for the implementation process. The St. Elisabeth Hospital (SEHOS) in Curaçao, Dutch Caribbean, is affiliated to university teaching hospitals in the Netherlands. It is a teaching hospital in a resource-limited environment.

Aims: Following the revision of the curriculum of the Dutch paediatric residency training, the country's paediatric society mandated its implementation in all training institutions within Dutch sovereignty. We set out to implement the revised curriculum in SEHOS and highlight the benefits it may have in a setting like Curacao.

Method: The intended learning outcomes for the 16-week project were (1) the implementation of the core changes in the curriculum and (2) conduct an assessment of the learning environment. The strategies used included informative meetings, workshop and lecture-based trainings, questionnaire surveys, and focus group interviews.

Results: Registrars and staff were successfully trained in how to use brief clinical assessments and digital portfolios. ‘Departmental portfolio’ was introduced as an innovative method of evaluating and monitoring departmental educational activities.

Conclusions: Competency-based curricula can be successfully implemented in resource-limited environments. Educational experts in the local setting are crucial for the success of the process.

Introduction

The professional training of specialist registrars (SpRs) in the Netherlands is undergoing major reform. The country's Central College of Medical Specialists (CCMS) has recommended that the curricula of specialist training programs in the country should be revised, in order to address the growing changes in the medical profession, as well as the shifting demands for health care services in society (Framework for the Dutch Specialist Training Programs 2004). The rationale for this recommendation was to re-define the criteria for successful professional training; from the traditional conception that the duration of clinical exposure determines the content and quality of what learners acquire to an outcome-based training, where competencies are used to assess the amount and quality of learning that has taken place within a predefined period of time. Seven professional competencies, culled from the essential role and key competencies of specialist physicians of the Canadian Royal College of physicians and surgeons, formed the basis for this recommendation (CANMEDS 2005). These competencies (i.e. medical expert, communicator, collaborator, manager, health advocate, scholar and professional), were identified as mandatory requirements for the training of medical specialists in the Netherlands, and were proposed to constitute the curriculum of a new competency-oriented, training program for Dutch SpRs (Bleker et al. 2004). In order to facilitate this transition process, an advisory board for postgraduate curriculum development was set up to support the 27 Dutch medical societies in developing (and implementing) their respective curricula in accordance with the new guidelines. Besides their primary advisory role, the board was also responsible for disseminating best practices and new ideas among the different medical societies based on the medical educational literature and new insights from the ongoing curriculum reform process (Scheele et al. 2008).

Table

Practice points

A ‘Departmental portfolio’ is an innovative tool that can be used to evaluate and monitor the educational activities of clinical departments. With careful and pragmatic approaches, it is feasible to implement competency-based curricula in environments where resources are limited. For successful implementation of innovative educational initiatives, adequate information on the process and approval by all concerned parties are mandatory.

In a government sponsored project (IN VIVO project 2006), the Dutch society of Paediatrics (together with the society of Obstetrics and Gynaecology) began with the formal implementation of their revised postgraduate curricula. The IN VIVO project was set up as a pilot study for the curricula reform process within the Netherlands and the aim was to gather helpful information that other medical societies could use for the successful implementation of their respective curricula.

The most notable changes in the curriculum of Dutch paediatric residency training program were the introduction of educational portfolios and new methods of evaluation that included self reflection reports, brief clinical assessments and multi-source feedback (revised curriculum for the Dutch paediatric specialist training program). The brief clinical assessment otherwise referred to as direct observation in practice (Scheele et al. 2008) were in training assessment methods similar to the mini-CEX described by Norcini & Burch (2007). The goal of these new methods of evaluation was to ensure that the assessment and quantification of any form of learning during the SpRs’ professional training was objective and quantifiable.

Method

The St. Elisabeth hospital in Curaçao (educational context)

The St. Elisabeth Hospital (SEHOS) was established in 1855 and is the largest and oldest hospital on the Dutch Caribbean island of Curaçao. It has 430 beds and provides services in all major clinical specialties, including adult, paediatric, and neonatal intensive care. The hospital has a close professional alliance with tertiary medical institutions in the Netherlands (the universities of Groningen and Amsterdam) and provides training services for Dutch medical students and residents. The scope of its programs include accredited residency training in paediatrics and pre-residency training in, internal medicine, surgery, obstetrics and gynaecology, neurology, otolaryngology, ophthalmology, dermatology and paediatrics (Busari et al. 2001). The curricula of the respective training programs are catered for by the University Medical Centre Groningen (UMCG) and Academic Medical Centre Amsterdam (AMC) and implemented in line with the recommendations of the Dutch medical specialist registration body (MSRC).

Compared to the other teaching hospitals within the sovereignty of the Netherlands, the SEHOS is an example of an affiliated teaching institution in an environment where resources are limited. This is reflected by a low gross domestic product (GDP) and high perinatal mortality rate. The GDP per capita of the Netherlands Antilles is estimated at US $17.032 compared to US $36.618 in the Netherlands (IMF 2006). According to the World Health Organization, perinatal mortality reflects the standard of a country's obstetric and paediatric health care. (WHO 2006) While the perinatal mortality rate in Curaçao is slightly lower than average in the Caribbean region (23.5 vs. 31/1000), it is substantially higher when compared to countries in the North American and European regions (7 and 13/1000 respectively). Curaçao is thus a country, with a less-developed health care system. This has implications for the prioritization and distribution of its available resources (PAHO 2005) as well as for the successful implementation of (new) educational initiatives.

Project description

In January 2005, the UMCG initiated the implementation of the new paediatric training curriculum in all of its affiliated teaching hospitals including the SEHOS. Due to limited financial resources and the lack of an educational expert in Curaçao, the UMCG appointed an educationalist (and paediatrician) in April 2005 to coordinate the implemenation process on-site, in collaboration with the directors of the residency training programs in UMCG and SEHOS. As little was known on the constraints of implementing competency-based curriculum in resource-limited country at the time, we also investigated the feasibility of implementing the core elements of the Dutch competency-based postgraduate program in Curaçao; it being an example of an environment with limited resources.

Two central objectives formed the focus of this project. The first was to adequately implement the core changes in the curriculum of the residency program, while the second was to assess the learning environment and in particular highlight (any) additional benefits a competency-based training in an environment like Curaçao could offer. The intended outcomes that we defined for the project included:

Implementation of core changes in the Curriculum

1. Introduce a new process of evaluation that focuses on professional competencies of SpRs as recommended by the CCMS.

2. Implement a new process of evaluation using the guidelines of the Dutch pediatric society that includes the implementation of (1) Portfolios and (2) Brief clinical assessments.

Assessment of the learning environment

1. Highlight any additional benefits of the clinical rotation for SpRs in Curaçao compared to the Netherlands.

2. Identify aspects of the residency training in Curaçao that may require improvement.

Prior to commencement of the 16 week project, a number of major and specific actions were defined. These were to be applied in achieving the outcomes of the project mentioned above (see Box 1).

Box 1 List of proposed actions for the 16-week project

Main actions

1. Implement Brief clinical assessments

2. Implement the registration of clinical evaluations and individual progress using Portfolios

3. Identify the competencies in the medical residency that are achievable during the clinical rotation in Curaçao

4. Identify the aspects of the medical training that are deficient or require improvement

5. Ensure that the objectives and the content of the training program in Curaçao are in alignment with the program in the main teaching hospital (UMCG)

Specific actions

1. Provide specialist physicians and SpRs with basic information on the use of portfolios and highlight the advantages for professional medical training

2. Train specialist physicians and SpRs in how to use portfolios

3. Train specialist physicians and SpRs in how to conduct brief clinical assessments

4. Set up a structure to facilitate an effective and feasible learning environment for SpRs (e.g. Grand rounds/clinical conferences/journal club)

5. Investigate and highlight the professional competencies that could best be learnt in Curaçao.

6. Train SpRs and staff in effective feedback and teaching skills

Implementation of the project

The implementation process was preceded by a number of separate meetings held in the first week with different involved parties in the hospital. These included:

1. members of the pediatric staff;

2. SpRs;

3. director of the medical clerkship;

4. director of the hospital's medical services;

5. head of the Computer and information technology (CIT) department.

The purpose of the meetings was to acquaint the aforementioned parties with the objectives of the 16 week project. During the meetings, explanations were provided on the changes in the residency program, the importance of the new evaluation process and the proposed plan of action during the 16-week period. The meeting with the head of the CIT department was necessary as we intended to develop an electronic portfolio. In the second week, two training sessions were conducted to inform and train the staff and SpRs on (1) how to use Portfolios to register clinical evaluations and monitor individual progress and (2) carry out brief clinical assessments (See Box 2). In the weeks that followed, rosters were designed in which each resident was scheduled to have a brief clinical evaluation with a member of staff. This was scheduled for once every four to six weeks. It was expected that within the period of a year, each resident should have had at least 10 evaluations. The roster was made available to both the residents and members of staff and the first series of brief clinical assessments were carried out during the period of the project. In order to highlight the additional benefits of the clinical rotation in Curaçao, we conducted a questionnaire survey to investigate perceptions of the benefits of the residency training in an educational setting such as SEHOS among (1) the current SpRs and (2) past graduates who trained in SEHOS (40 in total) from 1985–2004. The questionnaire we designed contained 28 different items that reflected the seven core competencies as recommended by the CCMS for the Dutch medical specialists training program (see Table 1). The reliability of the questionnaire was tested in a separate pilot study and found to be high (Cronbach's alpha = 0.96; average measure intra-class correlation of the items = 0.96 (95% CI 0.92–0.99)).

Box 2 Content of the training session

1st session:

• Information on the current reforms in medical education and the recommendations by the pediatric society of the Netherlands

• Information on the role of portfolio learning in medical education and how to use it in clinical practice

• Training on how to provide effective feedback

2nd session:

• Description of the objectives and content of brief clinical assessments

• Demonstration of how to conduct brief clinical assessments

• Training on how to conduct and assess (self-)reflection exercises

Table 1.  Items used to assess the respondents perceptions of preparedness in the different competencies

Medical expert

1. Appropriately apply the profession's diagnostic, therapeutic and preventive arsenal and where possible, based on available evidence

2. Provide effective and ethically responsible health care service

3. Possess adequate knowledge and skills relevant for the specialty

4. Promptly find and appropriately apply the required medical information for optimal patient management

Communicator

5. Adequately discuss relevant medical information with the patient and if necessary with relatives

6. Obtain relevant and purposeful information from patients/guardians through good listening skills

7. Build effective professional relationships with patients

8. Adequately prepare verbal and written reports on patient cases

Collaborator

9. Contribute to an effective interdisciplinary cooperation and health care network service

10. Adequately refer patients to other health care givers

11. Purposefully communicate with colleagues and other health care providers

12. Effectively handle inter-collegial consultations

Scholar

13. Develop and maintain my personal continuing medical education

14. Promote the professional abilities of students, colleagues, and other partners in the health care system

15. Adequately develop and expand my scientific/professional knowledge

16. Critically appraise medical information

Health advocate

17. Perform my duties in line with the relevant (local) judicial guidelines

18. Promote the general well being of the patient and community as a whole

19. Adequately respond to health care related incidents

20. Recognize and identify the determinants of illness and disease

Manager

21. Responsibly utilize the available resources for patient management

22. Work effectively and intentionally within the health care organization

23. Organize my work to achieve a balance between patient care and personal development

24. Use information technology for optimal patient care as well as for my own continuing professional training

Professional

25. Exhibit adequate personal and interpersonal professional conduct

26. Identify the limitations of my own professional competency and conduct

27. Practice medicine in line with the ethical norms customary to my profession

28. Deliver health care services reliably, truthfully and empathetically

(Scale: 0 = not applicable; 1 = totally disagree; 2 = disagree; 3 = agree; 4 = totally agree.

Results

Implementation of the core changes in the curriculum

At the end of the 16 week period, the department had been informed about the changes in the curriculum of the paediatric residency training, a new concept of ‘competency-based’ evaluation had been successfully introduced, and the implementation of the new methods of evaluation was initiated i.e. brief clinical assessments. A portfolio was designed based on the draft advised by the CCMS (framework for the Dutch specialist training programs). The content was modified to suit the local educational context in Curaçao and the end result was provided digitally in CD-rom format. A demonstration on how to use the CD-rom was given to both the residents and members of staff. Following the introductory meeting in the first week of the project, a second meeting was held with the SpRs in the 12th week (focus group interview). The goal was to evaluate the SpRs’ experiences with the brief clinical assessments and to address any problems they may have had with the content or use of the CD-roms for the digital portfolio. Although the SpRs were enthusiastic about the idea of the digital portfolio, they expressed dissatisfaction with the amount and frequency of the different evaluations that needed to be carried out and registered. The feedback we obtained was used to adjust the frequency of brief clinical assessments i.e. from once per month to once every two months.

Assessment of the learning environment

The results of the survey we conducted revealed that the past graduates perceived being well prepared in the 7 professional competencies through their training in Curaçao. They claimed that the training was accountable for >70% of their total professional development. The professional competency they felt best prepared in was as health advocate. The five SpRs who were currently undergoing their training in Curaçao at the time of the survey disagreed with the assertion that the training prepared them well in the 7 professional competencies. They perceived being well prepared in all, but the competencies as manager, scholar and collaborator (see Table 2). There was no clear explanation for their disagreement from the survey although one can assume that certain external factors like the uncertainty around the continuation of the residency in Curaçao (at the time) and the perceived lack of adequate clinical supervision and educational activities e.g. grand rounds, symposia, journal clubs, were responsible for this. Still, the SpRs felt that the training accounted for about 40–50% of there professional development and that their role as medical experts was the competency best developed from the professional training. We concluded from the survey that both past en present trainees in Curaçao perceived being well prepared in the role as medical expert, however the current SpRs felt less prepared in their roles as manager, scholar, and collaborator. Based on the responses of our survey, we identified two additional competencies that required further improvement. Besides the competencies as manager, scholar and collaborator, we included the competencies health advocate and medical expert in our recommendations. These competencies were considered relevant considering the distinct nature of the educational climate in Curaçao (Busari et al. 2001).

Tabel 2 Paediatricians’ and specialist registrars’ perceptions of their level of preparedness in the different competencies

Competency	Respondents	N	Mean	SD	95% CI
Medical expert	Specialist registrars	5	2.70	0.27	2.36-3.04
	Paediatricians	17	2.78	0.55	2.50-3.06
Collaborator	Specialist registrars	5	2.40	0.42	1.88-2.91
	Paediatricians	17	2.87	0.52	2.60-3.13
Communicator	Specialist registrars	5	2.65	0.42	2.13-3.17
	Paediatricians	17	2.75	0.56	2.46-3.03
Scholar	Specialist registrars	5	2.30	0.63	1.44-3.16
	Paediatricians	17	2.88	0.70	2.52-3.24
Health advocate	Specialist registrars	5	2.58	0.55	1.90-3.27
	Paediatricians	17	3.00	0.50	2.74-3.26
Manager	Specialist registrars	5	2.40	0.38	1.93-2.87
	Paediatricians	17	2.72	0.66	2.38-3.06
Professional	Specialist registrars	5	2.63	0.41	2.11-3.14
	Paediatricians	17	2.99	0.52	2.72-3.25
Total level of preparedness	Specialist registrars	5	2.51	0.28	2.16-2.86
	Paediatricians	17	2.93	0.47	2.69-3.17

Scale: (1 = totally disagree, 2 = disagree, 3 = agree, 4 = totally agree).

General recommendations for improvement of the clinical rotation in Curaçao

As is known, the competency as medical expert is central to the SpRs function as a physician (CANMEDS 2005). SpRs are expected to be able to integrate the entire CANMEDS roles and apply medical knowledge, clinical skills, and professional attitudes in the provision of patient-centred care. As medical experts, SpRs should be able to demonstrate the professional skills appropriate for their individual level of training in ensuring good health care for their patients. The clinical rotation in Curaçao is an ideal educational setting for developing this competency, bearing in mind the influence the cultural climate of the community may have on this process. Expanding on this, the organizational climate and culture of a community has been found to influence the way different communities coordinate their activities (Hofstede 1980, 2001). Some of the mechanisms involved in this process have been described as power-distance (i.e. extent to which the community encourages superiors to exert power), uncertainty-avoidance, individualism (i.e. emotional dependence of individual on organization or institution) and masculinity (i.e. extent to which the culture places emphasis on performance and difference in the sexes, rather than to quality of life). Applying these descriptors to Curaçao as a community, the cultural climate reflects a large power-distance, masculinity and low individualism. This is reflected by a health care delivery system that is not well-structured, a rather paternalistic doctor-patient relationship and more autonomy resting with care providers than with the individual patient. In contrast, western communities like the Netherlands exhibit highly specialized health care systems with less evident paternalistic doctor-patient relationships and active participation of patients in clinical decision-making. Hence, in a community where patient autonomy is low and the responsibility for self is often placed in the hands of health care providers, the physician is obliged to develop good managerial and advocacy skills.

As a manager, SpRs would be expected to participate in activities that contribute to the effectiveness of their health care organization. It would entail that they are able to prioritize effectively, balance their professional and personal lives efficiently, execute tasks collaboratively with other health care providers and make systematic choices when allocating scarce healthcare resources. As health advocates, the SpRs should be able to use their expertise and influence responsibly, to advance the health and well-being of the individual patient and community. Considering the socio-demographic constitution of Curaçao, it is our view that the training in SEHOS provides a rich opportunity for the SpRs to recognize their duty and ability to achieve this. For example, the health care insurance scheme in Curaçao is run by the government as well as by private organizations. However, about 40% of the population is registered as ‘pro paupertatis’ (the less privileged) under the government-run insurance scheme. In practice, it means that a large number of patients would require the assistance of SpRs to assist them in navigating the healthcare system and access the appropriate health resources in an efficient and timely manner. For this, expert advocacy skills by the latter are necessary.

The training in Curaçao also facilitates the development of good collaborative skills. As collaborators, SpRs are expected to work effectively within health care teams characterized by professionals working closely together at one site and affiliated with other teams in multiple locations with a variety of perspectives and skills. Since the health care system in Curaçao is less well-structured and the demand for good health care is high, SpRs ultimately find themselves working in partnership with others who are involved in the care of their (or other) patients. Being able to collaborate effectively with the patients, families, and other expert health professionals in providing optimal care, education and scholarship is therefore essential. Finally, as scholars SpRs should be able to demonstrate a lifelong commitment to reflective learning, as well as the conception, dissemination, application and translation of medical knowledge. By engaging in a lifelong pursuit of mastering their domain of expertise, they can develop relevant skills (clinical, cognitive, communicative skills) and cognitions required for optimal professional development (Akhund 2006). Based on the experiences from the 16-week project, a few specific recommendations adapted to the local situation, have been made to further improve the implementation process in Curaçao. An overview of these recommendations is provided in Table 3.

Table 3.  Recommendations for improving a competency-based training programme for specialist registrars in a limited resource environment (Curaçao)

Competency	Recommendations
Medical Expert	1. Clearly pre-define performance indicators that can be used to monitor progress of the SpR before commencement of individual rotations
	2. SpRs should register ‘critical incident’ encounters with patients or peers in logbooks. A critical incident in this context refers to particular events in a working day that turned out differently than desired. They encompass a spectrum of the SpRs’ personal experiences, which can be used as material for teaching
	3. Documentation of the SpRs achievements/educational progress should be filed in portfolios
Health	The SpRs should:
Advocate/Manager	1. Gain a better understanding of the local primary health care facilities and the services they provide
	2. Be involved in promoting preventive health care services through health visits to schools and primary health care clinics
	3. Acquire a good understanding of the epidemiological patterns of common infectious diseases and health problems on the island e.g. HIV, dengue fever, teenage pregnancies and drug abuse.
	4. Participate in developing primary health care strategies to prevent common diseases and health problems.
	5. Coordinate the transfer and management of their patients among the different health care providers within and outside the hospital environment
Collaborator	1. Interdisciplinary grand-rounds should be organized 4 times in a year with other departments in the hospital (e.g. Obstetrics & Gynecology, Internal medicine, Surgery)
	2. SpRs should learn how the local preventive health care system works through organizing clinical seminars/meetings with representatives of (para)medical services in the community e.g. Social welfare worker, representatives of departments of microbiology or epidemiology, Infant welfare and Maternity clinics.
	3. SpRs should organize and participate in informative school seminars/meetings on common health care problems among adolescents
Scholar	The SpRs should:
	1. Participate in monthly Journal clubs, critical appraisal of topics or article reviews
	2. Deliver clinical presentations at interdisciplinary Grand rounds as well as in weekly departmental grand rounds
	3. Publish at least one research article/case report during the residency period
	4. Attend one or more national/international scientific conference each year
	5. Participate in yearly ‘state of the art’ symposiums that involve contributions from pediatric (sub-)specialists, and SpRs
	6. Own individual subscriptions with at least one online self-education program e.g. AAP Prep package, to keep their knowledge abreast of new developments.

Conclusion

This report describes the implementation of a revised competency–based training program for paediatric SpRs in a developing nation (Curaçao). It also shows how crucial it is to have the support of an educational expert in the local setting and how this facilitates the implementation of a comprehensive training program. Digital portfolios and brief clinical assessments have been effectively implemented in the department of paediatrics in SEHOS. Both residents and members of staff have been trained in how to conduct and interpret brief clinical assessments and make use of portfolios. ‘Departmental portfolios’ have been introduced as an innovative method of evaluating and monitoring the educational activities of the various clinical departments (see Table 4). An advisory committee comprising of the Director of medical clerkships, departmental heads of the residency training programs, a medical educationalist and a representative from the teaching hospital (UMCG) in Groningen, has been set up to monitor the progress and quality of the training program. A yearly (re-)evaluation of the competency based training process would take place, including additional trainings in relevant educational methods.

Table 4.  Model for developing a departmental portfolio: Pediatrics

Curriculum	Component	Comments/Conditions
Structure	• Out patient clinics	• Define the duration of individual rotations
	• General Paediatrics	• Discuss educational goals prior to and after rotations
	• Neonatology	• Idenify performance indicators beforehand
Content	• Clinical lectures/demonstrations	• Define topics to be treated beforehand and the frequency of lectures
	• Clinical Presentations (e.g., Critical appraisals of a topic/Case presentations/Literature reviews)	• Design a schedule for the presentations with the list of the presenters
	• Scientific/research output (case-reports/research studies)	• Agree on the number of presentations per individual
	• Conference/symposia attendance	• Agree on number of case reports/research papers to be produced during the program
	• Trainings/continual medical education (e.g. Advanced pediatric life support training, APLS)
	• Self study (individual, e-learning)
Evaluatioin	Medical residents:
	• Portfolio (formative/summative/databank)	• Define clear plan on the frequency and schedules for evaluation
	• Brief clinical evaluations	• Responsibility for evaluation should be shared between residents & staff
	• Self reflecton reports • Structured assessments	• Define how different evaluation methods would be used/combined to determine competency
	• Structured Exams
	• Mentorship
	• External evaluators
	Attending doctors:
	• Training in teaching skills	• Requires full participation of department staff
	• Evaluation of the quality of supervision	• Full participation of departmental staff
	• Departmental visitation	• External assessor should be familiar with departmental portfolios

The experience as described in this report shows that it is possible and also feasible to implement a competency-based program in an environment where resources are limited. As it turned out to be, the paediatric department in SEHOS was the first centre to achieve significant results in the implementation of The revised Dutch Paediatric Training Program (2008) (Muskiet et al. 2006), and demonstrated that careful and pragmatic approaches are necessary when implementing competency-based curricula in a setting like Curaçao. The process was also dependent on adequate information and approval of all concerned parties and with the support of an educationalist on-site. It is our view that the experience from this exercise can be helpful for other limited-resource countries that are dealing with the new trend of competency-based training.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Additional information

Notes on contributors

Jamiu O. Busari

JAMIU BUSARI, Paediatrician, Atrium Medical Center, Heerlen and Assistant Professor in Medical Education, Maastricht University, the Netherlands.

Eduard A. A. Verhagen

EDUARD VERHAGEN, Paediatrician and Chief-attending physician, Beatrix Kinderkliniek, University Medical Center Groningen, the Netherlands.

Fred D. Muskiet

FRED MUSKIET, Paediatrician and Director, Paediatric residency training program, St. Elisabeth Hospital, Curacao, Netherlands Antilles.

Ashley J. Duits

ASHLEY DUITS, Director, Institute of Medical Education, St. Elisabeth Hospital, Curacao and Professor in Medical Education, University of Groningen, the Netherlands.