Evaluation of a national process of reforming curricula in postgraduate medical education

Figures & data

Table

Practice points

When using a top-down strategy of implementing a reform, it is important that authorities are clear in messages and communication throughout the process. Traditions and values of both pedagogical experts and doctors regarding educational principles must be considered when launching new educational concepts. There is a delicate balance between pushing deadlines in a reform and allowing time to work with and adapt to new educational principles. Initiating part of a reform as pilot studies might be of help in clarifying elements of new concepts and how they work in practice.

Table 1.  Interview guide for the telephone interviews

Interview guide for telephone interviews
Open-ended questions
The NBH^1 has defined requirements regarding content and form of the curricula and communicated these requirements via guidelines and different amendments to them.
How did these requirements influence the curriculum development process in your specialty?
The NBH^1 has set the frame for the specialties’ work with the curriculum development, i.e. regarding economy, time and deadlines, information, advisors at NBH^1 and other support, seminars, etc.
How did this frame influence the development process in your specialty?
Please describe the curriculum development process in your specialty?
Did educational culture or attitudes to specialist education change in your specialty during the process of curriculum development?
If yes, which change did you experience?
Please describe the most important positive and negative experiences you had in relation to the process.

¹NBH = The National Board of Health.

Table 2.  Opinions (mean, SD) about the elements of the reform and the process of developing new curricula expressed in the two rounds of the questionnaire survey

	Round 1 (N = 63)	Round 2 (N = 52)	Paired samples t-test	Effect size
	Mean (SD)	Mean (SD)	P	ES
Opinions about the concepts introduced by the reform
My specialty will benefit from the increased focus on educational aspects induced by ‘Guidelines for writing curricula’	3.8 (0.9)	3.9 (1.0)	0.272
It is an advantage that the new curricula will include minimum required competences	3.7 (1.3)	3.9 (1.2)	0.233
It is an improvement to the education that all learning goals have to be assessed	4.0 (1.1)	4.3 (0.9)	0.140
It is an improvement to the education that assessment will have to be performed for all learning goals regarding the seven roles of the doctor	3.5 (1.3)	4.0 (1.1)	0.006	−0.44
It is an improvement to the education that assessment in the future will have consequences	4.4 (0.9)	4.7 (0.6)	0.047	−0.30
I would myself have difficulties to end a young doctor's career because of lacking fulfilment of learning goals	2.8 (1.4)	2.6 (1.2)	0.366
Considerations of the learning environment are relevant for taking into account in the development of the new curriculum	4.0 (1.0)	4.2 (1.0)	0.245
Opinions about the task of developing curricula
The specialties are the appropriate ones to take care of developing the curricula	4.8 (0.5)	4.9 (0.3)	0.110
One of the biggest challenges in the curriculum development process is to formulate strategies for learning and assessment	3.8 (1.3)	3.8 (1.2)	0.960
One of the biggest challenges in the curriculum development process is to define an appropriate number of learning goals for the specialty	3.8 (1.1)	3.4 (1.2)	0.018	0.33
One of the biggest challenges in the curriculum development process is to define an appropriate level of detail for each learning goal	4.1 (0.8)	3.9 (0.8)	0.093
One of the biggest challenges in the curriculum development process is to formulate learning goals within all seven roles of the doctor	3.4 (1.1)	3.2 (1.0)	0.057
One of the biggest challenges in curriculum development process is to stick to the scheduled deadlines	3.8 (1.2)	3.3 (1.3)	0.014	0.38
I found that lacking knowledge about the content of ‘The Specialist Commission Recommendation Report’ in my specialty is a problem in relation to the new curriculum	3.3 (1.3)	3.3 (1.3)	0.835
The present educational culture in my specialty will impede the implementation of the educational approach in ‘Guidelines for writing curricula’	3.2 (1.3)	2.9 (1.1)	0.054
It is/will be a big challenge to start the necessary dialogue and change of educational practice in the specialty in relation to implementation of the new curriculum	4.5 (0.9)	4.5 (0.9)	0.589
The new curricula will contribute to showing that education requires resources	4.5 (0.7)	4.4 (0.9)	0.560
Opinions about the support in the development process
The curriculum development process is organised by the NBH^1 in a way that supports the work in the medical specialties	2.4 (1.1)	2.3 (1.1)	0.604
My specialty has had or will have benefit of our advisor at NBH^1.	3.1 (1.1)	3.2 (1.3)	0.781
The educational ambitions in ‘Guidelines for writing new curricula’ are appropriate to my specialty	3.3 (1.1)	3.3 (1.0)	0.740
‘Guidelines for writing curricula’ is written in a way that inspires me	2.9 (1.1)	2.7 (1.1)	0.405
The appendix in the Guidelines regarding learning goals is considerably easier to use than the appendices regarding teaching and assessment	3.2 (1.0)	3.5 (0.8)	0.095
The educational tools in the appendix regarding teaching make me think differently concerning learning methods	3.1 (1.0)	3.2 (1.1)	0.659
The theory regarding teaching in the Guidelines is difficult to understand	3.6 (1.0)	3.7 (1.0)	1.000
The theory regarding teaching in the Guidelines is difficult to use	3.8 (1.0)	3.8 (1.0)	0.863
The theory regarding assessment in the Guideline's appendix regarding assessment is easy to understand	3.1 (1.1)	2.8 (1.0)	0.412
The theory regarding assessment in the Guideline's appendix regarding assessment is easy to use	2.7 (1.0)	2.7 (0.9)	0.700
The educational tools in the Guideline's appendix regarding assessment make me think differently concerning assessment methods	3.1 (1.1)	3.3 (1.0)	0.274

Notes: Respondents indicated answers on a 5-point Likert scale (1 = disagree, 2 = partly disagree, 3 = neutral, 4 = partly agree, 5 = agree).

Table 3.  Results from telephone interviews of contact-persons working with curriculum development (N = 26). Examples of quotes are typed in italics

	Positive comments (n = number of persons)	Negative comments (n = number of persons)
Overall opinion of conditions for doing the work	Were good (13)	Were bad (2)
	‘The conditions have worked quite ok under the given circumstances’	‘The conditions were poor’
Written material including guidelines	Useful, supportive (18)	Difficult, strange terminology, not inspiring (4)
	‘The Guidelines worked a rule of conduct. They clarified the task’	‘It was very difficult for us to understand. It was a completely new and strange terminology’
Information from NBH		Unclear, changing, poor introduction
		(12)
		‘We had some frustrations because of changing directives from NBH. Perhaps not surprising since all this was new for NBH as well’
Advisor at NBH	Helpful	No benefit, inexpedient change of advisor
	(16)	(7)
	‘They were kind and helpful, although they were not always capable of giving useful advise’	‘Along the road we became more experienced than them’
Pedagogical support		Not qualified support
		(3)
		‘Some of the presentations at the seminar were useless theoretical anthropology’
Seminars	Helpful, inspiring	Limited benefit
	(13)	(5)
	‘It provided an opportunity to network and exchange ideas and problems’	‘We did not get any major benefit, it rather served to keep us on track’
Financial support	OK	Insufficient
	(7)	(5)
	‘No problems’	‘The financial support did not at all correspond to the amount of work we made’
Time frame	Deadlines OK	Lack of time, stressing
	(8)	(10)
	‘The tight deadlines were ok. If we had had the double time we would have taken the double time’	‘Tight deadlines made it difficult to do it as good as you wanted to’
Workload		Hard work, few persons
		(9)
		‘The fact that we were only a few persons made us feel overworked’
Dialogue in specialty	Constructive dialogue and back up	Poor back up and dialogue
	(16)	(3)
	‘The specialty has been engaged in the process. People were really supportive and enthusiastic’	‘It has been difficult to engage the specialty society’
Cultural changes in specialty as a result of the reform	Already happened (13)	No changes (yet) (14)
	‘People have realised that a change of attitudes is needed. The process has initiated a lot of reflections about learning and assessment’	‘Not yet. Maybe for a few of us. It will not come until the implementation’
Development in specialty	The process contributed to develop and define the specialty
	(12)
	‘We have had a positive discussion about clarifying the future identity of our specialty’
Impact on quality of education	Positive impact
	(11)
	‘The new curriculum will improve the quality of education’
Change in personal attitude towards education	Positive change of attitude to and thoughts about education
	(10)
	‘I changed my perception of my specialty and what we can expect from a trainee’
Personal experience	Benefit
	(13)
	'Interesting, exciting, funny. I have gained an enormous insight in education’