Development of an instrument (the COLT) to measure conceptions on learning and teaching of teachers, in student-centred medical education

Abstract

Background: Conceptions of medical teachers regarding learning and teaching affect their teaching practice. Therefore conceptions should be addressed in faculty development.

Aim: To facilitate this, we constructed the Conceptions Of Learning and Teaching (COLT) instrument.

Method: The COLT was adapted based on experts’ comments during a meeting and interviews, followed by a Delphi procedure (Part I). It was administered to teachers from two Dutch medical schools with different traditions in student-centred education (Part II; N = 646). The data were analyzed using confirmatory factor analysis and reliability analysis.

Results: 324 Teachers (50.2%) completed the questionnaire. Confirmatory factor analysis did not confirm the underlying theoretical model, but an alternative model demonstrated a good fit. This led to an instrument with eighteen items reflecting three underlying factors: ‘teacher centredness’, ‘appreciation of active learning’, and ‘orientation to professional practice’. We found significant differences in COLT scores between the faculty of the two medical schools.

Conclusions: The COLT appears to be a construct valid tool resulting in reliable scores of teachers’ conceptions of learning and teaching, in student-centred medical education. Two of the three factors are new and may be specific for student-centred medical education. The COLT may be a promising tool to improve faculty development.

Introduction

Conceptions of learning and teaching were described by Pratt as specific meanings attached to phenomena which can act as a filter through which new information passes as it is processed (Kagan 1992; Pratt 1992). Thus medical teachers’ consciously or unconsciously held conceptions affect their views of education. Conceptions should be distinguished from beliefs (Fishbein & Ajzen 1975) and perspectives (Pratt et al. 2001; Pratt 2001) related terms which are often used in the literature but encompass more aspects than conceptions do. Beliefs also include confidence in one's ability to affect student performance (teacher efficacy), nature of knowledge (epistemological beliefs) and causes of student performance. They are also formed early, often implicit and sometimes deeply rooted (Pajares 1992). Perspectives are a combination of beliefs, intentions and perceived actions.

Several authors have argued that teachers’ beliefs or conceptions influence their teaching approaches (Pajares 1992; Trigwell et al. 1999; Mathijsen 2006) and some researchers have suggested that teachers’ conceptions of learning and teaching indirectly affect students’ learning outcomes (Gibbs & Coffey 2004). Some authors even go so far as to argue that if no attention is paid to teachers’ conceptions in faculty development, changes in teaching behaviour are unlikely to be achieved (Kember & Kwan 2000; Postareff 2007). Assuming that conceptions of learning and teaching have an impact on education, it seems logical to incorporate them in faculty development. In recent years faculty development has received increasing attention (Steinert et al. 2006). Many studies have addressed its effectiveness in terms of achievement (Baroffio et al. 1999; Gibbs & Coffey 2004; Postareff et al. 2008) and transfer of training results to day-to-day teaching practice (Cheng & Hampson 2008). Medical education research pays little attention to teachers’ conceptions of learning and teaching (Williams & Klamen 2006; Taylor et al. 2007; Peeraer et al. 2011) and more particularly their role in faculty development. We therefore think that measuring teachers’ conceptions of learning and teaching could yield insights that can be useful for faculty development.

A literature search resulted in some instruments to measure teachers’ conceptions or perspectives of learning and teaching, developed in secondary and higher education research (Bolhuis 2000; Pratt et al. 2001; Bolhuis & Voeten 2004; Oolbekkink-Marchand et al. 2006; Donche et al. 2007) see Table 1.

The work done by Bolhuis (2000) offers an interesting perspective and theoretical framework (Bolhuis & Voeten 2004). In a study of educational reform towards more independent learning in secondary education, Bolhuis developed a framework of conceptions of learning and teaching with five underlying factors: (1) regulation of learning, (2) view of knowledge, (3) collaborative learning, (4) view of intelligence, and (5) teachers’ tolerance of uncertainty. Regulation of learning refers to regulation of learning processes. When the learner takes the lead (internal regulation), learning results appear to be better than when the teacher takes the lead (external regulation). View of knowledge distinguishes between knowledge-as-construction (Shuell 1988) versus knowledge as an objective, stable entity that must be acquired by learners. Collaborative learning focuses on learning as a social activity versus individual learning. View of intelligence distinguishes between intelligence as a stable attribute of learners and intelligence as a dynamic, developing attribute. Teachers’ tolerance of uncertainty refers to their ability to deal with new and unpredictable situations. It influences all other factors (Bolhuis & Voeten 2004).

Bolhuis developed the Learning Inventory based on a multi-factor framework. This framework contrasts with other studies on teaching approaches and conceptions of teachers in higher education, which usually address only two components: (1) ‘teacher centredness/focusing on knowledge transmission’ and (2) ‘student centredness/focusing on conceptual change in students’ (Kember & Kwan 2000; Trigwell & Prosser 2004; Postareff & Lindblom-Ylänne 2008). Nevertheless, we concluded that despite the relevant theoretical background of the Learning Inventory, it was not suitable for medical education mainly because of the wording of the items. The Inventory of Teaching Patterns was also considered not suitable, because it is tailored to teacher education (Donche et al. 2007) and the same applies for the Approaches to Teaching Inventory (ATI), which focuses on teaching approaches rather than teachers’ conceptions of teaching (Trigwell et al. 1999; Trigwell & Prosser 2004). Also the wording of the ATI items is not suitable for student-centred medical education, because its focus is on lecturing rather than on small-group student-centred educational formats (Trigwell 2009).

Student-centred medical curricula are gaining ground worldwide (Dolmans et al. 2005). Moving from traditional to student-centred education places increased emphasis on teacher-student interaction, which implies a change in the role of the teacher (De Grave et al. 1999). We therefore decided to construct a new instrument, Conceptions of Learning and Teaching (COLT), to measure the conceptions of learning and teaching of teachers in student-centred medical curricula.

Our research questions were: (1) Does the COLT have internal validity? Does it have a coherent internal factor structure representing a theoretical construct? (2) Does the COLT result in reliable scores? (3) Is there evidence of external validity? More specifically, can the instrument distinguish between conceptions of teachers from two medical schools with different traditions in student-centred education?

In order to answer question 1, we conducted a modified Delphi procedure (Part I), asking coordinators of faculty development to evaluate the statements of the instrument. We then administered the questionnaire to a large group of teachers of two Dutch medical schools, Maastricht University Medical Centre (MUMC), Maastricht and VU university medical centre (VUmc), Amsterdam (Part II). In order to obtain psychometric evidence of the internal validity of the COLT, we performed confirmatory factor analysis (CFA) to determine whether the data fitted the assumed model based on the framework of the Learning Inventory (Arbuckle 2006). In order to answer question 2, we estimated reliability coefficients. Question 3 was addressed by comparing the mean scores of the teachers from the two participating medical schools. We hypothesised a difference between the two schools. In MUMC there is a long-standing tradition in student-centred medical education. Since 35 years, with the founding of the university, a problem-based curriculum in Medicine was introduced and ever since teachers are used to this approach. In VUmc a student-centred curriculum was introduced in 2005, replacing a classical teacher-centred curriculum. Both curricula are characterized by small group learning activities as the core of the curriculum.

Table

Practice points

Conceptions of medical teachers on learning and teaching currently receive few attention in medical education. We constructed an instrument (the COLT) to measure these conceptions in student-centred medical education and describe its construction process and psychometric properties. Ultimately we are interested in the role conceptions of teachers should play in faculty development.

Methods

This section starts with the description of the participants and setting of our study. Afterwards we will focus on the procedure and in more detail, the two parts of our study to answer the research questions. In part I we describe the Delphi procedure, followed by part II in which we tested the questionnaire. We conclude the Method section with a description of the analysis.

Participants and setting

Before we conducted the Delphi procedure, we invited three experts who constructed three questionnaires mentioned in Table 1. They finished a PhD study on conceptions of teachers on learning and teaching. With them we discussed the aim of our study and the choice for an instrument. Afterwards we held interviews with six teachers with different backgrounds from the two medical schools (two clinicians, two basic scientists, one psychologist and one educationalist).

Table 1.  Overview of questionnaires reported in literature

Name of questionnaire	Authors	Measuring	Context	Notes
Learning Inventory	Bolhuis (2000); Bolhuis and Voeten (2004)	Conceptions of own learning and student learning	Secondary education; The Netherlands	For both own learning and student learning, each teacher is rated on 5 factors: (1) regulation of learning, (2) view of knowledge, (3) learning as social or individual activity, (4) view of intelligence, and (5) teacher's tolerance of uncertainty
ATI: Approaches to Teaching Inventory	Trigwell et al. (1999)	Approach to teaching (self report)	University teachers in physics and chemistry, Australia	Each teacher receives scores on two components: (1) conceptual change/student-focused [CCSF] approach and (2) information transmission/teacher focused [ITTF] approach
ATI plus modified MSLQ (Motivation Strategies for Learning Questionnaire)	Postareff (2007)	(1) Approach to teaching (self report) and (2) self-efficacy beliefs of teachers	Teachers from all 11 faculties of the University of Helsinki, participating in teacher development programmes, Finland	For each teacher a CCSF score, ITTF score and self-efficacy score are calculated. Scores were related to extent of pedagogical training
(no name)	Oolbekkink-Marchand (2006)	Perspectives	Secondary education and university teachers; in science education, The Netherlands	Every teacher is categorised in one of three perspectives: (1) development oriented and shared regulation; (2) knowledge oriented and strong regulation; (3) opinion oriented and loose orientation
Inventory of teaching patterns	Donche et al. (2007)	Conceptions (1) of own learning, (2) of student learning and (3) approach to teaching (self report)	Bachelor programmes, Belgium; in medical education in Belgium, by Peeraer et al. (2011)	For each teacher a score on the three components.
TPI: Teaching perspectives inventory	Pratt et al. (2001)	Perspectives on learning, teaching and knowledge: a combination of actions, intentions and beliefs	Freely available on the Internet; several countries, mainly USA	Each teacher is categorised in one of five teaching perspectives: (1) transmission, (2) apprenticeship, (3) nurturing, (4) developmental or (5) social reform

Subsequently we invited all members of the NVMO special interest group on Faculty Development (N = 9) in The Netherlands to participate in the Delphi procedure (Part I). This group coordinates faculty development activities in all medical schools, the faculty of veterinary medicine, and the faculty of dentistry in The Netherlands.

In Part II we administered the questionnaire to all teachers who were involved in at least one of three teaching activities (tutorials, lectures or practical classes) in the bachelor programmes of the undergraduate medical curricula of VUmc (N = 371) and MUMC (N = 275).

The Ethical Review Committee of VUmc advised positively on the feasibility of this study. In the e-mail to the Delphi panellists and the teachers we explicitly stated that participation was voluntarily and that full anonymity was guaranteed.

Procedure

The COLT was constructed in a stepwise process (Table 2). The first step was an expert meeting in which the aim of our study and the choice for an instrument was discussed with three experts. On the experts’ advice we modified the Learning Inventory by splitting the factor ‘regulation of learning’ into ‘internal regulation’ and ‘external regulation’. This resulted in six underlying factors. The experts also proposed changing the format of the items from a forced choice between opposite statements to single statements with Likert scales. This increased the number of items from 25 to 50. Thirdly, they advised us to tailor the wording of all statements to medical education while maintaining the underlying factor structure and to include additional statements about students’ affective (e.g. motivation), cognitive and metacognitive activities.

Table 2.  Steps in the development of the COLT

Step	Result
1. Literature search	Learning Inventory model: secondary education; five factors: regulation of learning, view of knowledge, collaborative learning, view of intelligence, tolerance of uncertainty; 25 double items.
2. Expert meeting	Factor regulation was split into internal and external regulation (total of six factors); change of opposite statements to single statements: total of 50 items; wording adjusted to medical context; addition of items on affective, cognitive and metacognitive activities: total of 55 items.
3. Try-out interviews (six teachers)	Removal of factor ‘tolerance of uncertainty’ (total of five factors; 49 items); improvement of wording of statements.
4. Modified Delphi procedure (two rounds; N = 9, N = 8)	Five items considered not relevant, including factor ‘View of Intelligence of Students’; Rewording of 14 of 49 items.
5. Testing questionnaire (N = 324/646)	CFA showed good fit of alternative four-factor model; Reliability analysis; T-test to compare teachers’ conceptions between two medical schools. The four-factor model consists of 20 items, one of them is ‘view of intelligence’ with two items.
6. Combining results Delphi procedure and CFA	Removal of factor ‘view of Intelligence’ (two items), based on outcome of Delphi procedure, content-related and psychometric grounds; Definitive COLT: 18 items, three underlying factors; (1) teacher centredness, (2) appreciation of active learning, (3) orientation to professional practice.

The advice of the experts resulted in a list of 55 items (with five-point Likert scales: 1 = strongly disagree, 5 = strongly agree) reflecting six underlying factors: internal regulation (eight items), external regulation (nine items), view of knowledge (nine items), collaborative learning (nine items), view of intelligence of students (five items), the teacher's tolerance of uncertainty (six items) and affective, cognitive and metacognitive activities of students (nine items). For the purpose of this study we added 12 statements (part I of the questionnaire) to obtain background information about the respondents.

Subsequently interviews were conducted with teachers from the two medical schools. We invited three teachers with different backgrounds from each medical school and all participated. They judged the completeness of the instrument and the relevance and wording of the statements. Saturation of information was reached after six interviews. The interviewees unanimously recommended removal of the statements relating to the factor ‘the teacher's tolerance of uncertainty’. This resulted in 49 items and five underlying factors. The interviewees considered the instrument to be complete and made some suggestions for changes in wording.

The next step was a modified Delphi procedure (Part I) (Jones & Hunter 1995). The participants were not asked to generate questionnaire items but to judge the relevance and wording of the items resulting from the prior steps. The aim of the Delphi procedure was to obtain consensus on the relevance and wording of each item. The participants rated the statements on a five-point Likert scale (1 = not relevant/poor wording, 5 = very relevant/very clear wording). After the first round, mean scores and standard deviations were computed and in the second round the experts were given the results of the first round and asked to rate the statements again.

Testing the questionnaire (Part II). After the questionnaire was modified in response to the Delphi procedure, a web-based version (using Netquestionnaires™) was sent to all teachers involved in the bachelor programmes of the two participating medical schools. Data collection started in November 2009 and ended in April 2010. Non-responders received one e-mail reminder followed by one reminder by telephone or letter. A small gift (booklet) was sent to the participating teachers.

Analysis

First research question: does the COLT have internal validity?

Part I: Delphi procedure

We computed means and standard deviations for each item of the questionnaire (Holey et al. 2007). If the mean score on wording was < 3.5, we changed the wording and if the mean score on relevance was <3.5, we removed the statement.

Part II: Testing the questionnaire

We performed a confirmatory factor analysis (CFA) to test the theoretical model of our questionnaire, a six-factor model based on Bolhuis’ Learning Inventory and thus determine its internal validity. A CFA results in several indices (see Table 4). A model fit is assumed if the majority of seven conditions were met: (1) chi-square divided by degrees of freedom (CMIN/DF) is <3 with a non-significant p-value (Kline 1998); (2) the root mean square error of approximation (RMSEA) is <0.1 (Browne & Cudeck 1993); (3) the Tucker-Lewis coefficient (TLI) and (4) the comparative fit index (CFI) are both >0.90 (Bentler 1990); (5) the root mean square residual (RMR) is <0.07; (6) the goodness-of-fit index (GFI) and (7) the adjusted goodness-of-fit index (AGFI) are both >0.80. The last three indices were specified by Saris and Stronkhorst (1984).

Second research question: Does the instrument result in reliable scores?

In order to estimate the reliability of the measurements we calculated Cronbach's α for each factor using SPSS version 17.

Third research question: Is there evidence of external validity of the COLT? Or: can this instrument distinguish between conceptions of teachers from two medical schools with different traditions in student-centred education?

We calculated means and standard deviations for each factor of the definitive instrument for each individual teacher and performed a Student's T-test to compare the mean scores of the two medical schools.

Results

First research question: Does the COLT have internal validity?

Part I: Delphi procedure

All members of the special interest group (N = 9, response rate 100%) took part in the first round of the Delphi procedure. The first round led to rewording of three of the 49 statements. Eight members (89%) participated in the second round, which resulted in consensus. The results of the second round provided clear information on the relevance and wording of the statements. The panel judged that five statements were not relevant. These statements included the two statements referring to the factor ‘view of intelligence of students’.

Part II: Testing the questionnaire

The response rate to the electronic questionnaire was 50.2% (N = 324/646; VUmc 50.9%, N = 189/371; MUMC 49.1%, N = 135/275). Missing values (N = 10) were excluded. Responders and non-responders were comparable on gender and discipline (Table 3) both for the whole group and for the two medical schools separately.

Table 3.  Teachers’ background characteristics, in percentages, for respondents and non-respondents and for the whole group and by medical school

	VUmc		MUMC		Total sample
	Total group (N = 371)	Respondents (N = 189)	Total group (N = 275)	Respondents (N = 135)	Total group (N = 646)	Respondents (N = 324)
Gender
Male	54.7	51.3	54.5	55.6	54.6	53.1
Female	43.7	48.7	36.4	40.0	40.6	45.1
Not specified	1.6	0	9.1	4.4	4.8	1.9
Total	100	100	100	100	100	100
Discipline
Clinical specialties:*	62.3	51.2	54.6	48.8	58.9	50.3
General medicine	28.9	24.3	21.5	17.8	25.7	21.6
Surgery	12.7	11.1	12.7	9.6	12.7	10.5
Psychiatry	4.3	2.6	13.1	8.1	8.0	4.9
Supportive	16.4	13.2	7.3	13.3	12.5	13.3
General Practice & Public Health	14.3	18.0	6.9	10.4	11.1	14.8
Basic Sciences	12.4	15.3	11.6	19.3	12.1	17.0
Psychology	3.5	5.8	3.3	1.5	3.4	4.0
Skills laboratory	1.1	2.1	5.1	8.1	2.8	4.6
Other	2.2	2.1	8.4	12.6	4.8	6.5
Total	100	100	100	100	100	100

Note: *General Medicine includes Internal Medicine, Cardiology, Neurology, Paediatrics and Dermatology.

Surgical specialties: General Surgery, Orthopaedic Surgery, Urology, Obstetrics and Gynaecology, Ophthalmology.

Supportive specialties: Pathology, Radiology, Microbiology and Anaesthesiology.

Table 4.  Goodness-of-fit indices of confirmatory factor analysis

	CMIN/DF	RMSEA	TLI	CFI	RMR	GFI	AGFI
Tested six factor model (based on Learning Inventory)	8.199; p = 0.000*	0.150	0.386	0.632	0.774	0.920	0.813
Alternative model, total group	1.689; p = 0.185	0.046	0.959	0.986	0.175	0.995	0.974
Alternative model, subgroup VUmc (N = 189)	0.220; p = 0.802	0.000	1.117	1.000	0.074	0.999	0.994
Alternative model, subgroup MUMC (N = 135)	2.365; p = 0.094	0.101	0.775	0.925	0.354	0.983	0.916
Conventional cut-off for adequate fit	<3; not significant	<0.1	>0.90	>0.90	<0.07	>0.80	>0.80

Notes: CMIN/df, chi-square divided by degrees of freedom; RMSEA, root mean square error of approximation; TLI, Tucker-Lewis Index; CFI, comparative fit index; RMR, root mean square residual; GFI, goodness of fit index; AGFI, adjusted goodness of fit index.

*in bold is indicated which requirements were met.

Table 5.  Definitive instrument Conceptions of Learning and Teaching (COLT)

Each item is rated on a five-point Likert scale (1 = strongly disagree, 5 = strongly agree)
Factor 1: Teacher Centredness
1. Students should first master basic science knowledge before they can formulate their own learning goals.
2. I think that in small group learning the tutor determines what the students should learn, instead of the students determining their own learning goals.
3. Students learn best when the learning process is guided by an expert who has an overview of the field of interest.
4. When students discuss a topic without a tutor being present, they do not know at the end of the session if the questions have been answered correctly.
5. There is a logical sequence to learning.
6. As a teacher I have to indicate clearly what is important and what is less important for the students to know.
7. I think that as an expert in my field I am eminently suitable to transmit my knowledge to students and that students should not have to look up that knowledge for themselves.
8. When students collaborate, they teach each other the wrong things.
Factor 2: Appreciation of Active Learning
9. Students learn a great deal by explaining subject matter to each other.
10. Learning materials and teaching should invite students to come up with examples to illustrate the subject matter.
11. Small group learning motivates students to study.
12. I think it is more important for students to be able to analyse and critically appraise subject matter than to memorise facts.
13. I think it is important that students advise each other about the best ways to study.
Factor 3: Orientation to Professional Practice
14. I think it is important that educational assignments are derived as much as possible from the students’ future professional practice.
15. Being introduced to the day-to-day practice of their future profession motivates students to learn.
16. It is a good learning outcome when students demonstrate that they can apply their knowledge during activities in situations in professional practice.
17. I think that interactions between me and the students are an important aspect of my teaching.
18. Discussing topics with each other helps students to learn how to deal with different points of view, so as to gain a deeper understanding.

Note: A professional translator rendered the original Dutch questionnaire into an English version. Another professional translator translated the questionnaire back into Dutch. This version was comparable to the original version.

Table 6.  Descriptive statistics

	N items	Cronbach's α	Mean (SD); Total group (N = 324)	Mean (SD); VUmc (N = 189)	Mean (SD) MUMC (N = 135)	T-test (Sig. two-tailed)	Effect Sizes (Cohen's d)
Factor 1: Teacher centredness	8	0.73	3.43 (0.56)	3.56 (0.50)	3.25 (0.59)	t = 5.048 (p = 0.000)	0.57
Factor 2: Appreciation of active learning	5	0.57	3.84 (0.53)	3.75 (0.50)	3.98 (0.54)	t = −3.978 (p = 0.000)	0.44
Factor 3: Orientation to professional practice	5	0.63	4.23 (0.46)	4.14 (0.47)	4.36 (0.43)	t = −4.132 (p = 0.000)	0.49

Note: Number of items, reliability coefficients, means (Likert scale: 1 = strongly disagree, 5 = strongly agree) and standard deviations for total group and the two subgroup. The results of t-tests comparing the subgroups and effect sizes (Cohen's d) are also presented. (D ≈ 0.20 is defined by Cohen as a small, negligible effect; d ≈ 0.50 as medium, moderate effect; and d ≈ 0.80 as a large effect of crucial importance).

Based on the Delphi procedure we modified the instrument, but maintained the two items referring to the factor ‘view of intelligence of students’. The modified Learning Inventory was not supported by the CFA, but an alternative model showed a good fit with four factors: ‘teacher-centredness’ (eight statements), ‘appreciation of active learning’ (five statements), ‘orientation to professional practice’ (five statements), and ‘view of intelligence of students’ (two statements). This reduced the questionnaire from 49 to 20 items. The four-factor model also showed a good fit when it was tested for each medical school separately, although the fit was better for VUmc than for MUMC (Table 4).

Combining the results of the Delphi procedure and the confirmatory factor analysis

The Delphi procedure and the CFA yielded contradictory results for ‘view of intelligence of students’. The Delphi panellists concluded that the two related statements were not relevant but the factor was included in the four-factor model that resulted from the CFA. We nevertheless decided to remove the factor, based on the argument that teachers’ conceptions of students’ intelligence are less important in university education compared to secondary education. The factor originated from the Learning Inventory, which was developed for secondary education where intelligence shows more heterogeneity than among medical students (aged 18–23 years) where it is usually high and relatively homogeneous. Also, university teachers have less intensive contact with students than teachers in secondary education. On top of that a two-item factor is psychometrically questionable. In Table 5 we present the definitive instrument to measure conceptions of learning and teaching of teachers in a student-centred undergraduate medical programme: the COLT.

The factor ‘teacher centredness’ correlated negatively with the two other factors (−0.362 and −0.229 respectively, p = 0.001). The factors ‘appreciation of active learning’ and ‘orientation to professional practice’ correlated positively (0.394, p = 0.001).

To summarize, based on the Delphi procedure and the CFA, we conclude that the definitive questionnaire can be considered to have internal validity based on the consensus of the Delphi panel and the coherent structure of the underlying factors.

Second research question: Does the instrument result in reliable scores?

Cronbach's alpha was 0.73 for the factor ‘teacher centredness’, 0.57 for the factor ‘appreciation of active learning’ and 0.63 for the factor ‘orientation to professional practice’ (Table 6).

Third research question: Is there evidence of external validity of the COLT? Or: can this instrument distinguish between conceptions of teachers from two medical schools with different traditions in student-centred education?

In Table 6 means and standard deviations are presented for the three factors of COLT for all the teachers and for VUmc and MUMC separately. Student's T-test showed that overall VUmc teachers rated ‘teacher centredness’ significantly more highly, while MUMC teachers gave significantly higher ratings on ‘appreciation of active learning’ and ‘orientation to professional practice’. Effect sizes are also presented, and these indicate moderate effects. The ability of the COLT to distinguish between teachers of two medical schools supports the external validity of the instrument.

Discussion

We described the development of an instrument to assess the conceptions of learning and teaching of teachers in two medical schools with student-centred curricula. We assessed the internal validity of the instrument, its quality in producing reliable scores and sought evidence of its external validity.

Even though teachers are only partly conscious of their conceptions of learning and teaching, these conceptions influence their teaching practice (Pajares 1992; Trigwell et al. 1999) and therefore deserve more attention in faculty development. As far as we know no other studies of conceptions of teachers have specifically focused on student-centred education. Because student-centred education differs from other educational approaches in its increased emphasis on student–teacher interaction (De Grave et al. 1999), we considered it appropriate to develop a new instrument tailored to student-centred education. We used the Learning Inventory as developed by Bolhuis (2000) as the starting point for instrument development and adapted and tested the instrument for use in medical education.

The development and testing steps resulted in the COLT, an instrument to measure teachers’ conceptions of learning and teaching, consisting of an 18-item questionnaire reflecting three underlying factors: ‘teacher centredness’, ‘appreciation of active learning’, and ‘orientation to professional practice’. ‘Teacher centredness’ reflects two components that are frequently described in the literature: (1) ‘teacher centredness/focusing on knowledge transmission’ as opposed to (2) ‘student centredness/focusing on conceptual change in students’ (Trigwell & Prosser 2004; Postareff & Lindblom-Ylänne 2008). ‘Appreciation of active learning’ and ‘orientation to professional practice’ are new compared to earlier research on conceptions of (Bolhuis & Voeten 2004; Donche et al. 2007) and approaches to teaching (Trigwell et al. 1999). We assume that these two factors are specific for student-centred education and medical education, respectively.

As we had hypothesised based on the schools different traditions in student-centred education, the results showed a difference between the conceptions of teachers from the two participating medical schools. This appears to support the instrument's external validity. VUmc teachers gave higher ratings on ‘teacher centredness’, while MUMC teachers did so on ‘appreciation of active learning’ and on ‘orientation to professional practice’.

We expect that the COLT can have an indirect effect on quality improvement of student-centred medical education. For individual teachers the results may promote reflection (Schön 1983) and possibly a change of conceptions. From a faculty development perspective, insight into teachers’ conceptions can help to better align faculty development with the needs of individual teachers. Discussing conceptions of learning and teaching may help to achieve changes in teaching behaviour as a result of faculty development (Kember & Kwan 2000; Postareff 2007). However, further studies are necessary to reveal the relationship between conceptions on teaching and faculty development. Do conceptions deserve attention before training teachers’ teaching skills, or can we start with training teaching skills and will conceptions change subsequently as a result of the teachers’ experience in daily practice (Devlin 2006)? Furthermore, the COLT may be useful for curriculum changes towards student-centred curricula, because for successful curriculum change it is important that teachers’ conceptions are in alignment with the conceptions underlying the new curriculum.

Strengths and limitations

We used several strategies to construct a valid instrument resulting in reliable scores of conceptions of learning and teaching of teachers in student-centred undergraduate medical curricula. Different stakeholders contributed to the process. Despite a somewhat moderate response to our questionnaire (50.2%) and the participation of only two medical schools, we think the design of our study was sufficiently robust to produce a sound instrument.

A limitation of our study with respect to the third research question (Is there evidence of external validity of the COLT?) might be caused by a heterogeneity in conceptions of teachers in VUmc, by the time of the study. Due to the fairly recent introduction of student-centred education, a few of the more long-standing faculty may have teacher-centred conceptions on learning and teaching. We explored this and identified eight subgroups of teachers in both medical schools, by dichotomizing their scores (above or below the mean score on each of the three factors). For example subgroup 1 contains teachers who score low on ‘teacher centredness’, high on ‘appreciation of active learning’ and high on ‘orientation to professional practice’. However, when computing the mean scores for all teachers of each medical school, we see that VUmc teachers rated ‘teacher centredness’ significantly more highly, while MUMC teachers gave significantly higher ratings on ‘appreciation of active learning’ and ‘orientation to professional practice’. To our opinion, these findings support the conclusion that there is evidence for the external validity of our instrument.

Another limitation of our study, which is common in studies using written questionnaires among large groups, is the possibility that the answers were influenced by social desirability. We tried to counteract this by including positively and negatively phrased statements in the questionnaire.

Implications for future research

Larger scale studies are recommended to further examine the external validity of the COLT and its quality to produce reliable scores. It is also important to conduct studies where the instrument is used in traditional medical schools to establish standards of conceptions of teachers on learning and teaching. Ultimately, we think that the use of this instrument will enable researchers to pursue a new direction in studies into faculty development, teacher behaviour and human resource development in medical education.

Acknowledgements

The authors thank all participants in this study: the experts, the teachers who participated in the interviews, the Delphi panel and the 324 teachers from VUmc and MUMC who filled out the questionnaire. We are also grateful to Mereke Gorsira for her support in editing this paper's English. Furthermore we thank Antoinette Vestdijk, Gabby Reinders and Louise van der Neut for their contribution to the data collection.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.