Blended learning in CME: the perception of GP trainers

Abstract

Introduction: Blended learning (the combination of electronic methods with traditional teaching methods) has the potential to combine the best of traditional education with the best of computer-mediated training. We chose to develop such an intervention for GP trainers who were undertaking a Continuing Medical Education (CME) course in evidence-based medicine (EBM). This study reports on our experience and investigated the factors influencing the perception on usefulness and logistics of blended learning for learners in CME. Methods: In total, 170 GP trainers participated in the intervention. We used questionnaires, observations during the four face-to-face meetings and evaluations in the e-course over one year. Additionally we organised focus groups to gain insight in some of the outcomes of the questionnaires and interpretations of the observations. Results: The GP trainers found the design and the educational method (e-course in combination with meetings) attractive, instructive and complementary. Factors influencing their learning were (1) educational design, (2) educational method, (3) topic of the intervention, (4) time (planning), (5) time (intervention), (6) learning style, (7) technical issues, (8) preconditions and (9) level of difficulty. A close link between daily practice and the educational intervention was considered an important precondition for the success of the intervention in this group of learners. Conclusion: GP trainers were positive about blended learning: they found e-learning a useful way to gain knowledge and the meetings a pleasant way of transferring the knowledge into practice. Although some preconditions should be taken into consideration during its development and implementation, they would participate in similarly designed learning in the future.

Keywords:

• Evidence based medicine
• continuous medical education
• blended learning

What is known in the area

Blended learning is at least as effective as nonblended instruction.

Much has been published about blended learning compared to other forms of education, but little about (the design of) the process and the perception of the learners.

What this work adds

This study adds the learners’ views of this, for them, unknown educational method.

These views, expressed in advantages and disadvantages on nine identified themes, give recommendations for the design of next courses.

Suggestions for future work or research

More research on the process of blended learning would give more insight into the most effective blend.

Another important issue is which blend suits which subject: is it possible to develop a scheme appropriate to each subject?

Introduction

Education has become more tailored and active. Blended learning is an educational strategy which has become increasingly important in achieving this,[1–3] although it is mainly used at the undergraduate level.[4,5] Vanderput et al. [6] described blended learning as ‘a mix of online and face-to-face methods and learning activities, in which learners participate actively with the learning content, individually and in interaction with each other and with the teacher’. This working definition was based on that of colleagues.[7–9] It fits our conception that active learning, assisted by developments in IT and is the best educational model for the twenty-first century.

Osguthorpe and Graham [10] identified six reasons for designing and applying a blended learning environment: (1) pedagogical richness, (2) access to knowledge, (3) social interaction, (4) personal agency, (5) cost-effectiveness and (6) ease of revision. Undergraduate students confirm these principles and report that blended learning enables them to achieve their learning goals by choosing the learning experience that matches their preferred learning style. It allows flexibility, for example, in attendance and the use of learning materials.[11,12]

General practitioners (GPs) and GP trainers face several barriers related to both insufficient knowledge and skills and a negative attitude to the use of evidence-based medicine (EBM).[13–17] However, current health care practice considers the use of EBM as essential.[18,19] It is also considered an important component of competency-based learning.[20] Continuing medical education (CME) may have a positive influence on the EBM competency of the GP trainer.[21] The current EBM course consisted of four complete days in the institute. Given the above mentioned didactical benefits and heavy workload of GPs, blended learning as educational model may be optimal to train GP trainers in EBM.

We therefore designed an evidence-based blended CME learning intervention to teach EBM to GP trainers.[16,22] As studies of blended learning in CME are scarce, the aim of this study was to identify the factors influencing the perception on usefulness and logistics of blended learning for learners in CME.

Method

This study was embedded in a larger study of the effectiveness of a blended learning programme on EBM for GP trainers.[23] For the current study, we used a mixed-method approach,[24] including questionnaires, observations and focus groups, to ensure the collection of a rich dataset. All data were collected between February 2012 and April 2013.

Participants

All GP trainers at the Academic Medical Centre of the University of Amsterdam (AMC-UvA), and seven GP trainers at Leiden University Medical Center (LUMC) volunteered for participation in the study. All participants gave their written informed consent, and ethical approval was obtained from the Ethical Review Board of the NVMO (Dutch Association for Medical Education).

Context

GP specialty training in the Netherlands is a three year programme. In the first and the third year of the training, the trainee works in the practice of a GP trainer, while in the second year, they participate in three to six month clinical rotations. New GP trainers (all with at least five years of experience as a GP) follow a four-year training programme (eight days/year), focused on didactic competences. After these four years, they are required to complete eight days (48 h) of courses each year at the training institute. EBM blended learning was a compulsory part of the programme in 2012, accounting for four days in total.

Course

This course consisted of: (1) four face-to-face (F2F) meetings of 3 h each, focusing on EBM attitudes and behaviour in daily practice, including role-modelling, and (2) an e-course focusing on knowledge and skills, requiring 12 h of self-study in total, including home assignments. At least three weeks before the first meeting, all GP trainers received an email, including a link to the e-course and assignments per session. The e-course could be accessed from any computer at any time. The user could navigate freely through the course. We used a ski-metaphor to clarify the levels of difficulty (blue = easy, red = moderate and black = difficult). Before each meeting, participants were asked to complete about 3 h of e-learning and an assignment. The assignment focused on collaboration with their trainee and their daily practice. The GP trainers were assigned to one of 13 groups (12 AMC, 1 LUMC) of 7–12 participants, divided over four teachers. All teachers had a medical background. All of the teachers were talked through the course before each meeting. Eight groups had their meetings scheduled between 9 am and 12 noon and five groups between 1 and 4 pm

This blended course was based on educational evidence, as well as evidence regarding technical aspects of e-learning. A detailed description of the development of this course has been given elsewhere.[23] In short, we based the design of our course on principles from social constructivist learning theory [25] and adult learning theory.[26] During the F2F sessions, attention was paid to the social component; social interaction and collaborative learning.[23] This choice was based on our learning goals (a change of behaviour in practice) and our students (GP trainers). To give direction to the design of the e-learning component of our course, we used the four-component instructional design (4C/ID) model.[27] This model was developed to assist in the design of education or training programmes that continue over a longer period of time and focus on complex learning, that is, the integration of knowledge, skills and attitude, as well as their transfer to the work situation.

Study design

We used (1) questionnaires, (2) observations of the face-to-face meetings and (3) focus groups (after the last course day).

Questionnaires

Personal characteristics such as sex, age, year of graduation, research experience, experience as a trainer and composition of the patient population were collected.

Before the start of the course (T₀) and on the last course day (T₁) we asked all participating trainers questions on their computer skills,[28,29] and on the advantages and disadvantages they expected (T₀) or were familiar (T₁) with e-learning, on a 7-point Likert scale. Furthermore, before each face-to-face meeting we asked participants about their time investment on the e-course and the level of difficulty experienced (3-point Likert scale).

Observations

The observers did not participate in the intervention. Of the 52 face-to-face meetings, we were able to observe 43 meetings. Remarks made by the participants that related to the educational method were noted by the observers.

Focus groups

To further explore findings from the questionnaires and observations during the meetings, all trainers were invited by email to participate in a focus-group session after the last training day at the institute. Of the 170 trainers, 16 responded positively. The reasons given for non-participation were lack of time or not attending the last course day. Due to the low response rate, we accepted all those who responded, rather than choose a sample taking into account specific characteristics. However, the group consisted of GP trainers with high and low levels of EBM knowledge and computer skills. The group discussions were led by a moderator and both were audio-recorded.

The discussion was guided by two questions: (1) What was your experience of the new educational programme? and (2) Did the subject (EBM) influence your experience with this new educational design? The latter question was based on the assumption that the opinion on an educational method is influenced by the interest in the subject studied.[30]

Analysis

Quantitative data

All data were analysed in predictive analytics-software (SPSS 20.0). Categorical data were summarised as proportions. Continuous data were visually checked for normality. Normally distributed continuous data were expressed as means with standard deviations, and non-normal data as medians and quartiles. A Kruskal–Wallis test was performed to compare the medians of ordinal data between groups.

Qualitative data

The remarks from the observations were typed up, while the focus group sessions were transcribed literally. Each transcript was independently coded by two researchers (EtP and JM) using ‘open coding’.[31] No theoretical framework was used during the coding procedure. We searched for themes that summarised the various statements. The consistency of the themes was checked by the researchers by coding the transcripts once again while looking for blanks or inconsistencies that did not fit into the themes, and establishing whether the themes were mutually exclusive. We also explicitly looked for contradictory statements. Consensus was reached through discussion.

We combined the results of the quantitative and qualitative data to obtain an in-depth overview of the results.

Results

The personal characteristics of the participants are presented in Table 1. Of the 170 GP trainers, 71.8% were present at three or more of the four meetings (Table 1).

Table 1. Characteristics of respondents: GP trainers Amsterdam and Leiden combined.

Participants (n)	170
Gender (Female)	56 (33.5%)
Age (Mean [SD])	55.9 (± 5.4)
Years practising as GP (Mean [SD])	22.4 (± 7.2)
Years practising as GP trainer (years; Median [quartiles])	8 (6–11)
Research experience (%)	15.6
Attendance meeting 1× (%)	8.2
Attendance meeting 2× (%)	20.0
Attendance meeting 3× (%)	40.6
Attendance meeting 4× (%)	31.2

All of the meetings had a delay of 5–10 min before starting. All of the observers mentioned that the GP trainers actively participated during the meetings.

After the analysis, we identified nine themes influencing the perception of learners of a blended learning intervention in CME. The identified themes were: (1) educational design, (2) educational method, (3) topic of the intervention, (4) time (planning), (5) time (intervention), (6) learning style, (7) technical issues, (8) preconditions and (9) level of difficulty. Elements of all themes were considered as strength or weaknesses of the educational intervention. Statements from the observations and the focus groups are presented in Table 2.

Table 2. Quotes from the observations and focus groups.

Theme	Statement
Educational method	‘I think it is relatively well suited for this type of education. Even that combination, and so um, but can always be fine-tuned so to speak, but I just think it was a very good theoretical subject.’
	‘ … and, I’ve done a few times before a course, and I lost the focus because other people ask questions and just go faster than me. So I think it’s very nice that I can do this course at my own pace and with peace around me’
	‘ … the palliative course, that was just the whole day at the institute, from start to end a very nice, informative, interactive day. I would say, I do not want to change this into a computer day.’
	I particularly like that you already have a theoretical background before you come here, yes I like that …
	‘Then you benefit even more from what is discussed here.’
Time (planning) & educational method	‘Yes, at first I thought, yes, same objections as you have, for it used to be nicely rounded, and not all day, and now it has been pulled apart and you have to find your own time. Whether in your own time, time you might have or have not, which is always out of your hands, or when you are not very fresh. But it’s actually one hundred percent better than expected. I’m actually, I find it a very pleasant method. We did have the meeting in the afternoon, so you could also use the morning to do the preparation. And then you’re still fresh.’
Educational design & Topic of intervention	‘But what you really want is a kind of summary of the important points.’
	‘An topic that is not quite popular with GPs.’
Educational design	‘Well, the nice thing is that you can also occasionally look back again … Go back to ‘what was it again?’ Or the library or the glossary. I think that is an advantage, you can just switch back.’
	‘You can see what you had done, which was not possible in the other course. So you have a view of what you not yet have done.’
	‘… but for example if you, well you need to relate five things, and then you have got one not connected properly, but then it says “error, do this again”. And it is eighty percent good and twenty percent error, so to speak.’
	‘Answers in e-learning too rigid.’
Time (intervention)	‘A disadvantage is, if you want to do it well, but that also depends on the subject and also what you know about it to start with, you can lose a lot more time than half a day, as is given.’
	‘It (the e-learning) is more time consuming then stated because I look at other chapters and text because you want the complete picture’
Educational method & Topic of intervention & Time (intervention)	‘It is precisely because it was a very difficult subject for most of us. It is an advantage that you can do it at your own pace. Let me put it this way. That is strong, huh? That you can look back at what it means. On the other hand, it is therefore a difficult test case. So uhm, and you lose, there is so much more time needed than actually suggested.’
Technical problems and computer skills	‘Of course there are always some flaws in such a program (the e-learning). It would be motivating if the opportunity would be to respond to it’
Level of difficulty	‘To take this topic to you, you have to do it more often. So this week I had to repeat basically everything, for this really very difficult subject.’

Educational design was of significant influence on the perception of the interventions. Factors such as the skiing-metaphor, flexibility in the navigation, use of text/voice, glossary and a library were considered useful, although others judged them as childlike or were unaware of functions. Some perceived offline hand-outs/ summarised information on for instance abbreviations or formulas to be useful. Visual attractiveness for some influenced the decision to proceed with a chapter. As was their opinion on whether content could be linked to GP practice and whether they felt feedback on questions was fair and/or they could comment on this feedback. Personal advice on how best to proceed, for instance using pre-tests could influence their decisions as well.

Educational method

At T₀ (before the first meeting) and T₁ (at the last meeting) there was no difference between the expectations of and experience with e-learning (Table 3). Of the GP trainers, 57.9% would choose a blended learning concept again. In addition, they found e-learning a useful way to gain knowledge (median 5, quartiles 3–5) and the meetings a pleasant way of transferring the knowledge into practice (median 5, quartiles 4.25–6).

Table 3. Expectations educational intervention (1 = strongly disagree; 7 = strongly agree).

(T0 N = 104; T1 N = 91)		Median (IQ)
I expect that the benefits of participating in this education via the internet outweigh the disadvantages.	T0	4 (4–5)
	T1	5 (4–5)
I expected that participating in this education via the internet would give me the ability to spend more time on other activities.	T0	3 (2–4)
	T1	3 (2–4)
I expect no prior serious drawbacks to participating in this education via the internet.	T0	4 (3–5)
	T1	5 (4-5)
I expected participation in this education via the internet would offer me the opportunity to organise my work more effectively.	T0	4 (3–5)
	T1	4 (3–5)

Both components (e-learning and meetings) fulfil important needs of learners. The e-course, allowed participants to follow the learning path which suited their learning needs, while the meetings deepen understanding and cooperation, offer support, greater sociability, as well as being motivational and enjoyable. Furthermore during the meetings difficult concepts could be elaborated and translated into practical use, reinforcing the relation between e-learning and meetings. Some participants felt a paper version rather than an e-course would better fulfil their learning needs.

Topic of intervention

Fifty-seven percent of the GP trainers stated that the subject of the course (in this case EBM) influenced their opinion about the learning intervention.

The nature of the content influences the opinion on the use of e-learning. The theoretical component of EBM makes an e-course useful. Examples should however be as closely related to clinical practice as possible to motivate trainers. More easy or appealing topics would possibly result in a more positive judgement of the educational method.

Time (planning)

The most often-mentioned advantage of e-learning was ‘own time’ (at a convenient time) (66%). More than 60% of the GP trainers scheduled time for the e-learning part of the programme before each meeting. Working in their own time and their own place, with less pre-scheduled time by the institute both resulted in positive and negative feedback, as several trainers found it difficult to schedule the e-course due to a high workload and relatively long travelling time for shorter meetings.

Time (intervention)

The time spent on the e-course varied depending on the e-learning task (Table 4). The proportion of GP trainers who had completed the assignments before each meeting also varied. The time spent on the e-course generally took longer than indicated because trainers (sometimes by mistake) studied more chapters than required, studied chapters more intensively or repeated earlier chapters. The opinion on the use of this time depended on the perceived quality of the outcome/ amount of learning.

Table 4. Time involved in e-learning task.

	Meeting 1	Meeting 2	Meeting 3	Meeting 4
Time spent on assignment (min; mean[SD])	152 (± 70)	161 (± 104)	171 (±89)	214 (±135)
Time scheduled in diary (yes; %)	65.9	67.1	65.0	61.1
Assignment completed before meeting (%)
Yes	54.7	85.2	85.6	61.1
Partly	38.8	0	0	13.7
No	6.5	14.8	14.4	25.3

Learning style

An often-mentioned advantage of e-learning was ‘at your own pace’ (64%). Some participants stated that the e-course was in line with their learning style – self-directedness, others however prefer face-to-face education: a teacher in front of the classroom teaching the theory.

Technical problems and computer skills

The computer skills of the GP trainers concerning the use of the internet were quite good (Table 5). Whether technical problems were considered frustrating depended on the possibility to report them. Losing information on, for instance, progress, or the need to switch between pages annoyed participants.

Table 5. Computer skills (1 = strongly disagree; 7 = strongly agree).

Computer/internet skills (N = 107)	Median (quartiles)
I feel confident in starting the internet.	6 (4–7)
I feel confident when searching for a web page.	6 (4–6)
I feel confident when downloading necessary materials from the internet.	5 (3–6)

Course preconditions, such as the level of communication, number of reminders, compulsory nature and factors related to the teacher were also considered by participants. The quality of the teacher was indicated as a very important factor in relation to the motivation of the participants. The mandatory nature of the course led to resistance, but was also needed by some to address a more difficult topic such as EBM.

Level of difficulty

Some GP trainers considered it important that the course was at an adequate scientific level. As several trainers found the concepts difficult and the level too high for them and could not complete the assignments successfully in one attempt, they did not continue.

Discussion

Main findings

Most GP trainers found both components of the blended learning intervention – the e-course and the meetings – attractive, useful and complementary, despite their resistance to the topic. We identified nine themes influencing this general opinion of GP-trainers on a blended learning intervention: (1) educational design, (2) educational method, (3) topic of the intervention, (4) time (planning), (5) time (intervention), (6) learning style, (7) technical issues, (8) preconditions and (9) level of difficulty. The opinion of the trainer on the intervention was strongly influenced by the content of the intervention, link with the daily practice of the GP trainer, and time-related aspects.

Comparison to other studies

CME is important to ensure medical practitioners remain up to date with developments in their profession, but it requires the investment of time. In their studies of the perception of GPs on the role of CME, Gagnon et al. [32] and Goodyear-Smith et al. [33] found time to be the most important barrier; time as constraining factor and having other priorities. In their review of internet versus non-internet courses, Cook et al. [34] found that the ‘time spent on learning’ was similar, although the wide variability suggested that time varied for specific implementations. Interventions to enhance learning took more time. In our study, time was also an important factor, specifically when looking at time spent on the e-course and the assignments undertaken in preparation for the meetings. Time was seen as a strength, as the e-course allowed greater flexibility in terms of when and how much time each participant spent. These strengths, choosing one’s pace and time, have been reported by several studies.[35] But in this study it was also experienced as a weakness because several GP trainers found it difficult to use the scheduled study time actually for study and not for small jobs. This could implicate that the advantages of e-learning – own time and pace – are not explicit if there is a high workload. And especially this high workload might be a specific feature of CME learners as compared to, for instance, student learners. Kirby and collegeus [36] also identified the importance of workload in relation to learning and Delva and collegues [37] concluded in their study that ‘a perception of overwork is associated with a surface dis-organised approach to learning, external motivation, and perception of many barriers to CME’. They suggested further study on attention to the practice environment for support learning. Our intervention was based on 50% e-course and 50% F2F meetings. Perhaps we should reconsider this segmentation.

The 4C/ID model of Van Merrienboer et al. [27] integrates knowledge, skills and attitude, as well as their transfer to the work situation. Choosing tasks connected with daily practice is important in this model. The statements of the GP trainers in our study supported the relevance of this model. In this respect, and particularly in relation to the transfer of knowledge and skills to the workplace, the background of the teacher might be important, as they could provide this connection. In our study, the GP trainers preferred their teachers to have a background as a GP or a GP with statistical knowledge, which could indicate that the transfer of new knowledge and skills to daily practice was considered very important. Scientific expertise, in our opinion important for explaining the more difficult concepts in EBM, was considered not as important by our participants. So despite the presence of the teacher is decreased, it seems that the interpretation of the lectureship is still important.

Strengths and limitations

One of the limitations of this study was the low response rate for participation in the focus groups, so we could not determine data saturation. Fortunately, those who did participate were diverse, with high and low levels of knowledge about EBM and high and low levels of computer skills.

A strength of this study is that we were able to incorporate all GP trainers at the AMC-UvA, along with seven at the LUMC, representing a large and varied group of GP trainers. A second strength is that we were able to collect both qualitative and quantitative data.

Relevance and future research

A blended learning educational intervention could be the first step in the process of integrating more practice-based assignments (optionally with a trainee) with knowledge-deepening at one’s own pace and time.

Future research might study the aspects of a flipped classroom in depth and the contribution of various components of the blended learning programme (e-learning, face-to-face meetings, assignments). It could also focus on behavioural changes based on the development of competences related to attitudes, knowledge and behaviour.

As part of our target population was expected to have relatively low ICT skills, in this study, ICT possibilities of social and collaborative learning were not included. Social networks can however improve academic skills and critical reflection, necessary to use EBM and as pointed out by Sandars et al. [38] to improve patient care and professional performance. A next educational design could incorporate this aspect as well.

Conclusion

Several factors influence the opinion of learners on CME in a blended format, but properly constructed and delivered there are benefits to a blended approach from the point of view of learners. A close link to the daily practice of participants is an important factor when designing a blended learning intervention.

Disclosure statement

No potential conflict of interest was reported by the authors.

Ethical approval

Ethical approval was obtained from the Ethical Review Board of the NVMO (Dutch Association for Medical Education).

Funding

This study was financed by CASH (Commission that Stimulates Activities related to GP training) of the SBOH Foundation. The SBOH is the official employer of trainee GPs, and finances all Dutch GP training programmes.