How can we teach EBM in clinical practice? An analysis of barriers to implementation of on-the-job EBM teaching and learning

Abstract

Introduction: Evidence-based medicine (EBM) improves the quality of health care. Courses on how to teach EBM in practice are available, but knowledge does not automatically imply its application in teaching. We aimed to identify and compare barriers and facilitators for teaching EBM in clinical practice in various European countries.

Methods: A questionnaire was constructed listing potential barriers and facilitators for EBM teaching in clinical practice. Answers were reported on a 7-point Likert scale ranging from not at all being a barrier to being an insurmountable barrier.

Results: The questionnaire was completed by 120 clinical EBM teachers from 11 countries. Lack of time was the strongest barrier for teaching EBM in practice (median 5). Moderate barriers were the lack of requirements for EBM skills and a pyramid hierarchy in health care management structure (median 4). In Germany, Hungary and Poland, reading and understanding articles in English was a higher barrier than in the other countries.

Conclusion: Incorporation of teaching EBM in practice faces several barriers to implementation. Teaching EBM in clinical settings is most successful where EBM principles are culturally embedded and form part and parcel of everyday clinical decisions and medical practice.

Introduction

The amount of medical knowledge is growing exponentially, but integrating research into practice is slow (Berwick 2003; Lenfant 2003). Health professionals often fail to implement clinical manoeuvres that have established efficacy (Hayward et al. 2005). As a consequence, patients might receive suboptimal treatment. To stay up to date and deliver optimal health care, health care professionals need to incorporate life-long learning in their profession (Raza et al. 2009). Evidence-based medicine (EBM) equips doctors with skills to integrate evidence from research in clinical decision making and improves the quality of health care. Professional organisations therefore increasingly promote training in EBM for all health care professions at all levels of education (General Medical Council 2009; Kunz et al. 2009; O’Connor & Pettigrew 2009). It has been shown that clinically integrated teaching of EBM is the best way to improve evidence-based behaviour in practice (Khan & Coomarasamy 2006). Unfortunately, integration of EBM teaching for postgraduate junior doctors in everyday clinical practice is uncommon and remains a challenge (Dawes et al. 2005; Hatala et al. 2006).

Courses on how to teach EBM in practice are (scarcely) available in Europe (Oude Rengerink et al.). Improving knowledge about how best to teach EBM does not automatically lead to implementation of good teaching and learning practice (Khan & Coomarasamy 2006).

Many previous studies focused on attitudes and barriers for implementing EBM in health care practice (McColl et al. 1998; Hannes et al. 2008) but barriers for implementing the teaching of EBM in clinical practice have only been studied briefly. These barriers and facilitators are currently not well understood. They might differ within and between countries, as they might be related to health care organizational culture, language and availability of evidence and resources to find the evidence (Matsui et al. 2004; Scherrer et al. 2006; Bosch et al. 2007; Letelier et al. 2007; Liu & Stewart 2007).

In this article, we aim to identify and compare barriers and facilitators for teaching EBM in clinical practice in European countries of varying backgrounds. This may provide opportunities for improved strategies of teaching and practicing EBM, which, ultimately, may lead to higher quality and effectiveness of health care delivered to patients. It also provides the opportunity to diminish differences in EBM teaching between countries by tackling joint barriers collectively.

Table

Practice points

Incorporation of teaching EBM in practice faces several barriers to implementation. Important barriers for teaching EBM in clinical practice were the lack of teaching time in busy practice and lack of curriculum requirements for teaching EBM. There also seem to be differences between countries in the perception of being a barrier or facilitator.

Methods

We conducted a questionnaire survey. Based on literature review in PubMed and input from experts in EBM teaching participating in the EU EBM TTT project (www.ebm-unity.org), a questionnaire was constructed and tested listing potential barriers and facilitators for EBM teaching in clinical practice. The questionnaire also collects demographic characteristics as well as information about how often participants taught EBM in a clinical setting in the past month (questionnaire available upon request).

The survey targeted senior clinicians who teach EBM on-the-job in a clinical setting and explored whether they perceive a certain issue as a barrier or facilitator for their teaching. They provided answers ranging from not at all to an insurmountable barrier or facilitator, on a 7-point Likert scale. Issues included attitude, available time, hospital hierarchy, level of understanding of English literature, availability of resources, EBM knowledge and skills of teachers, requirements for EBM teaching in curricula or at workplace, and availability of Teaching the Teacher courses.

The questionnaire was distributed to participants taking part in an e-learning course Teaching the Teacher, a EU-EBM project funded by the EU Leonardo da Vinci program (www.ebm-unity.org) (Thangaratinam et al. 2009). It was also distributed to EBM teachers participating in a validation study of an assessment tool for this course, and to members of the steering committee of our project.

We additionally distributed the questionnaire at an international conference for teachers and developers of EBM (Oxford, 8–9 December 2008).

Median scores on the 1–7 Likert scale were used to report the level of being a barrier with 1=not a barrier, 2=very mild barrier, 3=mild barrier, 4=moderate barrier, 5=severe barrier, 6=essential barrier and 7=insurmountable barrier. For facilitators, responses were scored as 1=not at all relevant, 2=may be important, 3=slightly important, 4=moderately important, 5=important, 6=very important and 7=essential.

Barriers and facilitators were analyzed over all participants and additionally explorative analyses were stratified per country. Differences between countries were tested using the non-parametric Kruskal–Wallis test or the Wilcoxon Rank Sum test. Differences within countries were tested using the Related Samples Wilcoxon Signed Rank Test. p-values less than 0.05 were considered statistically significant. SPSS version 16.02 was used for all analyses.

Results

A total of 120 clinical EBM teachers from 11 predominantly European countries completed an online or paper questionnaire: 29 from the United Kingdom, 21 from Hungary, 18 from Switzerland, 18 from the Netherlands, 17 from Germany, 4 from Poland and 11 from a variety of other countries (Italy, Belgium, Canada, Finland, Greece and USA). All the 74 participants of the pilot project, all 24 participants of validation of the pilot project, 17 conference participants (small percentage) and 5 members (50%) of the steering committee filled out the questionnaire. Participants of the pilot project filled in an electronic version, and all others answered on paper.

Of the participants, 82 were male and 38 female. Thirty-seven of them were under 40 years old, 78 between 40 and 59 and 4 older than 60 years. Almost all teachers worked in a teaching hospital (N = 103, 91%). Fifty-one (43%) teachers stated to teach EBM to postgraduates, 20 (17%) to teach EBM to undergraduates, 20 (23%) to both post- and undergraduates and 20 (17%) stated not to teach EBM in clinical practice.

The most frequently used EBM teaching activities in the past month were demonstration of an electronic search of literature or search strategy and attending a journal club or an equivalent activity for critical appraisal of research papers; half of the EBM teachers used it frequently to always (51% and 50%, respectively).

Barriers

In Figure 1, median values of barriers for teaching EBM in clinical practice are ranked, and barriers which differ significantly between the countries (overall) are marked. In the text below, the level of being a barrier on the Likert scale is expressed as a median with the interquartile range (IQR). Based on median rankings of 120 participants in all countries, severe barriers (median=5) for teaching EBM in clinical practice were overall lack of time for teaching EBM (median 5; IQR 3–6) and lack of time available for trainees to do a literature search (median 5; IQR 3–6). Moderate barriers (median=4) are lack of requirements for EBM, i.e. lack of requirements for EBM skills later in doctors’ career (median 4; IQR 3–6), for EBM skills at exams both at postgraduate (median 4; IQR 3–6) and undergraduate level (median 4; IQR 2–5) and lack of EBM requirements in curricula (median 4; IQR 2–5) and when medical universities are accredited for medical education (median 4; IQR 2–5). A traditional pyramid hierarchy of junior and senior clinicians and authority of clinical/managerial leadership are, if used, also perceived as major barriers for teaching EBM in practice (median 4; IQR 2–5).

Figure 1. Barriers for teaching EBM in clinical practice (median and IQR): overall and split per country. Barriers marked with an asterisk (*) differ significantly between countries (p < 0.05).

In Figure 2, the ranking of barriers being at least a mild barrier is shown for the United Kingdom, Hungary, the Netherlands, Germany, Switzerland, and Poland. For clarity, we grouped the barriers. Figure 2 shows that although the lack of time and lack of EBM requirements are the main barriers in all countries, the trainee's level of understanding English articles and time required for reading English articles are very severe barriers in Poland (median 6; IQR 5–6), moderate barriers in Germany (median 4; IQR 2–6), Hungary (median 4; IQR 2–6) and Switzerland (median 4; IQR 3–5) and hardly a barrier in the Netherlands (median 2; IQR 1–2). Perceived lack of improvement of patient outcomes following EBM teaching and perceived lack of improvement by peers were considered moderate barriers in Hungary (median 4; IQR 3–5) and mild barriers in Germany (median 3; IQR 2–5) Poland, Switzerland, the Netherlands, and the United Kingdom (median 3; IQR 2–4).

Figure 2. Ranking of the main barriers (with a median ≥3) for the United Kingdom, Hungary, the Netherlands, Switzerland, and Poland.

Additional barriers mentioned by the teachers were; EBM is not used as a tool in health insurance reimbursement policy decisions, there is no trust in EBM models, EBM is not a priority in some organizations and there is a lack of availability of training rooms.

The total sum of the medians of the barriers was lowest in the Netherlands (total of medians 77) and the United Kingdom (total of medians 84), while rankings were highest in Hungary (total of medians 126) and Germany (total of medians 128) (ANOVA p < 0.001).

Facilitators

In Figure 3, median values of facilitators for teaching EBM in clinical practice are ranked, and facilitators which differ significantly between the countries (overall) are marked. Computer access in clinics and wards is an essential facilitator for teaching EBM and ranked highest in all countries (median 7; IQR 6–7). Improved access to relevant databases or journals (median 6; IQR 6–7), regular teaching activities in the trainers hospital or department (median 6; IQR 5–7), need for EBM skills in quality improvement projects (median 6; IQR 5–7), courses in EBM (median 6; IQR 5–7), requirements for EBM skills when medical universities are accredited for medical education (median 6; IQR 5–6), requirements for EBM skills at exams at postgraduate level (median 6; IQR 5–6), need for EBM skills in policy decisions (median 6, IQR 5–6), need for EBM skills in developing guidelines (median 6; IQR 5–6), a mentor guiding the teacher on how to teach EBM (median 6; IQR 4–6), increased availability of evidence summaries (median 6; IQR 5–6), evidence of improvement of clinical practice following teaching EBM (median 6; IQR 5–6), a qualification for EBM teachers (median 6; IQR 4–6) and more time allocated for learning EBM for the trainee (median 6; IQR 5–6) are also very important facilitators for successful implementation of teaching EBM in clinical practice.

Figure 3. Facilitators for teaching EBM in clinical practice (median and IQR): overall and split per country. Facilitators marked with an asterisk (*) differ significantly between countries (p < 0.05).

Important facilitators were more time allocated for teaching EBM for the trainer (median 5; IQR 5–6), requirements for EBM skills at exams at undergraduate level (median 5; IQR 4–6), expectation by the training place to have skills in providing evidence-based health care (median 5; IQR 5–6) and availability of clinical librarians with knowledge of EBM (median 5; IQR 4–6). Moderately important facilitators were an e-course or handbook on how to teach EBM in native language (median 4; IQR 3–5, for both e-course and handbook) or English (median 4; 3–6, for both e-course and handbook) or financial incentives for teaching EBM (median 4; IQR 3–5).

Moreover, in Hungary and Germany, a handbook or e-learning course in the native language scored both significantly higher than an handbook or e-learning course in English (Related Samples Wilcoxon Signed Rank Test p < 0.01). For Switzerland and the Netherlands, an English or native tongue course did not score differently (p > 0.20).

Additional facilitators mentioned by the teachers were; summary cards by which you can teach trainees how to critically read an article, an audit of outcomes in the clinical setting, a good team relationship, an EBM mission statement of the department, a few ‘problem-owners’ who are responsible for EBM, advice from experienced clinical epidemiologists, a fee for teaching, free accessibility of the Cochrane Library online, good English–German online dictionaries, international conferences, journals with review articles, organisational commitment, patient empowerment, senior clinicians as role models, technical equipment of course rooms: easy access to the WLAN, trials and reviews actually running in the department, a culture of EBM in the clinic with support from chief of department, a course on what, where and how to search.

Overall facilitators were ranked lowest in Germany (total medians 107) and the Netherlands (total of medians 111) and highest in Switzerland (125) and Hungary (123) – ANOVA p <0.001).

Discussion

In this predominantly European survey, we found that important barriers for teaching EBM in clinical practice were the lack of teaching time in busy practice, lack of curriculum requirements for teaching EBM in clinical practice and the hierarchical nature of the medical profession, which inhibited teaching in light of perceived threat of criticism of seniors. Computer access in clinics and wards was seen as an essential facilitator for teaching EBM on-the-job and ranked highest in all countries. Improved access to relevant databases or journals, regular teaching activities in the hospital or department and formal requirements for EBM skills are very important facilitators for EBM teaching.

Many barriers and facilitators were common in all countries, but there also seem to be differences between countries in the perception of being a barrier or facilitator, e.g. concerning the lack of understanding and time required for reading English language articles.

We found that overall barriers for teaching EBM were ranked lowest in the Netherlands and the United Kingdom and highest in Poland and Hungary: this might be because in Poland and Hungary, the barriers for teaching EBM are truly bigger, or they are perceived higher due to different levels of perception or expectations in the various countries. In any case this study shows a West-to-East gradient of these perceived barriers, which might be explained by historical, cultural, societal, educational and economic and health service differences between these countries.

A study by Matsui et al. (2004) in Japan found that the lack of English proficiency is the main barrier for teaching EBM, followed by the lack of time. Letelier et al. (2007) concluded that language barriers should be taken into account when teaching EBM to Spanish-speaking physicians.

Melnyk et al. (2008) found resources, including time and money and traditional clinical mindsets/attitudes as main barriers for teaching EBM in nurse practitioner curricula. According to these participants, main facilitators were teamwork and mentoring. Meats et al. (2009) point out that EBM undergraduate teaching is restricted by the lack of curriculum time, trained tutors and teaching materials. Davis et al. (2009) tried to improve evidence-based continuing medical education (CME) by an evidence-based CME credit designation, but found time constraints and limited understanding of the approval process to be barriers.

Similar to this study, those studies also point to difficulties with language and time. Next to their findings, we identified other barriers for teaching EBM in practice such as lack of requirements for EBM in curricula in Europe.

A strong point of our study is that, to our knowledge, no previous study focused systematically on a comparison and differences between countries. As clinical teachers ranged from different countries in Europe, we can assume that barriers found in all partner countries will be barriers in practice irrespective of the local specifics of any individual hospital, corporate culture, language or societal values.

There are also some limitations which require a remark. The teachers who filled out the questionnaire might not be representative for all clinical teachers in a country, as most of them filled out the questionnaire as part of their voluntary participation in the piloting of a Training the Trainer EBM course of the EU-EBM project. These participants are inherently more interested in EBM and might see more opportunities or on the contrary more barriers for teaching EBM in clinical practice, which made them decide to take part in the course.

The relatively small number of participants per country makes it difficult to draw firm conclusions about (the magnitude of) differences between countries. This study was exploratory. To be able to adjust for potential confounders, which might mask or introduce differences between countries, a larger, representative sample should be used.

To our knowledge, there was no validated questionnaire available which could be used to identify and compare barriers and facilitators for teaching EBM in clinical practice. We therefore constructed a questionnaire ourselves, based on the literature review and input from experts in EBM teaching.

Many barriers restrain implementation of on-the-job training in practice, which might implicate that improvements in knowledge and skills of clinical teachers do not automatically imply its teaching in practice.

Some barriers can be tackled jointly on a European level, such as requirements for EBM can be laid down in curricula. Other barriers will need to be dealt with on a local level, e.g. translation of materials in the native language or by making good (online) dictionaries and translation programs available.

Acknowledgments

We thank all participants for filling out the questionnaire. This study is funded by the EU Leonardo da Vinci program.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.