Abstract
Background: Regular use of evidence-based medicine (EBM) among general practitioners (GP) is insufficient.
Objective: To analyse whether knowledge and attitudes about EBM can be improved among mentors in general practice by involving sixth-year medical students as academic detailers.
Methods: An interventional non-randomized before-and-after study included 98 GPs (49 in the intervention group of mentors and 49 controls) and 174 medical students attending family medicine clinical rotations. A telephone survey on knowledge and attitudes towards EBM was conducted among participating physicians before, and six months after the rotation. During the rotation, each mentor chose two cases from real life, and the students’ task was to form an answerable clinical question, find the evidence-based answer and to write a brief report. The mentor reviewed the report and discussed it with the student.
Results: Students’ EBM detailing intervention led to significant improvement in knowledge and attitudes about EBM in the intervention group of mentors in general practice compared to control GPs (relative increase in knowledge was 20 ± 46.9% vs 6 ± 12.1%, respectively; P = 0.042). Among participants with Ph.D. or specialization in family medicine, the observed effects of the intervention were similar as in the total sample, and statistically significant, but not in the group of participants with neither scientific degree nor specialization in family medicine.
Conclusion: Knowledge and attitudes of GP mentors towards EBM can be improved by involving medical students as academic detailers. Further studies should explore the effectiveness of this method among GPs that are not mentors, and who do not have a specialization or research degree.
KEY MESSAGE:
Academic detailing is an interactive, non-commercial educational outreach, which involves face-to-face education of physicians.
Medical students can perform academic detailing for their mentors in general practice.
Academic detailing may enable mentors in general practice to improve their knowledge and attitudes towards evidence-based medicine.
INTRODUCTION
Since its first appearance in public as a concept, evidence-based medicine (EBM) became an integral part of professional medical practice in all medical specializations (Citation1). In most encounters between general practitioner (GP) and patient, one or more clinical questions are generated and need to be properly answered within a reasonable time (Citation2–4). This applies to the use of evidence in daily practice, in diagnostics and cure as well as in prevention and prognosis of disease. Decision making in medicine includes three elements of Sackett's triad: physician's individual expertise, the best evidence resulting from the quality research, and the patient's choices and personal values (Citation1). EBM by its definition is ‘patient- centred’ just as general practice is essentially. Therefore, it would be expected that GPs recognize this similarity and actively practice EBM to help them solve practical problems in daily work. GPs show positive attitudes about EBM practice, but nevertheless regular use of EBM in general practice is far from sufficient (Citation5–7).
The lack of translation of research findings into medical practice and personal behaviour has already been recognized as a problem (Citation8). This finding deserves attention considering that the adoption of evidence-based approaches has been associated with a large and sustained reduction in patient mortality (Citation9). Therefore, interventions that could improve physicians’ knowledge and attitudes towards EBM would be welcome.
Academic detailing is a recognized teaching strategy in physicians’ continuous medical education. The method includes ‘one-to-one’ intervention by peers/academic detailers who visit GPs in their practices and personally interact with them in familiar surroundings (Citation10,Citation11). In several studies, GPs preferred this teaching method within their practice more than attending classical lectures outside the practice (Citation12,Citation13).
It has been emphasized recently that partnership with all stakeholders, patients, GPs and their healthcare colleagues, as well as students, should be encouraged to facilitate a rapid learning healthcare system (Citation14). This initiative recognizes that students can also be a valuable agent of change in the learning process of physicians.
The aim of this study was to analyse whether knowledge and attitudes about EBM can be improved among mentors in general practice using a modified academic detailing provided by medical students.
METHODS
Study design
This study was an interventional non-randomized before-and-after study, conducted in the academic year 2012/ 2013, in medical schools at the University of Split, University of Zagreb and University of Rijeka in Croatia.
Sample and participants
The study included 98 GPs and 174 medical students. Half of the GPs (n = 49) were in the intervention group (20 from Split, 15 from Zagreb and 14 from Rijeka) of mentors and half (n = 49) in the control group of physicians practicing general medicine.
Final group sample sizes of 39 and 39 were needed to achieve 80% power to reject the null hypothesis of equal mean relative change in self-evaluated knowledge between the intervention and control group when the population difference is 0.2 with a standard deviation for both groups of 0.35, significance level of 0.005 and one-sided two-sample, unequal-variance t-test. As about 15% of lost-to-follow up and invalid data collection was expected, the initial sample size was decided to be n = 49 in each group. The power analysis was performed by PASS 13 (Citation15).
Physicians from the intervention group mentored sixth-year medical students attending clinical rotations within family medicine course in medical schools in Split, Zagreb and Rijeka. The response rate among mentors was 100%.
A control group of GPs included contractors with the Croatian Institute for Health Insurance (CIHI) in 2012, similar to their intervention peers in all characteristics except that they did not serve as mentors to medical students. For each initially contacted control GP, a reserve sample of four more GPs was made. If a GP declined to participate, a most similar GP from a reserve sample was invited (four declined to participate; 92% response rate). A control group of GPs was formed by a third party, Biometrika Healthcare Research Company. The study investigators did not participate in sample formation.
Sixth-year medical students participating in the study regularly attended family medicine course in academic year 2012/2013. Within their training in family medicine, they are obliged to do a rotation in the office of a GP mentor. Prior to this academic year medical students did not solve EBM tasks during their family medicine rotations.
Questionnaire
Before the beginning of the family medicine course in the academic year 2012/2013, a telephone survey was conducted among GP mentors and the controls. The same survey was conducted in the same manner six months after the intervention was completed. The survey consisted of 30-item questionnaire used with permission of the authors (Citation16,Citation17) (Appendix). GPs were asked about sociodemographic data (age, sex, work and scientific experience, specialization, academic degree), practice organization and location (urban, suburban, rural), practice size and number of patients they see daily.
Using the combination of open and closed-ended questions, GPs were asked about their need for assistance in diagnosing and deciding on therapy, internet usage, access to EBM resources and knowledge of EBM and The Cochrane Collaboration.
There were six questions about self-evaluated knowledge relevant for the EBM, on: research design, research biases, sample size and power analysis, generalization of research results, statistical analysis and evaluation of the overall research value.
For measuring attitudes, eight questions/statements with a five-item Likert scale ranging from ‘completely agree’ to ‘completely disagree’ were used. The topics were: EBM and its benefit for medical practice, EBM as ‘passing fad,’ sufficiency of EBM teaching during the studies. Some questions addressed systematic reviews and their importance, the significance of experts’ opinions, importance of evidence from the literature and wishes and preferences of the patients.
Intervention
The intervention applied in this study was the modified academic detailing method, performed by sixth-year medical students as academic detailers. The students qualified for that role because they were taught about EBM in several courses during their undergraduate curriculum. During the family medicine rotation, each GP mentored six students in total. In Split, each student visited two mentors for two weeks. In the course of rotation, each mentor chose two patients (cases) from real life who represented diagnostic, therapeutic or prognostic challenge. The students’ task was to form an answerable question, using the patient, intervention, comparison, outcome (PICO) scheme, find the best evidence-based answer to that question and to write a brief report according to practical evidence about real life situations (PEARLS) pattern. The mentor then reviewed the report and discussed it with the student.
Statistics
Descriptive statistics was used for sociodemographic questions. Normally distributed data were expressed as mean ± standard deviation.
Six questions about self-evaluated knowledge were analysed principal components analysis. Extraction criterion was Eigenvalues greater than one. Only one principal component was extracted. It accounted for 78.3% of the total variance of the six questions. Saturation of particular questions varied from 0.85 to 0.92. Cronbach's alpha coefficient of internal consistency was α = 0.95. So, the scale of six questions may be considered one-dimensional and satisfactorily reliable. For these two reasons it was decided to use the sum of the raw scores of all six items as the indicator of self-evaluated knowledge. For the easier interpretation, the raw scores with the theoretical maximum of 30 points were divided to produce the proportions achieved out of the theoretical maximum of self-evaluated knowledge score.
As we had the directional hypothesis that the intervention would increase the knowledge in the intervention group, a one-tail test of statistical significance was used. The statistical significance of the difference between and after the intervention between intervention and control group was accessed by Mann–Whitney U test with Monte Carlo statistical significance on 10 000 tables. The level of statistical significance was set to P < 0.05. The probable confounding effects of the baseline differences between study groups were controlled by stratified analysis. Standardized effect size r was calculated as Z/ (√(n)) where Z was the Mann–Whitney U test statistic, n was the sample size. To test changes in attitudes before/after intervention, a non-parametric Wilcoxon's matched pairs test was used. The analyses were carried out using SPSS 17.0 statistical software package (SPSS Inc., Chicago, IL, USA).
Ethics
The work has been approved by the Ethical Committee of the University of Split School of Medicine.
RESULTS
Participants
Control and intervention study groups of GPs were not equivalent at baseline in regard to three important characteristics. Participants from the intervention group more often had the specialization in family medicine, master or doctoral scientific degree, and access to specialized EBM databases (). All of these characteristics are associated with the studied main outcomes; therefore, their confounding effects were controlled by stratified analysis. In all other monitored characteristics, intervention and control groups were similar ().
Table 1. GP's baseline characteristics.
Change in self-evaluated knowledge
Relative difference in self-evaluated knowledge between intervention and control groups before and after the intervention was statistically significant (Mann–Whitney test, U = 939; Z = –1.76; Monte Carlo one-tail P = 0.042; r = 0.20) (). Before the intervention, self-evaluated knowledge in the intervention group was 56 ± 20.4% out of the theoretical maximum knowledge indicated on six domain items. In the control group, it was 46 ± 15.0%. After the intervention, relative growth in self-evaluated knowledge was 20 ± 46.9% in the intervention group, and 6 ± 27.9% in the control group. Absolute self- evaluated knowledge increase was 6 ± 12.1% in the intervention group and 1 ± 11.8% in the control group. Because the intervention and control groups were different at baseline in regard to the academic degree and specialization in family medicine, an analysis stratified by these two characteristics was performed. In the group with Ph.D. or specialization in family medicine, observed effects of the intervention were similar as in the total sample, and statistically significant (Mann–Whitney test, U = 480; Z = –1.78; Monte Carlo one-tail P = 0.039; r = 0.24) (). In the group of participants with neither a scientific degree nor a specialization in family medicine, the effect of the intervention was not statistically significant ().
Table 2. Self-evaluated knowledge of GPs before and after an educational intervention presented as the percentage out of the maximum achievable score on six five-point scales ranging from ‘insufficient’ to ‘excellent’.
Change in attitudes
There were only two significant changes in attitudes after the intervention. Using the Wilcoxon's matched pairs test, the control group showed statistically significant changes in attitude that they needed more knowledge about EBM (Z = –2.027, P = 0.027). After the intervention, participants in the intervention group felt that during the studies they learned enough about EBM compared to the initial measurement (Z = –1.967, P = 0.049).
DISCUSSION
Main findings
This study showed that academic detailing intervention, involving students solving clinical problems, can improve knowledge about evidence-based medicine among general practitioners serving as mentors to medical students. Stratified analyses showed that this educational intervention was significantly more effective among GPs with Ph.D. or specialization in family medicine, but not among GPs without scientific degree or specialization in family medicine. Although this study had several limitations, it is the first step in assessing the effectiveness of medical students as academic detailers.
Strengths and limitations
This is the first study showing that involving medical students as academic detailers can influence knowledge and attitudes of their mentors in general practice. However, the study has several limitations. The study sample was small and the study was not randomized; thus, interfering variables independent of intervention could influence the results. Furthermore, the main outcome measure was self-evaluated knowledge, which is partly subjective and indirect. It is likely that the mentors were more confident in their knowledge compared to non-mentors and possibly, they overestimated their basic knowledge and the influence of academic detailing. Awareness about the intervention among mentors could also influence their answers.
Comparison with existing literature
Academic detailing is a one-on-one interactive educational outreach programme provided by an educator, who has been trained to discuss certain topics in a manner likely to induce evidence-based practice change (Citation18). Usually, academic detailing is provided by a clinician, either a pharmacist or physician (Citation18). In this study, students were used as educators and academic detailing using students succeeded in improving mentors’ knowledge.
It has recently been described that involving dental students in an academic detailing programme positively affects both practitioner and a student, but this study did not test knowledge or attitudes of practitioners that were visited by dental students as academic detailers (Citation19).
Involving medical students as academic detailers in this study was particularly suitable because Croatian medical schools have recently introduced extensive curriculum on evidence-based medicine (Citation20). Students that are taught EBM in this manner become skilled in finding evidence and show improvement in knowledge and attitudes towards EBM (Citation17). However, it is still uncertain whether this students’ knowledge and attitudes will indeed translate into different real-life practice (Citation17).
Such interventions are necessary because it has already been shown that Croatian physicians have insufficient knowledge and attitudes regarding EBM (Citation16). EBM is unfortunately not included in classes taught within their vocational training or specialization. It has been shown previously that introduction of EBM curriculum into family medicine residency not only increases residents’ knowledge, but also increases the use of EBM constructs during patient care (Citation21).
Promoting the use of high-quality scientific information is a fundamental component of EBM. Therefore, all physicians should have some EBM skills and regularly apply them in daily work (Citation22,Citation23). Even though many physicians are positive towards the notion of practicing EBM and there is evidence showing that teaching can increase knowledge and skills in EBM (Citation24), few physicians have attended formal training in EBM because of various reasons, including negative attitudes or lack of confidence, time or funding (Citation25).
There are initiatives for providing instruction in principles and methods of medical education to family medicine teachers as a component of their training as medical teachers. Such curricula have been developed across Europe, covering also Croatian family medicine teachers (Citation26). Such initiatives should include modules, workshops and journal clubs about EBM to improve competency of family medicine teachers in this important area of medical practice and education.
Implications for practice and future research
Most of the GP mentors that participated in this study have specialized family medicine, and a third of them had a master's or doctoral degree. All of the GP mentors were community faculty and it has already been shown that community faculty has less knowledge about EBM than full-time faculty. Therefore, faculty development programmes for community faculty would especially benefit from modules about using and teaching EBM concepts (Citation27).
Furthermore, by comparing mentors and non-mentors important differences in the level of education and level of pre-existing knowledge of EBM was found. Therefore, it would be useful to include mentors and non-mentors in both intervention and control group in the future studies on this subject. Further studies should explore whether the same method of education is also effective in general practitioners who are not mentors. It would also be useful to design future studies on this topic as a cluster randomized trial.
As this study shows, using students as academic detailers seems to be worthwhile if the GPs are specialists or hold a Ph.D. degree. It remains to be explored whether GPs without these degrees can equally benefit from a similar intervention, as these doctors could need the intervention most.
Conclusion
Academic detailing method using students as tutors via solving EBM cases can improve knowledge on EBM among physicians serving as mentors in general practice. Previous studies showed that better EBM knowledge translates into better patient care. Such interventions should be systematically explored and encouraged for the benefit of patients and health systems.
ACKNOWLEDGEMENTS
The authors are grateful to the students and physicians who participated in the study.
Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
REFERENCES
- Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn‘t. Br Med J. 1996;312:71–2.
- Sood A, Ghosh AK. Literature search using PubMed: An essential tool for practicing evidence-based medicine. J Assoc Physicians India 2006;54:303–8.
- Kinder K, Pettigrew LM. Improving primary care through information. A Wonca keynote paper. Eur J Gen Pract. 2014;20:333–6.
- Bjerre LM, Paterson NR, McGowan J, Hogg W, Campbell CM, Viner G, et al. What do primary care practitioners want to know? A content analysis of questions asked at the point of care. J Contin Educ Health Prof. 2013;33:224–34.
- Zipkin DA, Greenblatt L, Kushinka JT. Evidence-based medicine and primary care: Keeping up is hard to do. Mt Sinai J Med. 2012;79:545–54.
- Shuval K, Shachak A, Linn S, Brezis M, Reis S. Evaluating primary care doctors‘ evidence-based medicine skills in a busy clinical setting. J Eval Clin Pract. 2007;13:576–80.
- Swennen MHJ, van der Heijden GJMG, Boeije HR, van Rheenen N, Verheul FJM, van der Graaf Y, et al. Doctors‘ perceptions and use of evidence-based medicine: A systematic review and thematic synthesis of qualitative studies. Acad Med. 2013;88:1384–96.
- Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.
- Jernberg T, Johanson P, Held C, Svennblad B, Lindback J, Wallentin L, et al. Association between adoption of evidence-based treatment and survival for patients with ST-elevation myocardial infarction. J Am Med Assoc. 2011;305:1677–84.
- Markey P, Schattner P. Promoting evidence-based medicine in general practice—the impact of academic detailing. Fam Pract. 2001;18:364–6.
- Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. J Am Med Assoc 1995;274: 700–5.
- Grol R, Grimshaw J. From best evidence to best practice: Effective implementation of change in patients‘ care. Lancet 2003;362: 1225–30.
- Owen PA, Allery LA, Harding KG, Hayes TM. General practitioners‘ continuing medical education within and outside their practice. Br Med J. 1989;299:238–40.
- Van Royen P, Rees CE, Groenewegen P. Patient-centred interprofessional collaboration in primary care: Challenges for clinical, educational and health services research. An EGPRN keynote paper. Eur J Gen Pract. 2014;20:327–32.
- Hintze J. PASS 13. power analysis and sample size software. Kaysville, UT: NCSS LLC; 2014. Available from http://ncss.com/software/pass. 2014.
- Novak K, Miric D, Jurin A, Vukojevic K, Aljinovic J, Caric A, et al. Awareness and use of evidence-based medicine databases and Cochrane Library among physicians in Croatia. Croat Med J. 2010;51:157–64.
- Balajic K, Barac-Latas V, Drenjancevic I, Ostojic M, Fabijanic D, Puljak L. Influence of a vertical subject on research in biomedicine and activities of The Cochrane Collaboration branch on medical students‘ knowledge and attitudes toward evidence-based medicine. Croat Med J. 2012;53:367–73.
- Solomon DH, Van Houten L, Glynn RJ, Baden L, Curtis K, Schrager H, et al. Academic detailing to improve use of broad-spectrum antibiotics at an academic medical center. Arch Intern Med. 2001;161:1897–902.
- Rugh JD, Sever N, Glass BJ, Matteson SR. Transferring evidence-based information from dental school to practitioners: a pilot ‘academic detailing‘ program involving dental students. J Dent Educ. 2011;75:1316–22.
- Marusic A, Malicki M, Sambunjak D, Jeroncic A, Marusic M. Teaching science throughout the six-year medical curriculum: Two-year experience from the University of Split School of Medicine, Split, Croatia. Acta Med Acad. 2014;43:50–62.
- Ross R, Verdieck A. Introducing an evidence-based medicine curriculum into a family practice residency—is it effective? Acad Med. 2003;78:412–7.
- Guyatt GH, Meade MO, Jaeschke RZ, Cook DJ, Haynes RB. Practitioners of evidence based care. Not all clinicians need to appraise evidence from scratch but all need some skills. Br Med J. 2000;320:954–5.
- Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;376:1923–58.
- Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH, Kunz R. Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. Br Med J. 2002;325: 1338–41.
- Greenhalgh T, Douglas HR. Experiences of general practitioners and practice nurses of training courses in evidence-based health care: A qualitative study. Br J Gen Pract. 1999;49:536–40.
- Bulc M, Svab I, Radic S, de Sousa JC, Yaphe J. Faculty development for teachers of family medicine in Europe: Reflections on 16 years‘ experience with the international Bled course. Eur J Gen Pract. 2009;15:69–73.
- Beasley BW, Woolley DC. Evidence-based medicine knowledge, attitudes, and skills of community faculty. J Gen Intern Med. 2002;17:632–9.