https://orcid.org/0000-0003-1629-1706
ABSTRACT
Background
Evidence-based medicine (EBM) is defined as the integration of the best available evidence from scientific studies with clinical experience (and context) and with patients’ values and preferences. The objective of the present study was to describe self-perceived EBM competencies in physicians and medical students enrolled in a massive virtual EBM course.
Methods
Analytical cross-sectional study. People interested in a free virtual EBM course fulfilled their data in a virtual form for their registration in September 2020. In this form, 22 competencies related to four dimensions of EBM were evaluated: asking a clinical question, search, analysis, and application; using a 5-option Likert scale. The resulting database was analyzed, selecting people who claimed to be physicians or medical students of 18 years or more.
Results
1793 participants were included: 1130 medical students and 663 physicians; more than 80% lived in Peru. The frequency of participants who agreed or strongly agreed with feeling qualified in each competence ranged: from 39.2% to 57.8% for the competencies of the ‘Asking a clinical question’ dimension, from 39.2% to 56.1% for ‘Search,’ from 19.9% to 32.0% for ‘Analysis,’ and from 19.6% to 29.9% for ‘Application.’ Both in physicians and students, the lowest frequencies were for the competencies of interpretation of impact measures, graphs, and results of systematic reviews; as well as shared decision making and calculation of expected benefit. Physicians who graduated more recently scored better on competencies from search and analysis dimensions.
Conclusion
Among physicians and medical students enrolled in the course, self-perception of competencies was lower in the dimensions of analysis and application. More recently graduated physicians seem to have a greater self-perception of their research and analysis skills, probably due to curricular updates.
List of abbreviations: EBM: Evidence-based medicine; CIMBE, for its acronym in Spanish: International Course on Evidence-Based Medicine; SOCIMEP, for its acronym in Spanish: Peruvian Medical Student Scientific Society
Introduction
Evidence-based medicine (EBM) is defined as the integration of the best available evidence from scientific studies with clinical experience (and context) and with patients’ values and preferences [Citation1]. The objective of EBM is to help make rational decisions for the patient’s benefit [Citation2–4]. There are five steps for the EBM process: to ask a clinical question (formulating a clinical question), search (looking for scientific studies that answer the question), analyze (critically reading the studies), application (applying the evidence to make a decision), and evaluation (assessing the decision made) [Citation5].
The use of EBM has barriers such as lack of digital skills, low access to online information, lack of time, the science-practice gap [Citation6], and lack of competencies among health personnel [Citation7,Citation8]. Accordingly, previous studies in Peru [Citation9–11] and other countries [Citation12–15] have found that physicians and medical students often have conceptual and procedural shortcomings regarding EBM competencies. However, these studies focus on assessing a few competencies (usually literature search and interpretation of measures of association) without exploring other competencies relevant to evidence-based decision making.
The scarcity of studies on this subject makes it difficult to know in which competencies physicians and medical students may have more deficiencies, and therefore the design of interventions in this area [Citation16]. Thus, the objective of the present study was to describe self-perceived EBM competencies in physicians and medical students enrolled in a massive virtual EBM course.
Materials and methods
Study design, context, and objective
This was a cross-sectional study that analyzed a database obtained from the registration to a free virtual course called ‘International Course on Evidence-Based Medicine’ (CIMBE, for its acronym in Spanish), organized by the ‘Peruvian Medical Student Scientific Society’ (SOCIMEP, for its acronym in Spanish), whose objective is to carry out academic-scientific activities that promote health research at the undergraduate level [Citation17]. The research team, formed by medical students and health professionals with training and experience in EBM, were also organizers and faculty for the CIMBE course.
Participants
The CIMBE was a free course available to Spanish-speaking students and health professionals (physicians, nurses, obstetricians, dentists, nutritionists, etc.). To register for the course, those interested had to fill out a form. Only those enrolled in the course who described themselves as physicians or medical students were included in the present study. Those under 18 years of age and those who did not complete the variables of interest (EBM competencies) were excluded.
Procedures and variables
The dissemination of the CIMBE was made from 5th to 19 September 2020, through the official social networks of SOCIMEP (on Facebook, Twitter, and Instagram) and in study groups to which the course organizers belonged, on WhatsApp and Telegram. In addition, invitation emails were sent to the following institutions: Epistemonikos and Cochrane associated centers from Chile, Mexico, Ecuador, and Argentina; whose members also participated as speakers in the course.
The course registration was performed from September 5th to 26th, 2020 through Google Forms. This form had two parts. The first part included general data on the participants such as age, sex, country of residence, profession studied or currently studying, current academic degree, whether they were undergraduate students, current year of study or year of graduation, whether they have a master’s or doctoral degree, whether they belong or belonged to a student scientific society and the self-reported number of articles published in scientific journals.
The second part of the form was an instrument created to evaluate the self-perceived competencies in EBM in health care students and professionals. The development of this instrument was based on the essential competencies in evidence-based practice for health professionals, which were defined by consensus in a previous study [Citation3]. This study established 27 competencies divided into six dimensions: introduction (5 competencies), asking a clinical question (3 competencies), search (4 competencies), analysis (9 competencies), application (4 competencies), and evaluation (2 competencies). This instrument was originally developed to obtain baseline data on course participants.
For the development of the instrument, we included all the competencies of asking a clinical question, search, analysis, and application dimensions (20 competencies in total). The introduction and evaluation dimensions were not included in the instrument because we considered that their competencies were unspecific and difficult to operationalize in a questionnaire that evaluated self-perception.
Questions were asked to assess whether participants feel able to perform each of the competencies on their own, with responses on a Likert scale of five categories (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree). For this purpose, the phrasing of the competencies of the reference mentioned above was kept verbatim.
Each competency was independently asked by one question, except for the second competency of the analysis dimension (competency 3.2: ‘interpret different types of measures of association and effect, including graphical presentations’). This competence was divided into three questions (evaluating: measures of association, measures of impact, and graphs). Therefore, a total of 22 questions were formulated. Finally, the frequencies of having responded ‘agree’ or ‘strongly agree’ with feeling qualified to perform each competency were calculated, and the average score for each competency was calculated. Also, we asked whether during undergraduate studies the participant had taken classes in which each of the dimensions mentioned in the form was addressed.
A preliminary test of the form was carried out on a group of 10 students and 10 health professionals to check whether the questions were understood. The members of the course organizing team and their advisors discussed the doubts and suggestions obtained in this test, according to which the questions were reformulated.
Statistical analysis
The database was imported into the Stata/SE version 14 (StataCorp, Texas, USA) statistical software. Before statistical analysis, we evaluated whether there were duplicate records, taking into account the coincidence of surnames and names, and then proceeded to anonymize the database. Then, we searched for possible implausible data in the age variable and the number of articles published. If they were considered not plausible, they were considered as missing data.
For the descriptive analysis, absolute and relative frequencies were used, as well as measures of central tendency and dispersion. For the subgroup of physicians, we considered that due to the changes in the curricula regarding research and EBM during the past years, the perceived competencies could be higher in those recently graduated (who were exposed to a newer curriculum) than in those that graduated earlier. In order to test this hypothesis, we assessed the association between the year of graduation (in tertiles) and the mean score for each competencies dimension. For this, we calculated the coefficients (β) and their 95% confidence intervals (95% CI) using simple linear regression with robust variance, as well as a mixed-model linear regression adjusted for sex, master degree, doctor degree, having belonged to a students scientific society, and having published papers; with the country as a random effect.
Ethical aspects
The present study was approved by the Ethics Committee of the Universidad Peruana Union (Code: 2020-CEUPeU-00028). The database to be used did not collect sensitive information from the participants and was anonymized before starting data processing. It was not possible that individuals could be identified based upon any demographic item. The authors were granted permission to use the course participants’ registry database by SOCIMEP.
Results
After deduplication, data were collected from 2325 persons enrolled in CIMBE. 425 were excluded because they were not physicians or medical students, one because they were under 18 years of age, seven because they did not provide their academic degree, and 99 because they did not provide data on their EBM competencies. Finally, data from 1793 medical students or physicians enrolled in the CIMBE were analyzed.
Of these 1793 individuals, 663 (37.0%) were physicians and 1130 (63.0%) were medical students. The median age was 23 years old in medical students and 30 years old in physicians. Regarding the gender of the participants, 63.9% of the medical students and 45.7% of the physicians were female. More than 80% of students and physicians reported living in Peru. Regarding scientific publications, 13.9% of the students and 30.0% of the physicians reported that they had published at least a scientific article ().
Table 1. Characteristics of the physicians and medical students enrolled in the CIMBE (n = 1793)
Regarding self-perceived competence in EBM, the frequency of participants who agreed or strongly agreed that they felt trained in each competency ranged from 39.2% to 57.8% for the competencies of ‘Asking a clinical question’ dimension, from 39.2% to 56.1% for ‘Searching’, from 19.9% to 32.0% for ‘Analysis’, and from 19.6% to 29.9% for ‘Application’. In both physicians and medical students, the lowest frequencies were found for the competencies of interpretation of impact measures, graphs, and results of systematic reviews, as well as for results of systematic reviews; as well as shared decision making and calculation of expected benefit ().
Table 2. Percentage of participants who agree or strongly agree with feeling trained to carry out each of the activities described (in = 1793)
Out of the 663 physicians enrolled in the CIMBE, 632 recorded their year of graduation. In these, the association between the year of graduation and the average self-perception score in each dimension of competencies (obtained by averaging the scores of all the items corresponding to the dimension) was evaluated. We found that for the dimensions ‘Asking a clinical question’, ‘Searching’, and ‘Analysis’, those who graduated more recently presented higher average scores ().
Table 3. Association between graduation year (in tertiles) and mean score in each dimension, in physicians enrolled in the CIMBE (n = 632)*
Discussion
Summary of findings
The present study was conducted in a sample of physicians and medical students enrolled in an EBM course. Their self-perceived competencies in EBM were generally better for asking a clinical question and searching for evidence, compared to competencies in critical analysis and applying evidence to make decisions. Also, physicians who graduated longer ago were less likely to perceive themselves as having competencies in the search and analysis dimensions compared to those who graduated less time ago.
Comparison with previous studies
In the present study, an exhaustive list of EBM competencies has been used for the assessment of self-perception, whereas previous studies have focused on certain specific competencies. A systematic review of studies assessing EBM competencies [Citation18] found that the instrument most commonly used by previous studies was the McColl questionnaire [Citation13]. This questionnaire focuses only on assessing knowledge of databases, interpretation of statistical results, and concepts of systematic reviews. Other questionnaires used in previous studies [Citation12,Citation14,Citation15,Citation19,Citation20] also focus on some specific statistical and methodological topics. Likewise, studies that have assessed competencies in the form of a graded test [Citation8,Citation21–24] have focused on certain basic methodological concepts such as the definition of EBM, the hierarchy of evidence, and the interpretation of statistical results.
The systematic review above [Citation18] included 57 studies and reported that on average about 60% of physicians reported a high level of knowledge of EBM concepts. However, because these studies have only evaluated some specific competencies and to the great heterogeneity of the instruments used, likely, this result does not reflect the full range of competencies required to apply EBM in medical practice. Therefore, it is important to carry out studies with a more exhaustive approach to these competencies, allowing their evaluation.
We found that the competencies with the highest self-perception were those belonging to the dimensions of asking a clinical question and searching for evidence, and those with the lowest self-perception were those of critical analysis of the evidence and application of the evidence to make a decision. Regarding the competency of asking a clinical question, a study conducted among physicians in Mexico found that 55% of the participants were aware that the first phase of EBM is the formulation of the clinical question, but did not evaluate this competency in more depth [Citation25]. In the present study, a similar frequency was found for the competencies of identifying a clinical question; however, this decreased when asked about the competency of formulating a clinical question in PICO or a similar format. Concerning the search dimension, a study conducted on physicians in training in gynecology in the UK revealed that 35% had minimal literature search skills [Citation21]. Similar to the previous dimension, we reported that half of the respondents were aware of databases where to search and obtain scientific articles, however, self-perceived competence to design and perform a literature search decreased by about 39%.
Regarding the ‘analysis’ dimension, a systematic review conducted in 2017 found 57 studies. In that study, authors reported that between 17.5% and 92% of physicians had some knowledge of the concepts of systematic reviews and meta-analyses and between 8.7% and 95% for biostatistics concepts [Citation18]. These results are very imprecise and difficult to compare with the present investigation since they have not evaluated concepts such as attributable risk, the number needed to treat, or the interpretation of the most commonly used graphs in clinical studies [Citation26,Citation27]. For the application dimension, we did not find previous studies that have evaluated this dimension from the approach proposed by EBM. What stands out from our results is that almost all the competencies evaluated in these two dimensions do not exceed 30%. This is probably because the isolated non-formal training in EBM in recent years has prioritized the questioning and searching dimensions. However, we recognize that adequate competencies in these dimensions alone will not be sufficient to apply EBM well. So, training and assessing competence such as understanding research results, evidence summaries, and their application with each patient according to his/her preferences should be also prioritized [Citation28].
One of the challenges for medical education in Peru is the training in EBM [Citation29]. During the last few years, extracurricular courses have been performed in several universities, and three universities implemented an EBM course into their curricula. However, these courses are mainly focused on theoretical background and application of the three first steps of EBM. This could explain why more recently graduated Peruvian physicians have higher self-perceived competencies for search and analysis than those graduated before. This is contrary to a study conducted in Sri Lankan, where specialist physicians had the best competencies concerning EBM and the highest use of EBM compared to general practitioners [Citation25,Citation30]. So, Peruvian physicians with more professional experience may prefer to be guided by clinical experience or the opinion of their colleagues rather than by EBM, because of their lack of exposure to any formal or informal training in EBM [Citation31].
EBM training is mainly focused on theoretical courses and workshops, especially in the first steps of EBM [Citation32,Citation33]. Some educational interventions showed increased knowledge, skills, and willingness to apply EBM in clinical practice [Citation34–36]. However, little is known regarding which are the best interventions to reach an evidence-based practice, and how context-dependent they are [Citation37].
Our results suggest that a large proportion of the respondents perceive that they do not have the necessary skills to perform the analysis, application, and evaluation steps of EBM. This situation could lead to poor evidence-based decision-making in Peru, as seen during the COVID-19 pandemic [Citation38,Citation39]. Despite public institutions in the country having initiatives to make evidence-based decisions [Citation40,Citation41], there is still a huge need to establish structural interventions, which may include institutional motivation, an environment that promotes EBM application, and formal theoretical and practice training in EBM [Citation42]. This last element may be adapted to some populations’ specific needs according to their specific previous experience and training in EBM. So, before implementing training in EBM, it may be important to assess the base knowledge and competences of the target population to be trained.
Strengths and limitations
Some important limitations should be kept in mind when interpreting the results of the present study: 1) The study assessed self-perception of competencies, and not practice in real scenarios. Although self-perception may underestimate or overestimate the real competencies of medical students and physicians, its evaluation is important, understanding that a high self-perception in a certain activity would be related to a greater willingness to perform that activity [Citation43]. 2) Since this study was carried out on persons enrolled in a virtual EBM course, the extrapolation of the results obtained should be done with caution, since it’s possible that the persons enrolled in the course were those with a greater interest in the subject, who have already made previous efforts to acquire competencies in EBM, and therefore the competencies found in this group could overestimate those of the rest of the physicians and medical students. However, it is also possible that the opposite may have occurred: that those professionals and students who already believe they have competencies in this regard may have considered it of little relevance to enroll in the course. Therefore, we did not seek to extrapolate the descriptive results but rather focus on which competencies the participants feel less skilled. 3) The population of physicians included in our study has a large range of years of professional practice (from 1981 to 2020), so it is not a homogeneous population, as previous experience in clinical practice or research may affect their self-perception of their EBM competencies.
Despite its limitations, the present study is, to our knowledge, the largest population-based study that has assessed self-perceptions of EBM competencies in physicians or medical students in Latin America. Besides, it has evaluated an exhaustive list of competencies, covering the spectrum required for the adequate practice of EBM.
Conclusion
In conclusion, medical students and physicians have a low self-perception of having acquired competencies to perform EBM, especially in the dimensions of evidence analysis and application of evidence to make decisions. Particularly, physicians who graduated longer ago would have an even lower self-perception of competencies. Medical students and physicians require theoretical and practice training in EBM.
Authors’ contributions
MAAR, DRSM, IBCM, and ATR designed the study. MAAR, DRSM, FMGG, IBCM, CCSS, and SPBA participated in data collection. MAAR, DRSM, FMGG, IBCM, CCSS, SPBA, CAAR, ATR, and AVG drafted and revised the first version of the manuscript. All authors participated in the drafting of the article and accepted its final version.
Availability of Data and Materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The present study was approved by the Ethics Committee of the Universidad Peruana Union (Code: 2020-CEUPeU-00028). The database to be used did not collect sensitive information from the participants and was anonymized before starting data processing.
Acknowledgments
We thank SOCIMEP for allowing us to organize the course and to use their social networks for dissemination.
Disclosure statement
MAAR, DRSM, FMGG, IBCM, CCSS, SPBA belong to SOCIMEP and were organizers of CIMBE.
Additional information
Funding
References
- Sackett DL, Rosenberg WMC, Gray JAM, et al. Evidence based medicine: what it is and what it isn’t. BMJ. 1996 Jan 13;312(7023):71–8.
- Albarqouni L, Hoffmann T, Glasziou P. Evidence-based practice educational intervention studies: a systematic review of what is taught and how it is measured. BMC Med Educ. 2018 Aug 1;18(1):177.
- Albarqouni L, Hoffmann T, Straus S, et al. Core competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and delphi survey. JAMA Network Open. 2018 Jun 22;1(2):e180281.
- Hidalgo Mesa CJ, Rodríguez I, Jiménez López M, et al. [Contribution of evidence-based medicine to professional practice]. Rev Cuba Med: 2019;58(1):e978 Mar
- Dawes M, Summerskill W, Glasziou P, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005 Jan5; 51:1
- van der Keylen P, Tomandl J, Wollmann K, et al. The online health information needs of family physicians: systematic review of qualitative and quantitative studies. J Med Internet Res. 2020 Dec 30;22(12):e18816.
- Sadeghi‐Bazargani H, Tabrizi JS, Azami‐Aghdash S. Barriers to evidence-based medicine: a systematic review. J Eval Clin Pract. 2014;20(6):793–802.
- Lafuente-Lafuente C, Leitao C, Kilani I, et al. Knowledge and use of evidence-based medicine in daily practice by health professionals: a cross-sectional survey. BMJ Open. 2019 Mar 1;9(3):e025224.
- Nieto-Gutierrez W, Zafra-Tanaka JH, Pacheco-Barrios K, et al. Self-perception of competences in clinical practice among recently graduated physicians from Lima, Peru. Heliyon. 2020 Nov1;6(11):e05424.
- Segundo-Paredes JA, Gonzales-Medina CA, Francia-de-la-cruz RJ, et al. Conocimientos, actitudes y prácticas de la medicina basada en evidencias en médicos asistentes de un hospital público. Lima- Perú. Rev Peru Investig Materno Perinat. 2018;7(2):9–15.
- Canelo Aybar CG, Santos A, Amao Ruiz EJ, et al. [Differences in attitudes, knowledge and practice of evidence based medicine exist between internal physicians and residents]. Rev Medica Hered. 2007 Apr;18(2): 76–84.
- Philipp D, Poolman Rudolf W, Mohit B, et al. Perceptions and competence in evidence-based medicine: a survey of the american urological association membership. J Urol. 2009 Feb 1;181(2):767–777.
- McColl A, Smith H, White P, et al. General practitioners’ perceptions of the route to evidence based medicine: a questionnaire survey. BMJ. 1998 Jan 31;316(7128):361–365.
- Oliveri RS, Gluud C, Wille‐Jørgensen PA. Hospital doctors’ self-rated skills in and use of evidence-based medicine – a questionnaire survey. J Eval Clin Pract. 2004;10(2):219–226.
- Capraş R-D, Bulboacă AE, Bolboacă SD. Evidence-based medicine self-assessment, knowledge, and integration into daily practice: a survey among Romanian physicians and comparison between trainees and specialists. BMC Med Educ. 2020 Jan 16;20(1):19.
- Aldugieman TZ, Alanezi RS, Alshammari WMG, et al. Knowledge, attitude and perception toward evidence-based medicine among medical students in Saudi Arabia: analytic cross-sectional study. J Fam Med Prim Care. 2018 Sep 1;7(5):1026.
- Cvetkovic-Vega A. [Scientific student organizations as hotbeds for leaders and managers of scientific investigation in Peru: SOCIMEP]. Acta Med Peru. 2017 Jan;34(1): 70–71.
- Barzkar F, Baradaran HR, Koohpayehzadeh J. Knowledge, attitudes and practice of physicians toward evidence-based medicine: a systematic review. J Evid-Based Med. 2018;11(4):246–251
- Sur Roger L, Scales Charles D, Preminger Glenn M, et al. Evidence-based medicine: a survey of american urological association members. J Urol. 2006 Sep 1;176(3):1127–1134.
- Ulvenes LV, Aasland O, Nylenna M, et al. Norwegian physicians’ knowledge of and opinions about evidence-based medicine: cross-sectional study. PLOS ONE. 2009 Nov13;4(11):e7828.
- Jeve YB, Doshani A, Singhal T, et al. Knowledge, skills and attitude of evidence-based medicine among obstetrics and gynaecology trainees: a questionnaire survey. JRSM Short Rep. 2013 Nov21;4(12):2042533313498719.
- Shuval K, Shachak A, Linn S, et al. Evaluating primary care doctors’ evidence-based medicine skills in a busy clinical setting. J Eval Clin Pract. 2007;13(4):576–580.
- Ramos KD. Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003 Feb 8;326(7384):319–321.
- Ahmadi S-F, Baradaran HR, Ahmadi E. Effectiveness of teaching evidence-based medicine to undergraduate medical students: a BEME systematic review. Med Teach. 2015 Jan;37(1):21–30.
- Aguirre-Raya KA, Castilla-Peón MF, Barajas-Nava LA, et al. Self-perception and knowledge of evidence based medicine by physicians. BMC Med Educ. 2016 Jun 29;16(1):166.
- DiCenso A. Clinically useful measures of the effects of treatment. Evid Based Nurs. 2001 Apr 1;4(2):36–39.
- Pocock SJ, Travison TG, Wruck LM. How to interpret figures in reports of clinical trials. BMJ. 2008 May 22;336(7654):1166–1169.
- Tikkinen KAO, Guyatt GH. Understanding of research results, evidence summaries and their applicability—not critical appraisal—are core skills of medical curriculum. BMJ Evid-Based Med. 2021 Oct 1;26(5):231–233.
- Bermúdez-García A, Allagual A, Farfán F. [Medical education in Peru]. Rev Fund Educ Médica. 2020;23(1):5.
- Abeysena C, Jayawardana P, Wickremasinghe R, et al. Evidence-based medicine knowledge, attitudes, and practices among doctors in Sri Lanka. J Evid-Based Med. 2010;3(2):83–87
- Hay MC, Weisner TS, Subramanian S, et al. Harnessing experience: exploring the gap between evidence-based medicine and clinical practice. J Eval Clin Pract. 2008;14(5):707–713.
- Hecht L, Buhse S, Meyer G. Effectiveness of training in evidence-based medicine skills for healthcare professionals: a systematic review. BMC Med Educ. 2016 Apr 4;16(1):103.
- Halalau A, Holmes B, Rogers-Snyr A, et al. Evidence-based medicine curricula and barriers for physicians in training: a scoping review. Int J Med Educ. 2021 May 28;12:101–124.
- Bennett S, Hoffmann T, Arkins M, Bennett. A multi‐professional evidence‐based practice course improved allied health students’ confidence and knowledge. J Eval Clin Pract.2011;17(4):635-9.
- Yost J, Ciliska D, Dobbins M. Evaluating the impact of an intensive education workshop on evidence-informed decision making knowledge, skills, and behaviours: a mixed methods study. BMC Med Educ. 2014 Jan 17;14(1):13.
- Karimian Z, Kojuri J, Sagheb MM, et al. Comparison of residents’ approaches to clinical decisions before and after the implementation of evidence based medicine course. J Adv Med Educ Prof. 2014 Oct;2(4):170–175.
- Maggio LA, Tannery NH, Chen HC. Cate O ten, O’Brien B. Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 2006–2011. Acad Med. 2013 Jul;88(7):1022–1028.
- Herrera‐Añazco P, Uyen‐Cateriano A, Mezones‐Holguin E, et al. Some lessons that Peru did not learn before the second wave of COVID‐19. Int J Health Plann Manage. 2021 Feb 17;36(3):995–998.
- Taype-Rondan A, Herrera-Añazco P, Málaga G, et al. [Regarding the lack of transparency in technical documents for the treatment of patients with COVID-19 in Peru]. Acta Med Peru. 2020Apr;37(2):215–222.
- Gutierrez EL, Piazza M, Gutierrez-Aguado A, et al. [Use of evidence in heath policies and programs contributions of the Instituto Nacional de Salud]. Rev Peru Med Exp Salud Pública. 2016 Aug 1;33(3):580–584.
- García-Mostajo JA, Alva-Díaz C, Suárez V, et al. [The development of evidence-based clinical practice guidelines in Peru for public healthcare facilities]. Acta Med Peru. 2017; Nov,;2 34(3):203–207. doi
- Draaisma E, Bekhof J, Langenhorst VJ, et al. Implementing evidence-based medicine in a busy general hospital department: results and critical success factors. BMJ Evid-Based Med. 2018 Oct;23(5):173–176.
- Lai NM, Teng CL. Self-perceived competence correlates poorly with objectively measured competence in Evidence Based Medicine among medical students. BMC Med Educ. 2011 May 28;11(1):25.