Effect of changing journal clubs from traditional method to evidence-based method on psychiatry residents

Abstract

Introduction

Journal club is a valuable educational tool in the medical field. This method follows different goals. This study aims to investigate the effect on psychiatry residents of changing journal clubs from the traditional method to the evidence-based method.

Method

This study was conducted using a before–after design. First- and second-year residents of psychiatry were included in the study. First, the status quo was evaluated by standardized questionnaire regarding the effect of journal club. Then, ten sessions were held to familiarize the residents with the concept of journal club. After that, evidence-based journal club sessions were held. The questionnaire was given to the residents again after the final session. Data were analyzed through descriptive statistics (frequency and percentage frequency, mean and standard deviation), and analytic statistics (paired t-test) using SPSS 22.

Results

Of a total of 20 first- and second-year residents of psychiatry, the data of 18 residents were finally analyzed. Most of the subjects (17 [93.7%]) were females. The mean overall score before and after the intervention was 1.83±0.45 and 2.85±0.57, respectively, which showed a significant increase (P<0.001).

Conclusion

Moving toward evidence-based journal clubs seems like an appropriate measure to reach the goals set by this educational tool.

Keywords:

• journal club
• evidence-based
• residents
• education method

Introduction

Journal club is defined as a regular gathering of a group of people to appraise the latest published articles.¹ It is a flexible framework for teaching basic sciences and clinical skills. One of the earliest resources addressing journal club is the writings of Sir James Paget, a British surgeon and the founder of modern pathology in 1800.¹ At the time, students and residents gathered together at the cafeteria of the hospital and had discussions about journals.¹ However, the classic version of journal club was first employed by Sir William Osler in 1875 at McGill University in Montreal, Canada, and afterward at Johns Hopkins University to train medical assistants.¹ Since then, it has been used as a framework to study and discuss the latest medical research. Therefore, journal club has been used as an educational method in medicine for more than 100 years.¹ In the past, the aim of journal clubs was merely to update participants’ knowledge and information.¹ Today, however, it is used as a method to teach critical thinking and to encourage performing clinical procedures based on evidence-based medicine, research methodology, biostatistics, and epidemiology.^2,3 In some cases, the aim of journal clubs is to have group discussions in English about patients, diseases, and specific clinical conditions based on which a study is selected.^4,5 Today, journal clubs as an effective educational method have attracted more attention.^4–6 Some of the goals of journal clubs include answering important clinical questions raised during patient management process, teaching students and residents how to appraise and analyze the results of studies, and developing Critically Appraised Topics to respond quickly to questions raised in rounds, morning reports, and developing protocols.⁶

In fact, evidence-based practice in a journal club happens when an individual makes a question about a clinical problem a priority instead of a specific topic for the title of his/her presented article. In this educational method, evidence-based medicine is followed in each journal club session. In other words, the best studies and scientific evidence for a clinical question are sought. Moreover, articles are appraised, and they are not blindly accepted even if they are published in the most prestigious journals.

Interestingly, studies show that only 5% of the articles of the world’s top five prestigious journals enjoy an absolutely sound methodology, and only 0.5% of the findings are reliable and valid. Therefore, it is always recommended that articles and publications should be appraised and evaluated to reduce performance risk.⁶ On the other hand, the points mentioned in articles are not supposed to be entirely applied in physicians’ actual work environment. Rather, they should be considered along with individual experience, specific conditions, and patients’ needs and values so that they can be applicable. Hence, in the presence of experts, journal clubs can provide a suitable environment for solving health system problems. In other words, in an evidence-based journal club, one should try to find appropriate solutions and answers for actual clinical problems and questions instead of searching for one’s favorite topic among the titles of articles.⁶

Given the importance of this issue, and since according to researchers, evidence-based journal clubs are not held in many clinical educational departments in our university, this study was designed and carried out in the Department of Psychiatry at Mashhad University of Medical Sciences for the first time to investigate the effect of evidence-based journal club as an educational method on the skills of the residents.

Method

Ethics

This study was approved by ethical boards of Mashhad University of Medical Sciences (930785).

Research design and sampling

This study was conducted using a before–after design. All the first- and second-year residents of psychiatry were included in the study. The traditional journal club was held for many years in this department, and all the residents and faculty members were unsatisfied with it. Before holding the sessions, some items were examined and modified step by step. These steps included following the approved curriculum of psychiatric residency training program, selecting a moderator, determining goals, preparedness of presenters, and planning the active participation of department faculties, especially senior ones in journal club as a supervisor. This article was approved by the ethical board of Mashhad University of Medical Sciences (code: 930785) and verbal informed consent was given from all the participants.

Intervention

First, the status quo was evaluated using a standardized questionnaire regarding the effect of journal club (it was designed and validated by Shokouhi et al⁷). Then, ten sessions were held to familiarize the residents with the concept of journal club, its history, goals, types of studies, evidence pyramid (the evidence-based medicine pyramid is simply a diagram that was created to help us understand how to weigh different levels of evidence), and how to search and critically appraise research articles. After that, evidence-based journal club sessions were held for all subjects. In these sessions, the residents searched databases for their clinical questions with the help of the journal club organizer (one leader in each session) and community medicine specialist. Then, the best article was selected according to the evidence pyramid for each session in consultation with the supervisors. Articles were selected in a way that all residents could become familiar with different types of studies, from the bottom to the top of the evidence pyramid. The article was emailed to all members of the target group (residents and faculty members) about a week earlier. Moreover, the group was instructed to send the article through social networks limited to group members.

All sessions started at 8:30 am and ended at 10 am. They were held at the residents’ conference hall. A resident would present the article, and then scientific discussions about the article would take place, which were followed by questions from residents and the answers. Next, the methodology of the study was explained. Statistical analyses were elaborated with the help of a community medicine specialist, and residents’ questions were answered. After the article was fully understood, the article was appraised based on the critical appraisal guidelines (a systematic process used to identify the strengths and weaknesses of a research article to assess the usefulness and validity of research findings). All articles and their appraisals were archived from the very beginning of the sessions so that they could be later used as a guide for residents. This training course (evidence-based journal club) was held over 18 months, and it was continued till now in this department. The sessions were held once every 2 weeks, and the journal clubs were taught and supervised by a community medicine specialist. The questionnaire was given to the residents again after the final session. Those who were absent for more than three consecutive sessions were excluded from the study.

This questionnaire consists of eight domains, including searching for articles in databases (four questions), understanding and appraisal of scientific articles (five questions), slide preparation (two questions), speech and presentation (seven questions), study habits (three questions), improving treatment methods (three questions), collaboration with the faculty members of other departments (two questions), and overall effect of journal club sessions (two questions). The questions were rated from poor (1 point) to excellent (5 points) based on a 5-point Likert scale. To calculate the score in each area, the mean score of the questions of that particular area was calculated, which ranged from 1 to 5. Data were analyzed through descriptive statistics (frequency and percentage frequency for qualitative variables, and mean and standard deviation for quantitative variables) and analytic statistics (paired t-test) using SPSS 22 (IBM Corporation, Armonk, NY). P<0.05 was considered statistically significant.

Results

Of the total of 20 first- and second-year residents of psychiatry, the data of 18 residents (10 first-year residents and 8 second-year residents) were finally analyzed (two residents were excluded from the study due to absence from more than three sessions). One of the subjects was male (6.3%) and the rest (17 subjects) were females (93.7%). The mean overall score before and after the intervention was 1.83±0.45 and 2.85±0.57, respectively, which showed a significant increase (P<0.001). η² was calculated to evaluate the overall effectiveness, and it was 0.67, which indicated good effectiveness of this intervention. Moreover, the scores in all areas increased after the training course (Table 1). The highest increase belonged to the following areas: “collaboration with the faculty members of other departments” (1.46) and “overall effect of journal club sessions” (1.27). The lowest increase belonged to “searching for articles in databases” (0.86).

Table 1 Scores before and after the educational course

Domains	Before intervention	After intervention	P-value
Searching for articles in databases	1.81±0.6	2.68±0.83	0.003
Understanding and appraisal of scientific articles	1.52±0.42	2.61±0.81	<0.001
Slide preparation	2.02±0.71	3.05±0.93	0.001
Speech and presentation	2.05±0.69	2.92±0.67	<0.001
Study habits	2.03±0.76	3.0±0.84	0.002
Improving treatment methods	1.85±0.52	2.85±0.75	<0.001
Collaboration with the faculty members of other departments	1.58±0.79	3.05±0.92	<0.001
Overall effect of journal club sessions	1.63±0.68	2.91±0.80	<0.001

Note: Data are shown by mean ± standard deviation.

Layered analysis based on resident’s academic year showed that the overall score as well as the score of each area significantly improved among the second-year residents. However, although the overall score among the first-year residents improved, no significant change was observed in the areas of “searching for articles in databases” (P=0.129) and “study habits” (P=0.065). Since there was only one male in the study, it was not possible to analyze the data by sex.

Discussion

The results of this study showed that the overall scores and the scores for each area improved after the intervention, which shows the good effectiveness of an evidence-based journal club. One of the goals of evidence-based journal clubs is to help critically appraise medical literature and make fundamental changes in clinical practice.^8,9 Therefore, this study showed that this goal can be achieved by changing journal clubs from a traditional method to an evidence-based method.

It is believed that journal club sessions can improve study habits and provided updated information,¹⁰ as observed in this study after the intervention.

One of the major points in effective journal club sessions is the active participation of the faculty members and also members of other departments to review and appraise articles. Faculty members’ involvement in the teaching process of journal club sessions leads to more extensive participation of residents within sessions.^11–13 In this study, the highest improvement was observed in the area of collaboration with the faculty members of other departments. In the new method, a community medicine specialist attended all sessions to carry out a systematic review of the articles. In a study by Akhund et al,¹⁴ over 85% of respondents perceived a change in critical appraisal skills and over 65% indicated an increase in knowledge in the topic areas. In addition, residents indicated that they value attending and preparing for journal club. They agreed that presentation of their research work helps them to refine it.

Although the total score of the first-year residents improved, no significant change was observed in the areas of “searching for articles in databases” and “study habits”. This could be due to the first-year residents being less familiar with the technical terms and studying less compared to the second-year residents.

This study also had some limitations. One limitation was that individual changes were measured on a self-report basis, which may not be an accurate measure of skills improvement. This should be confirmed in future studies through more detailed objective assessments, if possible. However, various studies with randomized clinical trial and cohort designs have also confirmed improvement in areas, such as knowledge, skill, behavior, and critical thinking.^15–20

Another limitation of this study was using a 5-point Likert scale and including neutral opinions, which can lead to central tendency bias. The reason for this choice was because of some sensitive questions in the questionnaire. Nevertheless, in this study, the answers often followed one of the two choices, which dispelled our concerns. The small sample size and that the study was done in a single location were other limitations of this study.

One of the strengths of this study that makes the results of this intervention valuable was the acceptable number of psychiatry residents participating in the study.

Conclusion

In today’s world, good medicine requires good information and certain capabilities such as data interpretation, understanding information, and application of research results through learning critical appraisal. Accordingly, evidence-based journal clubs are a perfect educational tool in this area.

Acknowledgments

We hereby thank the Head of Department for her full support and emphasis on holding regular evidence-based journal clubs. We are also thankful to all residents who cooperated with us in conducting this study and Hoda Khatibi Moghaddam and Najme Shahini who helped us with collecting data.

Disclosure

The authors report no conflicts of interest in this work.