Abstract
Background
This pilot study conducted at the Peninsula Medical School is one of very few studies to compare the use of video podcasts to traditional learning resources for medical students.
Methods
We developed written handouts and video podcasts for three common ear, nose, and throat conditions; epistaxis, otitis media, and tonsillitis. Forty-one second-year students were recruited via email. Students completed a 60-item true or false statement test written by the senior author (20 questions per subject). Students were subsequently randomized to podcast or handouts. Students were able to access their resource via their unique university login on the university homepage and were given 3 weeks to use their resource. They then completed the same 60-item test.
Results
Both podcasts and handouts demonstrated a statistically significant increase in student scores (podcasts mean increase in scores 4.7, P=0.004, 95% confidence interval =0.07). Handout mean increase in scores 5.3, P=0.015, 95% confidence interval =0.11). However, there was no significant difference (P=0.07) between the two, with the handout group scoring fractionally higher (podcasts average post-exposure score =37.3 vs handout 37.8) with a larger average improvement. A 5-point Likert scale questionnaire demonstrated that medical students enjoy using reusable learning objects such as podcasts and feel that they should be used more in their curriculum.
Conclusion
Podcasts are as good as traditional handouts in teaching second-year medical students three core ear, nose, and throat conditions and enhance their learning experience.
Video abstract
Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:
Background
Ear, nose, and throat (ENT) surgery must compete with many other subjects for space in undergraduate medical education. As highlighted by Mace and Narula in their review of ENT education in 26 universities in the United Kingdom, ENT only receives an average of 1.5 weeks in the medical school curriculum and is often taught as a combined subject with other specialties.Citation1 Consequently, it is imperative for ENT education to be delivered in an efficient manner and utilize the best methods of delivery available. Medical education is undergoing vast changes. Education no longer follows the traditional model of instructor-centered education, it is now more centered around independent learning, with the emphasis on the learner constructing their own knowledge.Citation2
Reusable learning objects (RLOs) are a new method of delivering medical education that incorporate a spectrum of visual, auditory, and interactive learning resources. A learning object can be defined as a grouping of instructional materials to meet a specified educational objective. A digital learning object is one which can be electronically stored and may utilize text, graphic, animations, audio, and video to support and enhance learning. Due to their electronic nature, they can be considered to be reusable since they can be accessed by many users if hosted online.Citation3 RLOs are a resource that are designed to be used in self-directed learning (SDL) and are therefore an ideal tool to help students in modern medical education.
Video podcasts have the added advantage of engaging students in both auditory and spatial learning compared to other SDL tools such as handouts, that allow spatial learning only. There have only been very few studies that have directly compared the effect of podcasts with other SDL tools in terms of learning topics in medicine.Citation4–Citation6 This pilot study compares the use of podcasts with more traditional handouts in self-directed education across three common ENT conditions namely epistaxis, otitis media, and tonsillitis.
Methods
Ethics approval and recruitment
The Peninsula College of Medicine and Dentistry Ethics Committee approved the study. All second-year medical students at Peninsula Medical School within the Plymouth locality (103 students) were notified of the study via university email account. Attached to the email were the study protocol, a frequently asked questions sheet, and a consent form. Second-year students were chosen as the target population since they have received no formal ENT teaching in their prior curriculum. The students were initially given 1 week to respond to the email. This was extended to 2 weeks with an email reminder at the end of the first week in order to increase the chances of gaining a statistically significant sample size.
Randomization
Students were randomized to the podcasts group or the handout group using a computerized random number generator.
Pre-exposure test
Following randomization, the participants were sent a unique link to a Survey Monkey™ 20 true or false statements for each of the three topics (epistaxis, tonsillitis, and otitis media). The questions were written by the senior author, an ENT consultant from Plymouth Hospitals NHS Trust. An online and remote method was chosen to conduct the entire study because a previous attempt to invite the students to sit an exam under exam conditions onsite at the Medical School had only yielded eight responses. The students were asked to not use any resources while sitting the test and that it should take no longer than 10 minutes. The test would “lock” after one login attempt. Therefore, the test would need to be completed in one sitting. The 41 respondents were given a 2-week period to complete the test in an attempt to reduce the dropout rate.
Exposure to resources
Upon completion of the initial exam, the participants were given access to their allocated learning resource for a 3-week period. Each resource was hosted on the Peninsula Medical School’s virtual learning environment (EMILY). The relevant resource was made visible on the homepage of the student by using their individual login ID. Therefore, nobody apart from the investigators was able to see both resources. Students in the handout group were able to save or print their handouts. Students were requested not to share each other’s resources or use any supporting material including the printed handouts when completing the post-exposure test. There were no additional instructions, so the students were able to use their allocated resource as they saw fit.
Post-exposure test
After the completion of the 3-week period, the resources were removed from EMILY and the students were sent an email with a link to the same 60-item Survey Monkey™ test they sat previously. Again, students were given 2 weeks to complete this under the same conditions as before.
Student questionnaire
Qualitative data regarding level of enjoyment of the learning resource were also collected from the podcast group in the form of a 5-point Likert scale questionnaire.
Statistical analysis
Comparisons were made on the students’ performance scores before and after the interventions. The mean and standard deviations of the pre- and post-exposure test results of students in both groups were computed and the normality of data tested using the Shapiro–Wilk test. Mean difference between pre- and post-exposure scores were calculated for each group. The Shapiro–Wilk test showed the data to be normally distributed. These data were then analyzed for a significant difference using a Student’s paired t-test. The null hypothesis was that there was no improvement between pre- and post-test scores in the podcast group when compared with the handout group. The difference was considered statistically significant if P=<0.05. The breakdown of the data by subject allowed us to see whether podcasts were effective across a range of different subjects, or more effective in some and not others.
The data of the Likert questionnaire were analyzed by calculating the mode response for each question.
Results
is a flowchart of the study. Forty-one out of 103 students responded to the email inviting them to participate in the study (39.8%). Thirty out of the 41 students who agreed to participate in the study completed the pre-exposure test (73.2%). Eighteen students out of 41 who agreed to participate in the study completed the post-exposure test (43.9%).
Figure 1 Population flowchart.
and are box plots of test scores pre- and post-exposure. There was a statistically significant improvement in knowledge scores after provision of both podcasts and written handouts. illustrates the average post-exposure test scores for the podcast and written handouts groups. The average improvement from the podcast group was 4.7 (P=0.004, 95% confidence interval =0.07) compared to 5.3 from the handouts group (P=0.015, 95% confidence interval =0.11). There was however no statistically significant difference found between the use of podcasts and written handouts in improving students’ knowledge scores (average post-exposure scores 37.8 vs 38.3, P=0.07). and illustrate the findings of the Likert scale questionnaire responses of students in the podcast group. The podcast questionnaire showed in addition, that students valued this as a learning resource and found it to be a more enjoyable method of learning. The dropout rates post-exposure was identical in both groups (n=6). Only 40% (n=6) of the podcast group returned their satisfaction questionnaires.
Figure 2 Box plot of average pre-exposure test scores.
Figure 3 Box plot of average post-exposure test scores.
Figure 4 Average post-exposure test scores.
Figure 5 Students’ opinions on use of video podcasts.
Figure 6 Students’ opinions on usefulness of video podcasts compared to written material.
Discussion
Our study demonstrates that video podcasts are as effective as written handouts in teaching three common ENT conditions. However, the students’ learning experience in the podcast group appeared to be enhanced compared to written information they traditionally use as the podcasts were more enjoyable. The advantages and disadvantages of RLO’s have been summarized in which is adapted from a literature review.Citation7 Our findings are supported by the limited literature available on the use of podcasts for medical students.
Table 1 Advantages and disadvantages of digital learning objects
Cook published a review on the use of web-based learning (WBL) in medical education. He concluded that the advantages of WBL include overcoming barriers of distance and time, economies of scale, and novel instructional methods. The disadvantages include social isolation, up-front costs, and technical problems. WBL instructional designs failed to always abide by educational principles and may sometimes be used in an incorrect educational context or application.Citation8
Bhatti et al evaluated the benefit of educating medical students on a common surgical topic (hemorrhoids), through a website and podcast package vs a traditional lecture. Similar to our study, the authors used a pre-intervention questionnaire and an identical post-intervention one. They concluded that using augmented web-based educational tools reduces demands on teaching time with no decrease in quality for selected parts of the curriculum.Citation4
Kumar et al implemented a hybrid curriculum that involved both traditional teaching methods and a new online core curriculum that incorporates audio, video, and text using screen capture technology in a surgical intensive care unit clinical placement. The curriculum was hosted on a secure online portal and covered topics that were considered essential to meet the didactic objectives of the rotation. Students in both groups completed a pre-test on day 1 of the rotation and a post-test administered on the second to last day of the rotation. Unlike our study, the authors did not use identical questionnaires but randomly selected ones before and after the intervention. Again, the authors found that using this form of learning enhanced the satisfaction of students on a busy intensive care unit placement and improved their knowledge-based test scores.Citation6
A study by Peacock et al came to a different conclusion from our study. They conducted a prospective observational study of all medical students undertaking their first attachment in surgery. Students were randomly allocated to four groups: lecture, handout, website, and patient-based tutorial, all on the management of an inguinal hernia. Although the knowledge scores improved in the lecture and website groups, the learning in the website group was thought to be less engaging for students.Citation5
With the evolution of technology and its ever growing degree of accessibility, the potential role of video podcasts in medical education should be recognized and utilized. This can be especially useful in modern medical degree programs which favor a problem-based learning approach. These programs often have limited face-to-face teaching time and rely on individual independent learning. The use of podcasts here could help standardize the information which students are given on a particular topic and can be accessed in their own time and at their own pace.
The potential implications for teachers involved in medical education are highlighted in . However, the usefulness and enjoyment of podcasts from the perspective of the clinical teacher goes beyond the scope of this study.
The main limitation of this pilot relates to the number of participating students as a type II error cannot be excluded. The results of this study can assist us in calculating the sample size required to perform a larger scale study. The dropout rate for students completing voluntary assessments is not unusual. Similarly, it is not unusual to have low response rates for study questionnaires. The students did not sit the test under exam conditions and were able to sit both the pre- and post-exposure test at any point in a 2-week period. The investigators were aware of this limitation at the start of the investigation; however, it was deemed necessary in order to gain a much larger sample size than was previously obtained. Although students were requested not to share resources, there is no way to find out if this has occurred. Providing the students with the same pre- and post-test assessment could have led to a “repeat testing bias”. This risk is mitigated by the fact that students did not have prior experience of ENT when they sat the first test. Moreover, test equating has its challenges and using a different test post-exposure carries the risk of using an assessment that does not have the same degree of difficulty as the pre-test.
Given that there are very few studies that have looked at the effectiveness of video podcasts as teaching tools for medical students,Citation4–Citation6 we therefore propose a definitive followup study that includes more than one medical discipline. This study has shown that RLOs such as video podcasts are as good a resource as handouts in terms of knowledge test scores, but also that they are more enjoyable.
Conclusion
Video podcasts are as good as written handouts in the teaching and learning of three common ENT conditions. Video podcasts appear to enhance the students’ learning experience. Further large-scale studies are required to demonstrate the effectiveness of podcasts in medical education compared to other widely used learning resources and involving multiple disciplines.
Disclosure
The authors report no conflicts of interest in this work.
References
- MaceADNarulaAASurvey of current undergraduate otolaryngology training in the United KingdomJ Laryngol Olol20041183217220
- Bannan-RitlandBThe role of design in research: the integrative learning design frameworkEducational Researcher20033212124
- RuizJGMintzerMJLeipzigRMThe impact of E-learning in medical educationAcad Med200681320721216501260
- BhattiIJonesKRichardsonLForemanDLundJTierneyGE-learning vs lecture: which is the best approach to surgical teaching?Colorectal Dis201113445946220041922
- PeacockOWattsEForemanDLundJNTierneyGMEvaluation of teaching methods for students on hernias: an observational studyANZ J Surg2013831–2111423013104
- KumarABHataJSBaymanEOKrishnanSImplementing a hybrid web-based curriculum for an elective medical student clerkship in a busy surgical intensive care unit (ICU): effect on test and satisfaction scoresJ Surg Educ201370110911623337679
- SandarsJSchroterSWeb 2.0 technologies for undergraduate and postgraduate medical education: an online surveyPostgrad Med J200783985675976218057175
- CookDAWeb-based learning: pros, cons and controversiesClin Med (Lond)200771374217348573