https://orcid.org/0000-0002-4679-6054
ABSTRACT
Background
There has been a rapid development and adoption of online learning in medical education. However, it is difficult to adopt the currently available online learning management systems (LMS). This study aimed to examine learners’ perspectives on the evaluation of online LMS.
Methods
An online LMS was developed based on the evidence-based guidelines. Two cross-sectional studies were conducted. A short survey was conducted with 716 learners registered on the LMS to obtain their perspectives on the online participation. A satisfaction survey was conducted with 255 learners enrolled in the courses taught solely online. Data from the LMS monitoring system was used to report the uptake of online courses. Data were analyzed using descriptive statistics.
Results
Participants reported that the major factor influencing LMS uptake was the ability to be accessed anytime and anywhere (n = 556, 77.7%). The participants had good experience in using the LMS and were satisfied with it (n = 255, mean = 4.53, SD = 0.62). For online degree courses, the course had a high completion rate of 90% provided that a mark was assigned for course attendance. For non-degree courses, irrespective of whether they were free, paid, exam-based, or participation only, the completion rate was considered low (range 4.3–36.7%).
Conclusion
Under a limited budget, a medical school in a low- to middle-income country could develop an effective online LMS to meet learners’ needs. Our newly developed online LMS is relevant, accepted and to the satisfaction of the learners. Medical schools in the same context are encouraged to develop their own online LMS that serve and support learning in both degree and non-degree courses.
Introduction
During the past decade, there has been a rapid development and adoption of online learning in medical education [Citation1]. The use of online educational methods for undergraduate and postgraduate studies and continuing medical education (CME) has increased, particularly during the pandemic of Coronavirus-19 (COVID-19). The pandemic has disrupted medical education [Citation2] for both degree and non-degree courses. Although not the most popular mode of learning because of time consumption and constraints, online education is one of the most widely used modes [Citation3–6]. The use of online education is expected to increase in the future.
Since the COVID-19 pandemic, online educational methods have become the main available learning and teaching mode for undergraduate medical study [Citation7]. In the post-Covid era, the degree of use of online education as a part of degree courses varies depending on the institute and the educational system [Citation8–10]. Examples of online educational methods include the flipped classroom, where learners can learn from materials uploaded and structured online courses [Citation11,Citation12]. Similar to the demand for online education in formal undergraduate studies, the demand keeps on increasing continuously for non-degree courses in continuing medical education (CME) or continuing professional development (CPD). Although online CME has been reported to have low uptake and less popularity, CME providers have adopted this method because of its flexibility, convenience, cheaper travel cost and time [Citation13–15], particularly during the COVID-19 pandemic [Citation16].
Many studies have examined the effectiveness of online educational methods [Citation13–15,Citation17,Citation18]. The effects of online education were originally described on the basis of four levels of effectiveness [Citation15,Citation19,Citation20], namely, satisfaction, learning, performance, and patient and health outcomes. Despite the advantages of online medical education in enhancing learners’ knowledge and skills [Citation18], previous meta-analysis have demonstrated that the effectiveness of online learning is similar to that of traditional learning methods [Citation17,Citation18].
Currently, many institutions are developing educational programs and deliver them through online learning management systems (LMS). Examples of LMS include, Open Educational Resources (OER), Open Courseware (OCW), and Massive Open Online Courses (MOOCs). Transitioning to online education has many difficulties related to LMS, resources management, and instructional design. For example, using an LMS could add to financial constraints [Citation21], or technological limitations [Citation21], inadequate learner monitoring systems and reports or lack of customization abilities [Citation7], or the instructional design might not match learners’ and teachers’ needs [Citation7]. Furthermore, medical teachers do not receive sufficient support or time to develop online courses [Citation22].
Our medical school is a public medical school in Northeast Thailand. Thailand is a low- to middle-income country. There are many limitations to adopting the currently available online LMS by our medical school, such as financial limitations, inability to fully customize the LMS to match learners’ needs, and difficulties in accessing learners’ monitoring systems and reporting. In addition, although our university has an e-learning platform, it is not available for non-members to access online non-degree courses or online CME courses. Thus, we developed a customized online LMS named KKUMEDX to support both degree and non-degree online education within our budget. The present study’s research question, therefore, was what learners’ experiences with the newly developed online LMS were.
Methods
Study objective
The present study examined learners’ experiences with the online LMS.
Study design
Cross-sectional studies were conducted to explore learners’ opinions about the specific features of online learning had on their participation in online learning and their learning experiences through an online LMS.
Participants
This study was conducted with Khon Kaen University learners registered and enrolled in online courses through KKUMEDX. There were no exclusion criteria in this study.
The sample size was computed using OpenEpi version 3 based on the proportion of medical students who accepted e-learning [Citation23]. Assuming a proportion of 0.73, a design effect of 1, and a significance level of 0.05, a total number of 143 responses were sufficient for each survey. However, we included all 716 learners enrolled in the courses during the study period to avoid a potential source of selection bias.
Development of effective online LMS
The designs of the online LMS were based on literature reviews, together with the ideas of the authors (IT, SW, RM), who had vast experience in online education. In developing an online LMS, we followed the steps of Hays and Veitch to plan successful and effective CME programs [Citation24] and applied the guidelines of Cook and Dupras [Citation25], which suggested ten steps in developing effective online learning.
Semi-structured phone interviews were conducted for the pilot testing of the LMS. Five family medicine residency trainees and eight family medicine academic staff members were invited to enroll in the online diabetes course and provide feedback on their online learning experience. Informed consent was obtained from all participants. The participants were asked to complete the course and then for their phone feedback. Data were collected from both the LMS monitoring system and phone interviews.
Interview questions were developed based on a literature review and data from the LMS monitoring system. Phone interviews were conducted at the beginning of May 2020 and recorded by the interviewer (IT) using an iPhone recording device with a writing record. Each interview lasted for approximately 10 minutes. All the audio recordings were transferred to a desktop computer. Each interview was listened to several times, transcribed, and saved in Microsoft Word. The transcripts were then sent to each interviewee to check for accuracy and clarify unclear comments. The interview data were manually analyzed. The first author (IT) coded the interview data directly on the printed transcripts. Each interview transcript was read several times. The data were coded using only one coder. The codes were then grouped into themes. Interrelated themes and abstracting smaller sets of themes were then performed.
The details of the steps in developing the online LMS and results of the pilot test and interviews are reported in the supplementary file (see the supplementary material).
Evaluation of the online LMS and learners’ experience
The newly developed online LMS was evaluated through two surveys:
First, a short survey was conducted to explore learners’ opinions about factors influencing adoption of online learning to gain learners’ perspectives and obtain data for customization of the LMS to serve learners’ needs and, as a result, increase the recruitment and uptake of the LMS. The invitation letter was sent with an explanatory statement and an anonymous survey link through the channels that we used to communicate with learners registered in the LMS prior to enrolling in online courses. A total of 716 learners registered in the LMS voluntarily completed the survey. The survey started on 15 June 2020, and ended on 4 January 2022.
Second, a satisfaction survey was conducted with 255 learners enrolled in 13 courses. These courses were taught solely online (no face-to-face sessions for these courses) through KKUMEDX. An anonymous survey was conducted at the end of each course. Learners who had completed any of these courses voluntarily completed the survey. The survey started on 22 February 2022, and ended on 14 April 2022.
Data sources, questionnaires, and assessment
A short survey was designed to measure the factors influencing the adoption of online programs. For this survey, an online self-administered questionnaire was developed based on the literature on the use of online learning in health professional education [Citation23,Citation26–28] and the evaluation of online learning [Citation29] and LMS [Citation30,Citation31]. The questionnaire used a three-point Likert scale ranging from no influence to major influence.
The questionnaire for the satisfaction survey was designed to measure the following constructs of online learning effectiveness and attitudes toward the use of the LMS: individual learners, factors influencing the adoption of online learning, perceived satisfaction, perceived ease of use, context, pedagogy, perceived barriers to online learning, and interactivity in the LMS. These constructs included a five-point Likert scale ranging from disagree to agree, which was used to assess the learners’ agreement with each item. For the factors influencing the adoption of online learning, the constructs included a five-point Likert scale ranging from no influence to influence.
During questionnaire development, each item was assessed thoroughly regarding the intention of measurement, relativity, ambiguity, understandability, and necessity of the item by three independent experts in the field of medical education at our school to ensure the face and content validity of the questionnaire. The questionnaires were piloted with 30 learners and then revised accordingly. The Cronbach’s alpha coefficient of the questionnaire for the short survey was 0.88, and that for the satisfaction survey was 0.83 (notably, values of internal consistency reliability for the scale above 0.8 is preferable) [Citation32].
Statistical analysis
Data analysis was performed using IBM SPSS for Windows version 26.0. A pairwise deletion strategy was applied to handle missing data. Descriptive statistics were used to describe the demographic data. The participants’ responses on a three-point Likert scale to a 12-item questionnaire on the influence of online learning characteristics on the adoption of online learning (a short survey) and on a five-point Likert scales to a 60-item questionnaire on their online learning experiences (a satisfaction survey) were dichotomized by calculating mean scores. The mean scores were considered agreed at a mean score ≥ 3.5.
Ethical approval
This study was approved by the Human Research Ethics Committee of Khon Kaen University (project number HE631031).
Results
Results of a short survey
Characteristics of online learning that promote participation
A total of 716 learners, registered in KKUMEDX, completed a short survey that asked them to rate the degree of influence that specific features of online learning had on their participation in the activity. The results showed that the majority were medical students with an intermediate level of computer expertise, and more than half had previously experienced online learning ().
Table 1. Demographic data of the learners who registered to online courses in KKUMEDX.
Participants reported the influence of specific features of online learning on their participation in the activity, which were grouped into domains, including perceived usefulness, platform infrastructure, user-friendliness, content quality, and class interaction, in which all domains had a strong influence on their online participation ().
Table 2. The ratings regarding the influence of online learning characteristics on adoption of online learning by the learners who registered to online courses in KKUMEDX (n = 716).
Results of a satisfaction survey
Learners experience in the courses teaching solely online through KKUMEDX
A total of 255 learners, registered in the 13 courses that were provided solely online through KKUMEDX, completed a satisfaction survey. Most of the participants had an intermediate level of computer literacy. Approximately half of the participants were medical students (). The participants of the satisfaction survey reported the extent to which each factor influenced their participation in the online program. The ability to be accessed anytime and anywhere had a strong influence, whereas class interaction had the strongest neutral reaction in all the domains (). The participants had good experience using the KKUMEDX and were satisfied with the LMS (). The participants had mostly positive attitudes toward online learning through KKUMEDX. However, they had a neutral agreement that online learning enabled them to learn at their own pace and experienced social isolation ().
Table 3. Demographic data of the learners who enrolled in the 13 courses teaching solely online through KKUMEDX.
Table 4. The ratings regarding the influence of online learning characteristics on adoption of online learning by the learners who enrolled in the 13 courses teaching solely online through KKUMEDX (n=255).
Table 5. The learning experience of the learners who enrolled in the 13 courses teaching solely online through KKUMEDX (n = 255).
Table 6. The attitudes toward online learning through KKUMEDX compared to conventional learning (face-to-face learning) of the learners who enrolled in the 13 courses teaching solely online through KKUMEDX (n = 255).
The participants disagreed with the barriers to online learning through KKUMEDX. However, they tended to have neutral agreement with their preferred method of learning online when compared with face-to-face learning ().
Table 7. The barriers to online learning of the learners who enrolled in the 13 courses teaching solely online through KKUMEDX (n = 255).
The participants agreed that they would enroll more courses in KKUMEDX, suggesting that others would enroll. They also tended to agree that learning online through KKUMEDX could replace on-site learning (). The participants were satisfied with all the learning media provided through KKUMEDX ().
Table 8. The online learning experience after course completion of the learners who enrolled in the 13 courses teaching solely online through KKUMEDX (n = 255).
Table 9. The satisfaction with the online LMS learning media of the learners who enrolled in the 13 courses teaching solely online through KKUMEDX (n=255).
Acceptance and effectiveness of KKUMEDX
KKUMEDX was tested for its effectiveness on learner acceptance, satisfaction, and learning outcomes, and the results were published in another study [Citation7]. This study tested the effects of a newly developed LMS, KKUMEDX, on first-year medical students enrolled in one course through KKUMEDX on their knowledge and satisfaction. The results showed students’ satisfaction and improvement in their knowledge. Notably, while our previous study [Citation7] focused on testing the effectiveness of the KKUMEDX, the present study explored the learners’ experiences in online learning through KKUMEDX, therefore, the present study focused on comprehensively evaluating the online LMS.
Course uptake on KKUMEDX
It has been about two more years since we launched the first version of KKUMEDX and the first course (see the timeline of the LMS development and evaluation in the supplementary material). As of September 2023, the total number of learners was 10,250. In total, 4,860 were Khon Kaen University (KKU) learners and the remainder were non-KKU members. Within two more years, we developed 119 online courses, including 73 undergraduate, 11 postgraduate, 31 CME, and four other courses.
KKUMEDX has a monitoring system. Uptake data for courses that operated in the academic year 2022 and ended in April 2023 from the LMS monitoring system were reported. shows the uptakes of the courses in KKUMEDX and course completion. For the course organized to learn solely online through KKUMEDX, most of the courses had a high completion rate of more than 90% provided that a mark was assigned for course attendance. For courses that were organized online as optional, the completion rate among these courses was maximum at approximately 50% (range 0.0–53.8%). For non-degree courses, irrespective of whether they were free, paid, exam-based or participation only, the completion rate was considered low (range 4.3–36.7%).
Table 10. The learners’ enrolment and completion of the courses in KKUMEDX.
Discussion
This article provides a comprehensive development method for online LMS for medical education at a medical school in a low- to middle-income country. With this long period of development, we have learned lessons. Several steps must be considered when develop online learning [Citation24,Citation25]. We started by using the developing framework, covering online education for the degree (undergraduate, postgraduate residency training, master’s and PhD) and non-degree (CPD or CME) courses. This means that the LMS can support an online structured learning course that can provide self-directed adult learning for all types of higher education.
We developed an online LMS to meet the needs of both teachers and learners using evidence-based steps to develop effective online CME courses [Citation24,Citation25]. The designs covered the learner experience to ensure that the interface was easy to navigate, and the design elements enhanced learner experience. The LMS content management system was designed to handle a large volume of content, including multimedia such as video, audio, and images, and to allow the course owner to manage, upload, and organize the content easily. The LMS provides tools to facilitate the assessment and evaluation of learner progress, including quizzes and examinations. Communication and collaboration between facilitators/teachers and learners was also implemented through chat rooms that could perform individual or group chats. The designs covered mobile compatibility to allow learners to access content and features anywhere, at any time, and across different screen sizes and devices. The LMS provides analytics and reporting features to help teachers track learners’ progress and identify areas that need improvement. One of the important features of our LMS is customization and integration to meet the needs of learners, and integration with other tools and systems. All these features are key factors that make the LMS user-friendly, feature-rich, and effective in facilitating learning that contributes to its success (a full description is provided in the supplementary material).
Barriers to online learning were considered before the development of the LMS [Citation33]. Some of the identified barriers were corrected; for example, to make the LMS easy to access, provide the function of interpersonal interactions, produce quality content, and develop courses with the support of the production house team. The principle behind developing this LMS was based on the concept of ‘simple use’ by both learners and teachers. Therefore, while designing the LMS, we mainly focused on the simple use of all functions of the LMS [Citation7]. Course quality is an important factor in the adoption of online learning [Citation7]. Thus, we set up a production house team to support these teachers in the process of designing, developing, and presenting courses, such as designing instruction, video, animation, and graphics. This could affect the quality of the content and courses. Another barrier to developing online courses is derived from the teachers (facilitators, course owners, or content experts). We found a variety of interests and abilities of teachers in IT and in designing online learning courses. Based on this, we later customized the KKUMEDX functions for teachers to manage the courses themselves as course owners (a full description is provided in the supplementary material).
A short survey also provided learners’ opinions on the possible factors that led them to adopt the courses in KKUMEDX. These findings are consistent with past evidence that the factors influencing the adoption of online courses include infrastructure, perceived usefulness, content quality, user-friendliness, and class interaction [Citation7,Citation33]. Despite evidence suggesting that Internet based learning formats, including interactivity, practice exercises, repetition, and feedback, seem to be associated with learning outcomes [Citation34,Citation35], participants in the present study did not rate class interaction as a factor in adopting the online course. Possible explanations include the learners’ lack of social skills or feeling uncomfortable interacting in a virtual environment [Citation33]. Some learners may experience time constraints when engaging in class interaction [Citation36]. Our findings contradict those of previous studies in which online learners valued class interaction [Citation37,Citation38]. It can be noted that the level of interaction will depend on individual learning styles and preferences. Therefore, the instructional designs of courses should not only suit the content but also support a variety of learners’ learning styles and preferences.
We considered the factors that may relate to the uptake and completion of the online course, including giving marks for course attendance, quizzes, and exams at the end of the course. We observed that giving marks for course attendance impacted online undergraduate learners’ course completion. This may be because these help learners stay motivated and engaged throughout the course, and quizzes and exams would help them see the key points of the lessons and their progression through the course [Citation39–41]. However, we did not statistically compare each feature with that of the control group to examine this association.
For non-degree (CME) courses, uptake and completion were low despite the use of quizzes or exams at the end of the course. We did not compare courses with or without attendance marks or exams, because the number of courses we had might not have been sufficient to conduct such an analysis. Low initial recruitment and retention of learners in online non-degree courses was found in the present study. Barriers to the recruitment of learners to online courses have been identified in previous studies [Citation42–45], such as time constraints, lack of staff, and interruptions to routines. The low recruitment rate for online courses in the present study was similar to that reported in previous studies [Citation4,Citation33,Citation46,Citation47]. Strategies to improve learners’ participation in non-degree or CME courses have been the focus of several previous studies [Citation48–50], such as using reminders [Citation50], direct mail or promotion via search engine advertising and email [Citation48]. However, a study that used these strategies did not improve the recruitment rate of learners for online CME course [Citation33]. Therefore, further rigorous research is needed to examine effective strategies to increase recruitment for online courses and course retention.
The present study had several limitations. First, despite the high response rates for both surveys, the present study was a single-institution study. Therefore, further large-scale studies involving many medical schools are required to ensure generalizability. Second, despite the long period in designing and developing the LMS, the use of the LMS is still in a short period of approximately two more years. The uptake of each course through KKUMEDX is considerably small, particularly the number of uptakes for non-degree (CME or CPD) courses. Therefore, ongoing monitoring and observation of learners’ behavior in online learning are needed, particularly in Thailand, where CME is not mandatory yet. Third, there is a potential for selection bias in observational studies owing to the lack of randomization. Fourth, the present study used online questionnaires, and the participants in this study may have a different characteristic background from the general population since they may be more familiar with online technology. Therefore, there is a possible bias in these responses.
In summary, we encourage medical schools to develop their own fully customized LMS to maintain online education and serve each particular context of medical education. There are many factors involved in effective online education, including effective LMS, barriers to online learning, teacher support in developing the online course, course evaluation and feedback from the learners, evidence-based strategies to increase uptake, and research on effective instructional designs that can also improve online uptake and retention.
Conclusion
Under a limited budget, a medical school in a low- to middle-income country could develop an effective online LMS to meet learners’ needs. KKUMEDX is relevant, accepted, and to the satisfaction of the learners. Medical schools in the same context are encouraged to develop their own online LMS that serve and support learning in both degree and non-degree courses.
Supplementary_file.docx
Download MS Word (32.3 KB)Acknowledgments
We thank all the other members of the KKUMEDX development team, production house team, and staff in Academic Affairs, Faculty of Medicine, Khon Kaen University, for their assistance.
Data availability statement
The data from this research project are available upon reasonable request.
Disclosure statement
KKUMEDX version 1 and 2 are copyrighted to Khon Kaen University. Isaraporn Thepwongsa, Radhakrishnan Muthukumar, Sakda Waraassawapati, Kamonwan Jenwitheesuk, and Surapol Virasiri have patent #KKUMEDX versions 1 and 2 (online learning programs for non-degree and degree courses).
Supplementary material
Supplemental data for this article can be accessed online at https://doi.org/10.1080/10872981.2023.2299535
Additional information
Funding
References
- Cook DA, Garside S, Levinson AJ, et al. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ. 2010;44(8):765–11. doi: 10.1111/j.1365-2923.2010.03723.x
- Rose S. Medical Student education in the time of COVID-19. JAMA. 2020;323(21):2131–2132. doi: 10.1001/jama.2020.5227
- Yee M, Simpson-Young V, Paton R, et al. How do GPs want to learn in the digital era? Aust Fam Physician. 2014;43(6):399–402. Accessed May 14, 2022. https://www.racgp.org.au/afp/2014/june/learning-in-the-digital-era.
- Thepwongsa I, Muthukumar R, Sripa P, et al. Uptake and effectiveness of online diabetes continuing education: the perspectives of Thai general practitioner trainees. Heliyon. 2023;9(2):e13355. doi: 10.1016/j.heliyon.2023.e13355
- Bernardes CM, Ratnasekera IU, Kwon JH, et al. Contemporary educational interventions for general practitioners (GPs) in primary care settings in Australia: a systematic literature review. Front Public Health. 2019;7(JUN). doi: 10.3389/fpubh.2019.00176
- Cobb SC. Internet continuing education for health care professionals: an integrative review. J Contin Educ Health Prof. 2004;24(3):171–180. doi: 10.1002/chp.1340240308
- Thepwongsa I, Sripa P, Muthukumar R, et al. The effects of a newly established online learning management system: the perspectives of Thai medical students in a public medical school. Heliyon. 2021;7(10):e08182. doi: 10.1016/j.heliyon.2021.e08182
- Shehata MH, Abouzeid E, Wasfy NF, et al. Medical education adaptations post COVID-19: an Egyptian reflection. J Med Educ Curric Dev. 2020 Aug 27;7:2382120520951819. doi: 10.1177/2382120520951819
- Zaidi SJ, Shamim MS. Undergraduate medical education in the post covid era. Ann Jinnah Sindh Med Uni. 2022;8(1):1–2. doi: 10.46663/ajsmu.v8i1.1-2
- Kim KJ. Moving forward: embracing challenges as opportunities to improve medical education in the post-COVID era. Humanit Soc Sci Commun. 2022;9(1):419. doi: 10.1057/s41599-022-01451-7
- Gaur U, Majumder MA, Sa B, et al. Challenges and opportunities of preclinical medical education: COVID-19 crisis and beyond. SN Compr Clin Med. 2020;2(11):1992–1997. doi: 10.1007/s42399-020-00528-1
- Zhang Y, Liu J, Liang J, et al. Online education isn’t the best choice: evidence-based medical education in the post-epidemic era—a cross-sectional study. BMC Med Educ.2023 Oct 10;23(1):744. doi: 10.1186/s12909-023-04746-8
- Wutoh R, Boren SA, Balas EA. eLearning: a review of Internet-based continuing medical education. J Contin Educ Health Prof. 2004;24(1):20–30. doi: 10.1002/chp.1340240105
- Thepwongsa I, Kirby C, Schattner P, et al. Online continuing medical education (CME) for GPs: does it work? A systematic review. Aust Fam Physician. 2014;43(10):717–721.
- Curran VR, Fleet L. A review of evaluation outcomes of web-based continuing medical education. Med Educ. 2005;39(6):561–567. doi: 10.1111/j.1365-2929.2005.02173.x
- Gravas S, Ahmad M, Hernández-Porras A, et al. Impact of COVID-19 on medical education: introducing homo digitalis. World J Urol. 2020;1–7. Published online August 29. doi: 10.1007/s00345-020-03417-3
- Cook DA, Levinson AJ, Garside S, et al. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300(10):1181–1196. doi: 10.1001/jama.300.10.1181
- Pei L, Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Med Educ Online. 2019;24(1):1666538. doi: 10.1080/10872981.2019.1666538
- Kirkpatrick DL. Evaluating training programs: the four Levels. San Francisco, CA: Berrett-Koehler; 1994.
- Hutchinson L. Evaluating and researching the effectiveness of educational interventions. Br Med J. 1999;318(7193):1267–1269. doi: 10.1136/bmj.318.7193.1267
- Barteit S, Guzek D, Jahn A, et al. Evaluation of e-learning for medical education in low- and middle-income countries: a systematic review. Comput Educ. 2020;145:145. doi: 10.1016/j.compedu.2019.103726
- Phillips JA, Schumacher C, Arif S. Time spent, workload, and student and faculty perceptions in a blended learning environment. Am J Pharm Educ. 2016;80(6):102. doi: 10.5688/ajpe806102
- Bączek M, Zagańczyk-Bączek M, Szpringer M, et al. Students’ perception of online learning during the COVID-19 pandemic: a survey study of Polish medical students. Res Sq. Published online July 14, 2020. doi: 10.21203/rs.3.rs-41178/v1
- Hays R, Veitch C. Continuing medical education and divisions, In: Hays R, Richard B, Veitch C, editors. Continuing Medical Education for General Practitioners. Vol. 1999. Queensland, Australia: School of Medicine, James Cook University; 1999. pp. 5–17.
- Cook DA, Dupras DM. A practical guide to developing effective web-based learning. J Gen Intern Med. 2004;19(6):698–707. doi: 10.1111/j.1525-1497.2004.30029.x
- Ali N, Jamil B, Sethi A, et al. Attitude of nursing students towards E-Learning. Adv Heal Prof Educ. 2016;2(1):24–29.
- Ghanizadeh A, Mosallaei S, Dorche MS, et al. Use of E-Learning in education: attitude of medical students of shiraz, Iran. 2018; (c):10–13.
- Abbasi S, Ayoob T, Malik A, et al. Perceptions of students regarding e-learning during COVID-19 at a private medical college. Pak J Med Sci. 2020;36(COVID19–S4):S57–S61. doi: 10.12669/pjms.36.COVID19-S4.2766
- Attwel G. Evaluating E-Learning: a guide to the evaluation of E-Learning. Vol 2; 2006.
- Lasanthika W, Tennakoon W. Assessing the adoption of learning management systems in higher education. GATR Glob J Bus Soc Sci Rev. 2019;7(3):204–209. doi: 10.35609/gjbssr.2019.7.3(5)
- Cigdem H, Ozturk M. Factors affecting students’ behavioral intention to use LMS at a Turkish post-secondary vocational school. Int Rev Res Open Distance Learn. 2016;17(3):276–295. doi: 10.19173/irrodl.v17i3.2253
- Pallant J. SPSS survival manual: a step by step guide to data analysis using the SPSS program (fourth). NSW: Allen & Unwin; 2011.
- Thepwongsa I. Education of rural and remote general practitioners (GPs) in Australia on type 2 diabetes: impact of online continuing medical education on GPs’ knowledge, attitudes and practices and barriers to online learning. Monash University. 2017. Thesis. doi: 10.4225/03/58ae4580ba641
- Cook DA, Levinson AJ, Garside S, et al. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad Med. 2010;85(5):909–922. doi: 10.1097/ACM.0b013e3181d6c319
- Lam-Antoniades M, Ratnapalan S, Tait G. Electronic continuing education in the health professions: an update on evidence from RCTs. J Contin Educ Health Prof. 2009;29(1):44–51. doi: 10.1002/chp.20005
- Shen D, Cho MH, Tsai CL, et al. Unpacking online learning experiences: Online learning self-efficacy and learning satisfaction. Internet High Educ. 2013;19:10–17. doi: 10.1016/J.IHEDUC.2013.04.001
- Hrastinski S. Asynchronous and Synchronous E-Learning | EDUCAUSE review. [cited 2023 Jun 30]. https://er.educause.edu/articles/2008/11/asynchronous-and-synchronous-elearning
- Jaggars SS. Choosing between online and face-to-face courses: community college Student voices. Am J Distance Educ. 2014;28(1):27–38. doi: 10.1080/08923647.2014.867697
- Bonk C. The future of online teaching and learning in higher education: The survey says | EDUCAUSE review. [cited 2023 Jun 30]. https://er.educause.edu/articles/2006/11/the-future-of-online-teaching-and-learning-in-higher-education-the-survey-says
- Selwyn N. An investigation of differences in undergraduates’ academic use of the internet. Act Learn Higher Educ. 2008;9(1):11–22. doi: 10.1177/1469787407086744
- Sun PC, Tsai RJ, Finger G, et al. What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Comput Educ. 2008;50(4):1183–1202. doi: 10.1016/J.COMPEDU.2006.11.007
- Asch S, Connor SE, Hamilton EG, et al. Problems in recruiting community-based physicians for health services research. J Gen Intern Med. 2000;15(8):591–599. doi: 10.1046/J.1525-1497.2000.02329.X
- Perkins D, Harris MF, Tan J, et al. Engaging participants in a complex intervention trial in Australian General practice. BMC Med Res Methodol. 2008;8(1):8. doi: 10.1186/1471-2288-8-55
- Borgiel AEM, Dunn EV, Lamont CT, et al. Recruiting family physicians as participants in research. Fam Pract. 1989;6(3):168–172. doi: 10.1093/FAMPRA/6.3.168
- Sudman S. Mail surveys of reluctant professionals. Eval Rev. 1985;9(3):349–360. doi: 10.1177/0193841X8500900306
- Paul CL, Piterman L, Shaw JE, et al. Poor uptake of an online intervention in a cluster randomised controlled trial of online diabetes education for rural general practitioners. Trials. 2017;18(1). doi: 10.1186/s13063-017-1869-8
- Baso YS, Syafruddin A, Padlan AA, et al. Continuing medical education for general practitioners in Indonesia: moving toward accountability. Gac Sanit. 2021;35(Suppl 2):S120–S122. doi: 10.1016/J.GACETA.2021.07.008
- Harris JM, Novalis-Marine C, Amend RW, et al. Promoting free online CME for intimate partner violence: what works at what cost? J Contin Educ Health Prof. 2009;29(3):135–141. doi: 10.1002/CHP.20025
- Buchan H, Lourey E, D’Este C, et al. Effectiveness of strategies to encourage general practitioners to accept an offer of free access to online evidence-based information: a randomised controlled trial. Implement Sci. 2009;4(1):68. doi: 10.1186/1748-5908-4-68
- Waldorff FB, Siersma V, Nielsen B, et al. The effect of reminder letters on the uptake of an e-learning programme on dementia: a randomized trial in general practice. Fam Pract. 2009;26(6):466–471. doi: 10.1093/fampra/cmp072