Effect of COVID-19 on Continuing Education Activities and Learner Interactions: Report from Six Accreditation Systems

ABSTRACT

The COVID-19 pandemic has had disruptive effects on all parts of the health-care system, including the continuing education (CE) landscape. This report documents, what has happened in six different CE accreditation systems to CE activities as well as learners. Complete lockdown periods in the first part of the COVID-19 pandemic have inevitably led to reductions in numbers of the then predominant format of education, i.e. onsite in-person meetings. However, with impressive speed CE providers have switched to online educational formats. With regard to learner interactions this has compensated, and in some systems even overcompensated, the loss of in-person educational opportunities. Thus, our data convincingly demonstrate the resilience of CPD in times of a global health crisis and offer important insights in how CPD might become more effective in the future.

KEYWORDS:

• Covid-19 pandemic
• continuing education activities
• learner interactions
• accreditation systems

Introduction

The COVID-19 pandemic has had disruptive effects on all parts of the health-care system, including the continuing education (CE) landscape. This report aims to document, what has happened in six different CE accreditation systems to CE activities as well as learners.

Methods

Data presented in this report have originally been prepared for the session “Exploring our data: jurisdictional trends” on 14 September 2023 at Cologne Consensus Conference 2023 (Cologne, Germany) [1] and in part been presented elsewhere [2]. Accreditation systems have been asked to provide data for 2019 (as pre-pandemic comparator), 2020/21 (the pandemic years), and 2022/23 (post-pandemic period) to answer the following two questions:

1. What have been the three most striking findings in your system during and after the pandemic?

2. Did you (have to) change any rules in your accreditation/licencing framework during the pandemic? If yes, have these changes been maintained in the post-pandemic period?

Data are presented as numbers or percentages, they are descriptive only and no statistical analysis has been performed.

Results

This section reports the findings for each accreditation system separately.

ACCME

The Accreditation Council for Continuing Medical Education (ACCME) is a non-profit, U.S.-based accrediting body that empowers educators located around the world to develop and deploy innovative, effective, independent continuing education for the health professions. Currently, more than 1,640 organisations are accredited in the ACCME system, and those organisations offer more than 200,000 accredited educational activities and interventions each year. The ACCME collects data about the size, frequency and format of education offered to better understand the learning preferences, educational and practice settings, and other characteristics that make accredited continuing medical education (CME) effective for physicians and healthcare teams.

I. What have been the three most striking findings in the ACCME system?

Changes to the Number and Formats of Accredited Educational Activities (Figure 1)

Figure 1. Activities offered in the ACCME system by Education Format (2019–2022).

*Other educational formats include Manuscript Review, Test-Item Writing, Learning from Teaching, Committee Learning, Performance/Quality Improvement, and Other/Blended Learning.

From 2019–2022, the pandemic affected the number and formats of accredited activities, as well as the engagement by learners, within the ACCME system. In 2019, approximately 191,000 activities were offered. The most predominant formats were live (52%) and enduring materials (30%) (e.g. online, recorded videos, podcasts).

In 2020, the number of accredited activities decreased by approximately 20,000. Most of that decrease was seen in live activities. The decrease was somewhat offset by an increase in enduring materials, as many live activities were transformed into recorded video or audio-based education.

In 2021, there was some recovery in the number of activities offered as restrictions from the pandemic were lifted. Both live and enduring material education increased.

By 2022, the system had recovered and was growing again with more than 230,000 accredited activities offered and an increase in all formats. Notably, enduring materials as a percentage of the total overtook live activities.

Increase in Physician Learner Interactions by Format (Figure 2)

Figure 2. Activities offered in the ACCME system by Physician Learning Interactions (2019–2022).

*Other educational formats include Manuscript Review, Test-Item Writing, Learning from Teaching, Committee Learning, Performance/Quality Improvement, and Other/Blended Learning.

The United States has approximately one million licenced physicians. Those physicians engage in multiple educational activities in a year. Learner interactions are tracked across all ACCME’s accredited activities.

In 2019, the ACCME system reported a total of 17.3 million physician learner interactions. Approximately 12% of learner interactions were in live activities and 30% of engagement was in regularly scheduled series (hospital-based learning). Internet searching and learning, as an educational format, made up a small percentage of activities but recorded the same percentage of learners as live activities. Internet searching and learning activities are much shorter in duration than any other format and are often aggregated into micro-learning activities.

In 2020, there was an overall slight growth in physician learner interactions. Physicians were still engaged in learning, but the engagement in live activities dropped and engagement in enduring materials increased – likely because of increased time spent in pandemic patient care and the cancellation of in-person, live education.

In 2021, there was continued growth in physician learner engagement with a sharp increase in engagement in regularly scheduled series (hospital-based learning) and enduring materials.

In 2022, growth in physician learner interaction was sustained with total reported physician interactions of approximately 21.4 million.

Increase in Other Healthcare Learner Interactions by Format (Figure 3).

Figure 3. Activities offered in the ACCME system by Other Healthcare Learning Interactions (2019–2022).

Other healthcare professionals (e.g. nurses, pharmacists, physician assistants/associates [PAs], social workers, psychologists, dentists) are active learners in the ACCME system. There has been incredible growth in team-based learning in the U.S. over the last ten years.

From 2019 to 2022, healthcare learners showed a clear preference for enduring materials, followed by live activities and regularly scheduled series (hospital-based learning).

By 2022, the total healthcare learner interactions in the ACCME system was 32.3 million, an increase of approximately 60% from 2019. Education by the team for the team has encouraged a sustained engagement in the ACCME’s accredited education system by all members of the healthcare team.

II. Did you change any rules in your accreditation/licencing framework during the pandemic? If yes, have these changes been maintained in the post-pandemic period?

Like most jurisdictions, many of the licencing boards in the U.S. waived the continuing education requirements for re-licensure of physicians during the pandemic years. Interestingly, even in the absence of requirements, physician engagement in accredited CME increased. Those licencing rules have been reinstated.

As an accrediting body, the ACCME took a leadership role in establishing a COVID-19 Resource page (https://www.accme.org/covid-19-clinician-and-educator-resources.) for both clinicians and educators. The ACCME partnered with the CDC to ensure that the most current information about COVID-19 and the developing vaccines was available to physicians and healthcare professionals.

As well, the ACCME created a COVID-19 Educational Planning Form that allowed for the immediate deployment of accredited education, waiving accredited requirements that called for collection of financial relationship information of planners and faculty, mitigation of those relationships by the educators, and disclosure to learners. Given the urgent need for education around COVID-19, there was less concern about the insertion of commercial bias. While the COVID-19 Educational Planning Form has been retired, the ACCME was encouraged to develop similar quick tools for the planning and presentation of accredited education in circumstances where it was unlikely that commercial bias would be inserted. The ACCME has published those tools and resources on our webpage (www.accme.org).

AAFP

The American Academy of Family Physicians is the national association of family doctors in the United States. It is one of the largest national medical organisations, with 129,600 members in 50 states, D.C., Puerto Rico, the Virgin Islands, and Guam, as well as internationally. The AAFP was the first national accrediting body/credit system for physicians in the United States and is focused on accredited CME for family physicians. AAFP operates an activity-based system, where organisations apply on a per activity basis and those activities that meet AAFP’s eligibility requirements are approved for AAFP credit. In FY2023 (FY: fiscal year), AAFP engaged with 1,057 CME Provider Organizations and approved over 55,000 credits with AAFP members reporting over one million credits. All data is represented in AAFP’s fiscal years, which runs from June 1 through May 31.

• (1) What have been the three most striking findings in the AAFP system during and after the pandemic?

  • (a) First was the sharp transition from live in-person to live online sessions, which has remained the case post-pandemic. Sessions in our system represent each individual topic covered in a live meeting. For example, the Cologne Consensus Conference is one activity with eight sessions over two days. AAFP began differentiating between live in-person and live online sessions in FY2020. In FY2020 at its peak 8% of the live sessions were live online, while in FY2021 they were 41%. In FY2023 live online sessions dropped to 37%. It’s important to note that while live activities increased between FY2022 and FY2023, the overall number of live sessions declined. Live online is here to stay.

  • (b) Second, we saw a significant decline due to COVID in all activity formats except for enduring materials. The decline was highest in FY2021, as was the increase in enduring materials. From FY2019 to FY2021, we saw a 17% decrease in overall activities. By contrast, enduring materials increased by 50% from FY2019 to FY2021. While enduring materials appeared to have subsequently declined, it instead shifted to a new category, repurposed content, which was introduced in FY2022 in response to requests from our CME provider organisation community. Repurposed content allows CME provider organisations a streamlined pathway to accredit enduring education that was originally approved as live in our system. For example, if a CME provider organisation offered a live in-person or live online activity that was accredited in our system and they record (or repurpose) it and want to make it available to learners for credit, they can use our repurposed content option (which has a streamlined process and a reduced fee). We have seen roughly 20% (n = 80) enduring activities shift to repurposed.

  • (c) Third, was the test of our eligibility requirements and reviewers regarding how we reviewed controversial and emerging/new topics – early on when there was very little evidence on any treatment for COVID, then later with controversial treatments, misinformation, and so on. We found our eligibility requirements held up well to this test and we did not need to make any changes as a result.

• (2) Did you (have to) change any rules in your accreditation/licencing framework during the pandemic? If yes, have these changes been maintained in the post-pandemic period?

• (a) As an accreditor, the AAFP did not make any changes to our accreditation framework during the pandemic. We did offer flexibility for CME provider organisations to change dates and formats as needed. We also produced educational materials to assist CME providers in transitioning their activities from live in-person to live online or enduring. The AAFP as a medical speciality society did eliminate the live CME requirement for membership due to COVID and that remains in place.

ANDPC

The French National Agency for continuing professional development (CPD) is the public body in charge of the overall governance of CPD in France. The Agency processes the registration of all CPD providers and the assessment of all CPD activities for all health professions. Additionally, the Agency finances some categories of self-employed health professionals.

What have been the three most striking findings in the ANDPC system?

The emergence of a new formats such as virtual classroom and the e-learning. Indeed, a distinction is made in our system between these two formats.

The virtual classroom is defined as a distance learning format that requires the use of synchronous tools. It allows a time for direct exchange between speakers and the participants on the one hand and also between participants themselves on the other hand.

Figure 4, showing the distribution of number of published CPD activities according to format, clearly illustrates the significant increase of this type of format in 2021.

Figure 4. Distribution of number of published activities according to format.

The virtual classroom went from 0 to 10.2% of all CPD activities and it represented 5,7% of registrations in 2021 (Figure 5).

Figure 5. Distribution of number of registrations according to format.

The e-learning is defined as an asynchronous self-learning system using Information and Communication Technologies for Education (ICT).

As the figures show, this format experienced a clear increase during the COVID-19 pandemic. In fact, it represented 64,1% of all registrations of healthcare professionals in 2021 (Figure 5) and came in second position behind the traditional “Face-to-Face” format of all published CPD activities (Figure 4).

Even if most of the offer is face-to-face CPD activities, most of the registrations is related to e-learning CPD activities, and this trend continues over time.

Did ANDPC had to change any rules?

In front of the increase of new formats (virtual classroom and e-learning), the real challenge for the Agency is to define proper quality criteria. The ANDPC has initiated work in this way by working on recommendations for the deployment of these new formats by providers.

These recommendations have been written in the form of points of vigilance in terms of the number of participants by sessions, the duration, the adequacy of the theme with the format used, etc.

To illustrate the work done by the Agency, we provide some examples of recommendations according to format:

• - For the virtual classroom: maximum of 15 learners, regular breaks, necessary interactions between learners and speaker, no learning of technical gesture, a virtual classroom is not a webinar with e-learning

• - For e-learning: must have participatory modules so that the learner is not passive (just showing a slideshow or video without learner’s intervention are not accepted), pre and post-tests to evaluate acquired knowledge, the methods for monitoring the learner’s participation and the evaluation of training must be specified: QUIZZ, CHAT, etc.

These recommendations will be supplemented in 2024 by the work of the Agency’s scientific council, the High Council of CPD.

EBAC

The European Board for Accreditation of Continuing Education for Health Professionals (EBAC) is a non-governmental accrediting body not linked to any system of re-certification, re-licensure or something similar. Thus, categorisation of learning modalities is only for internal administrative reasons. For the purpose of this publication we have only been able to differentiate between in person onsite and online education, respectively, since some of the new categories have not been used in pre-pandemic times. EBAC offers activity as well as provider accreditation

I. What have been the three most striking findings in the EBAC system?

• (1) Sharp and sustained increase in e-learning in absolute and relative terms.

• As shown in Figure 6 online education substantially increased in 2020 and showed its highest values in 2021 with a slight reduction in post-pandemic years (data for 2023 are extrapolated from the first 6 months in 2023). Surprisingly, the number of educational activities accredited by EBAC increased during the pandemic. This increase is in part real, but in part pseudo-real, since

  Figure 6. Relation between onsite in person meetings and online education 2019–2023 (bars, %), number of all educational activities per year (2019–2023) (solid line, % in relation to total number of applications in 2019).

  

• - some providers have e.g. split up a one day in person meeting into several webinars

• - and we have counted hybrid activities as in person and online

Thus, these numbers reflect a sustained increase in learning opportunities during and post pandemic.

(2) Speed of change: in 2020 only about 5% of activities, for which accreditation had already been applied for, had finally to be cancelled. For 2021 providers obviously had decided to run no risk and switch to online education, reflected by the overall increase in educational activities as well as by the fact that no application had to be cancelled due to the pandemic.

(3) Complexity of programmes: During and after the pandemic EBAC has received educational programmes for accreditation in which in-person onsite parts have been combined with online only and hybrid (in-person onsite with live stream) sessions in the same (multi day) meeting. In addition, substantial parts of the programme have been recorded and offered later as “on demand” learning.

II. Did EBAC had to change any rules?

EBAC has already accredited e-learning including micro-learning prior to the pandemic. Thus, EBAC had not to change any rules, but just show some administrative flexibility.

MoPH-Qatar

The Accreditation Section (AS) of the Department of Healthcare Professions (DHP) – Ministry of Public Health (MoPH), Qatar, is the national authority responsible for both accrediting the CME/CPD activities of all healthcare professions in the state of Qatar (accreditor) and regulating the participation of all healthcare practitioners in Qatar’s National CME/CPD Programme (regulator). The DHP-AS, therefore, was in a good position to monitor and report on the changes and trends in CE in the years before, during and after the COVID-19 pandemic and answer the following questions:

• (I) What have been the three most striking findings in MoPH-Qatar system during and after the pandemic?

  • The number of conducted accredited (Category 1 & 3) CPD activities and their related statistics (e.g. the number of conducted CPD activity offerings, the accredited and educational hours, educational opportunities and granted credits) decreased markedly in 2020, when compared to 2019 (fig. 7 a-f). The numbers started climbing back in 2021 and nearly approached the pre-pandemic state in 2022.

    Figure 7. (Continued).

    

  • The percentage of accredited CPD activities conducted in online or hybrid formats (fig. 7e-f) increased markedly during the pandemic years (33.1% and 54.3% of offerings conducted in 2020 and 2021), when compared to the pre-pandemic years (6.6% of offerings conducted in 2019) (fig. 7-b). The educational opportunities and credits provided/granted by online and hybrid formats followed similar trends (educational opportunities and credits provided by online and hybrid formats were 7.2% and 3.5% in 2019 and reached up to 70.7% and 74.1% in 2021). The online and hybrid formats also continued to represent a major type of CE activities afterwards (accounting for 47.4% of the conducted CPD activity offerings, 66.2% of provided educational opportunities and 59.4% of granted CPD credits in 2022).

    Figure 7. Number of Conducted Accredited Activities.

    * Accredited Hours are the sum of Credit hours assigned for all accredited activities

    

  • The average duration of conducted accredited CPD activities (average number of conducted educational (instructional) hours) showed a noticeable decrease during the pandemic years (averaging 3.05 hours and 2.63 hours in 2020 and 2021), when compared to the pre-pandemic years (averaging 3.69 hours in 2019). This can be explained by the trending of online and hybrid format CPD activities which tend to be shorter in duration (averaging 1.76 hours) than face-to-face CPD activities (averaging 3.74 hours). On the other hand, the educational opportunities provided by online and hybrid format CPD activities averaged higher levels (50.8 educational opportunity) when compared to the average provided by face-to-face CPD activities (31.1 educational opportunities), which helped to sustain the availability of CPD credits with the decreasing number of educational hours provided by the accredited CPD activities.

(II) Did MoPH-Qatar have to change any rules in its accreditation/licencing framework during the pandemic?

• • As the national CPD accreditation body, DHP-AS made no changes in its rules of CPD accreditation during the pandemic, as the accreditation system was designed to encompass a wide variety of CPD activity types and formats (including the accreditation of online and hybrid formats of CPD activities). However, DHP offered some flexibility regarding aspects that do not affect the quality of accredited CPD activities (e.g. the minimum duration required for submission of accreditation applications, activity date changes, etc.)

  • As the regulator of healthcare practitioner participation in the National CME/CPD Programme, the DHP offered several means of support to the healthcare practitioner during the pandemic, including:

  • Provision of guidance on how to participate in DHP-accredited CPD activities (especially those in online/hybrid formats) during the pandemic and the presence of gathering restrictions,

    • - Ensuring the availability and easy access (of all healthcare practitioners) to evidence-based COVID-19-specific educational resources, in collaboration with the pertinent authorities and national CPD providers.

    • - Waiving part of the mandatory CPD credits and extending the licence validity for those who were due for licence renewal during the peaks of the pandemic

RCPSC

The Royal College of Physicians and Surgeons of Canada (RCPSC) is the National Professional Association that oversees the medical education of specialists in Canada and accredits organisations to develop and deliver that education. We protect the health of Canadians by helping physicians build skills, knowledge and expertise in their specialities. Our Maintenance of Certification Programme supports specialist physicians through lifelong learning and professional development and ensures their continued competence in a constantly evolving medical environment. We also accredit the university programmes that train resident physicians for their speciality practices, we compose and administer the demanding examinations that residents must complete to become certified as specialists. We support global standards in health care, training values and professional development by sharing our educational activities and tools with sister colleges in other countries. We establish relations with global partners and assist them to build their capacity to provide speciality medical education and create their own sustainable health systems.

• (I) What have been the three most striking findings in your system during and after the pandemic?

The data demonstrated in Figure 8 provides a high-level summary of the last five years of the Maintenance of Certification (MOC) Programme Data (2018–2022). It is important to note that MOC reporting requirements were waived for 2020 and that is reflected in the data.

Figure 8. The data demonstrated in this figure provides a high-level summary of the last five years of the Maintenance of Certification (MOC) Program Data (2018–2022). It is important to note that MOC reporting requirements were waived for 2020 and that is reflected in the data.

1. First finding: Participants

In the left panel is an overview of the Number of Participants reporting CPD overall as well as a break down by Section (there are 3 sections in the MOC programme):

• Section 1: Group Learning

• Section 2: Self Learning

• Section 3: Assessment

As of 2022, participants reporting Section 1, 2, and 3 activities have returned to pre-pandemic levels (2018/2019) and with modest increases in the number of participants reporting in each section as well.

2. Second Finding: Activities

The centre panel illustrates a breakdown of the Number of Activities reported overall as well as by Section. Following a drop in Section 1 activity reporting in 2020, there was an increase in credits claimed in 2021 and while the activity reporting dropped slightly in 2022, it is still higher than pre-pandemic levels. Interestingly, Section 2 reporting dropped in 2020 and while there has been a sustained increase in 2021/2022, the number of activities reported is below pre-pandemic levels of reporting. Activity reporting in Section 3 has increased 14% from 2018 and is higher than pre-pandemic and pandemic levels.

It is important to note that part of section 1: Group Learning also includes rounds, journal clubs, and small group sessions (12 individuals or less). These activities are part of our self-accredited programme and we have seen an increase in these activities once the pandemic had started.

3. Third Finding: Credits

The rightmost panel provides an overview of the Number of Credits reported overall with a breakdown by Section as well. While the number of credits claimed has declined in Section 1 and 2 since 2018, credits reported in Section 1 dropped the most at 12% with a smaller decrease in credits reported seen for Section 2 (−6%). Section 3 is the only section in which there was growth with more credits reported as compared to pre-pandemic levels.

II. Did you change any rules in your accreditation/licencing framework during the pandemic? If yes, have these changes been maintained in the post-pandemic period?

At the onset of the pandemic, understanding the need to support the development of COVID related activities, the RCPSC organised a group of representatives (called the Rapid Response Team) from 3 distinguished accredited groups: National Speciality Societies, University CME Offices, and Simulation Programmes. This team was tasked to support any providers to act as a Scientific Planning Committee (SPC) and even supported minor accreditation tasks. They established an auto-accredit policy for any COVID activities as long as they were developed by a Physician Organisation with no external funding; this policy is no longer in place.

By September the group transitioned to supporting providers in transitioning from in person meetings to virtual and developed the Webinar Series (still going today) that gathered and shared best practices in delivering online activities. The Rapid Response Team has now changed to the SPC for the revamped Webinar Series and many of the existing members are still play a part in supporting this.

The RCPSC also established a resource page for all its members and providers by gathering and curating any relevant COVID activities both within and outside of Canada. This resource was open to all of its members and anyone else seeking information.

A minor policy change that was implemented at the onset of the pandemic was allowing accredited providers to accredit COVID activities after they had been developed. This was done as many providers did not the administrative capacity to submit to the College their activities. This policy is no longer active.

As noted in the first part, MOC reporting requirements were waived for 2020.

Discussion

Data from six different accreditation systems demonstrate that complete lockdown periods in the first part of the COVID-19 pandemic have inevitably led to reductions in numbers of the then predominant format of education, i.e. onsite in-person meetings. However, with impressive speed CE providers have switched to online educational formats. With regard to learner interactions this has compensated, and in some systems even overcompensated, the loss of in-person educational opportunities. The main driving forces for speed of change probably have been

• - that technology to organise virtual meetings has already been available since pre-pandemic times

• - the need to quickly align with evolving knowledge and inform about ever changing regulatory frameworks in a pandemic caused by an agent so far unknown globally

• - that accrediting bodies have offered support to identify independent and trustworthy information on COVID-19 and/or facilitated rapid production of accredited content

• - to maintain activities aiming to facilitate scientific exchange and make progress in non-pandemic related areas

• - not to lose support of membership and commercial supporters.

The exclusive availability of virtual educational formats during the pandemic has further added a hitherto underestimated quantitative dimension to certain capabilities of online education, which has demonstrated to

• - effectively inform teams under time pressure (3)

• - substantially increase outreach, allowing in particular young and/or participants from low- and middle-income countries to participate (4,5)

• - allow learners to selectively choose content according to their preferences (4–6)

• - increase sustainability by allowing repetitive use of the same educational content (6)

It is probably because of these qualitatively and quantitatively positive experiences that online educational formats have come to stay also after the pandemic has ceased, as demonstrated by our data. However, since numbers of online educational activities recently have shown a slight decrease, the future will show what the long-term role of online education (the “new normal”) is going to be.

Of note, though medical workforce went from crisis to almost collapse during the pandemic, CE providers’ ability to deliver, and health workers’ motivation and practice of CPD, have turned out to be robust and withstood all pandemic challenges. Thus, regulatory authorities should critically evaluate which of the regulations, they have temporarily suspended during the pandemic, really matter, to not further distress a burnt-out profession [7,8].

In conclusion, our data convincingly demonstrate the resilience of CPD in times of a global health crisis and offer important insights in how CPD might become more effective in the future.

Limitations

This study retrospectively utilised administrative data from different accrediting bodies. It was therefore unavoidable that, for example, the survey periods were defined differently in some cases and the nomenclature for classifying the CE types was not standardised, the latter in particular because, for example, hybrid events were only offered to any significant extent from the pandemic onwards. Even though all of this may have had an impact on quantitative figures, the general trend is still very clearly recognisable.

The purpose of the analysis of similar data across different accrediting bodies was focused on trends related to formats and participation. The analysis did not query requirements related to content validity or independence. However, as online education was to a large extent conversion of CE originally planned as onsite in person events, it can certainly be assumed that the quality was equivalent. Online education proved to be critical for disseminating the rapidly changing patient care recommendations during the pandemic [3].

Disclosure Statement

Disclosure of interests of all authors can be found under “supplementary material”.