Report on Proceedings of the Sixteenth Annual European CME Forum, the Hague, Netherlands, November 2023

ABSTRACT

There was a renewed vigour among the participants attending the 16th Annual European CME Forum (#16ECF), which took place in the Hague, the Netherlands from 8–10 November 2023. This emanated from the meeting having reverted to an in-person format to promote engagement among attendees. The meeting was subdivided into three main sections to address this year’s theme “Voices in CME-CPD”: 1) Listening to others; 2) Listening to ourselves; 3) Listening to each other. The Forum unofficially began with the pre-meeting sessions led by two special interest groups and was then formally opened by the programme director. There were panel discussions on designing and implementing CME-CPD programmes, measuring and reporting outcomes, and valuing and defining independence, as well as smaller workshop and breakout sessions led by international presenters. Representatives for the Journal of CME presented on the 2023 Special Collection of articles, with this year’s topic “Expanding the voices in CME-CPD” mirroring the meeting theme. Participants interacted with providers, accreditors and grantors, as well as poster authors and a local physician who attended to share a learner’s perspective. The meeting concluded with the now familiar “CME unsession” to ensure everyone’s voices were heard and no one left the meeting with any unanswered questions.

KEYWORDS:

• CME/CPD
• CME
• medical education
• outcomes
• microlearning
• independence
• EACCME

On its return to live meetings in a venue in the Netherlands, the 16th iteration of the Annual European CME Forum welcomed a high number of first-time attendees who seemed to boost the energy and interactivity of the meeting. The meeting theme this year was “Voices in CME-CPD”, which was addressed from three angles: 1) Listening to others, including learners, patients and the many other groups and organisations that can help support us in CME-CPD; 2) Listening to ourselves, considering how we can learn from and support each other; and 3) Listening to each other, and how we can come together to work more collaboratively in European and global CME-CPD.

The Forum began with two “pre-meeting” sessions led by the Good CME Practice group (gCMEp, gcmep.org) and the Biomedical Alliance in Europe (BioMed Alliance, biomedeurope.org). The gCMEp is a coalition of providers based in Europe whose mission is to promote professionalism within their community in the delivery of top quality Independent medical education (IME) to the healthcare professional (HCP) learners. Representing the gCMEp, Monica Ghidinelli of AO Foundation and Sophie Wilson from International Medical Press discussed current challenges facing European providers and explored potential solutions through dialogue with the attendees.

Much of the discussion centred on recent changes in the accreditation criteria published by the European Accreditation Council for CME (EACCME), with participants voicing concern over the criteria for CME funding in Europe. Many of these concerns were previously outlined in a letter to the editor published in the Journal of CME [1], which focused mainly on the role of industry in funding CME with particular reference to satellite symposia occurring at large congresses. It would be fair to say that discussion did little to alleviate the concerns expressed! Figure 1 shows the letter’s signatories.

Figure 1. Letter to the editor outlining the ongoing challenges faced by providers of CME-CPD in Europe, prepared by members of the Good CME Practice group and published in the Journal of CME in October 2023 [1].

The following session was presented by David Vodušek and Robin Stevenson on behalf of BioMed Alliance, which represents 34 leading biomedical societies throughout Europe. These societies collectively have a membership of more than 400,000 researchers and health professionals. The presenters bemoaned the paucity of representation from European medical societies at the Forum, suggesting that it could provide them with a broader platform for useful exchange of ideas and information (Figure 2).

Figure 2. Member societies of Biomedical Alliance in Europe.

The presenters also expressed concern about the updated EACCME criteria but praised the move towards more provider-based accreditation in Europe, such as the EACCME Trusted Provider status. A discussion point was the identification of professional practice gaps by societies in the design and implementation of many of their educational activities. Stevenson likened the process to a reflex arc and criticised the dearth of information used in the “afferent limb” in this reparative process. The suggestion was made that, as key stakeholders, societies should engage more with the wider CME community and provide guidelines for the identification of professional practice gaps. Suggestions are summarised in Figure 3.

Figure 3. Suggestions for European medical societies to improve their approach to CME.

Opening of the 16th Annual European CME Forum

In his introduction to the main meeting, Programme Director Eugene Pozniak noted that more than half of the attendees were here for the first time. He reviewed the previous 12 months of the CME landscape in Europe and beyond and highlighted two key events for European CME Forum, namely the change to the Journal of CME (previously Journal of European CME) and the recent publication from the gCMEp [1]. Pozniak discussed the planning process for the meeting, which relies on input from respondents to the annual survey, and the importance of interactivity. To ensure the voices of participants would be heard throughout the meeting, participants were encouraged to use the Slido Q&A and polling platform with live feeds on social media (Figure 4). Pozniak also highlighted the resources available, which would be updated over the course of the meeting (Figure 5).

Figure 4. Use of social media to add participants’ voices at 16ECF.

Figure 5. Example list of resources available to attendees.

2023 Special Collection

The first session, under the section “Listening to others”, was a panel discussion led by the Journal of CME 2023 Special Collection Guest Editor Lisa Sullivan (In Vivo Academy) with Carolin Sehlbach (Maastricht University). The invited panellists Reinhard Griebenow (EBAC), Sarah Nisly (Clinical Education Alliance) and Stan Pogroszewski (DKBmed) were all authors of articles accepted for inclusion in the 2023 Special Collection on “Expanding the voices in CME-CPD”, which is freely available to read at the journal’s website [2]. As well as answering questions on the content of their personal submissions, the panel touched on various points such as:

• How to increase the submission rate for special collections

• The importance of choosing appropriate reviewers who can provide useful feedback to authors

• The need for a longer planning cycle for the promotion, collection, review and publication of special collection articles

• The cost of submission to an open access journal and the range of exemptions and support available for authors

Oral poster presentation

The first of three oral poster presentations featured at the Forum addressed patient-centric CME and was presented by Meghan Coulehan of Answers in CME (Figure 6). Coulehan outlined a rationale for the involvement of patients or patient advocates in the education of HCPs (Figure 7), described some best practices for the process, and introduced the newer concept of patient-tethered learning that encompasses separate but aligned educational activities for both patients and clinicians.

Figure 6. Excerpt from the poster “Patient-centric CME: A paradigm shift in medical education” by Meghan Coulehan, Emily Kitterman and Katie Eustace (Answers in CME).

Figure 7. Oral poster presentation “Patient-centric CME: A paradigm shift in medical education” by Meghan Coulehan (Answers in CME) highlighting the rationale for patient involvement in CME-CPD.

Workshops

The Forum continued with the “elevator pitches” by the workshop leaders, which are given to influence participants to select their workshop from a series of concurrent workshops. The four workshops were as follows:

Essential ingredients to cooking up a proposal: From concept to dissemination

Led by Celeste Kolanko (Liberum IME) and Sophie Wilson (International Medical Press). The workshop leaders in this hands-on cookie decorating session provided ideas for grant proposals based on the analogy of identifying the essential ingredients of a proposal and how they should be blended to form a cohesive grant application. Some key points discussed were:

• Be as specific and succinct as possible

• Objectives should be linked to the target audience

• Identify situations where a team-based intervention may be more appropriate

• The root causes of practice gaps should be identified

• Make the proposal a “good read”

Levelling up on the educational activity evaluation process

Led by Steven Kawczak (Cleveland Clinic) and Kurt Snyder (Stanford University). Kawczak and Snyder led a practical workshop that considered various methods used in the evaluation of educational activities, highlighting the role of evaluation in:

• Determining the effectiveness of educational activities

• Identifying opportunities for quality improvement (QI)

• Measuring change in participants’ knowledge, competence, skills and behaviour

• Providing outcomes information to stakeholders such as funders and parent organisations

• Publishing and sharing knowledge

• Demonstrating the value of education

They described various tools and frameworks for evaluation, such as Moore’s model (Figure 8), and helped the participants to review a case scenario and design an evaluation strategy using Moore’s framework.

Figure 8. Evaluation tools applied to Moore’s model as presented in the workshop “Levelling up on the educational activity evaluation process”, led by Steven Kawczak (Cleveland Clinic) and Kurt Snyder (Stanford University).

Performance improvement and quality improvement CME: An overview for European and global providers

Led by Suzanne Murray (AXDEV Group). This workshop presented attendees with some innovative ideas about the link between evaluation and performance improvement and quality improvement (QI/PI) that can be applied on a global basis. It was suggested, in a similar vein to ideas presented in some of the other workshops, that education alone may not improve clinical practice when system changes may also have to be implemented. One of the challenges that face educational providers is that HCPs may not be fully aware of what QI/PI actually means in practice and that a collaborative mindset is necessary to implement best practices in improving quality and performance in clinical settings. Quality improvement was described as a systematic, continuous set of structured procedures designed to improve the provision of services to, and ultimately better outcomes for, patients, whilst PI, with more emphasis on the human factor, aims to reduce variation of processes and improve overall efficiency and effectiveness within a healthcare setting.

Key takeaways from this workshop were:

• Terminology may need clarification, for example “systemic diagnostics” rather than “needs assessment”

• Eliminate the term “outcome” and measure “impact” that is linked to learning objectives

• Publish, publish, publish!

Grant proposals: The good, the bad, and the really, Rreally, really ugly

Led by Lawrence Sherman (Meducate Global) with Allison Gardner (Med-IQ) and Sue Ellen Touma (Regeneron). This workshop provided participants with the opportunity to discuss grant applications with two experienced providers and a representative of a grantor organisation. The main points that emerged were:

• The use of appropriate data is a key component of successful grant applications

• Proposals should be concise and focused, there is no need for 70-page documents

• Learning objectives should be actionable

• Budgets should be realistic and linked to outcomes

• Gaps may sometimes be system-based and not solvable through an educational activity

• The level of feedback from grantors will depend on company policy

• Applicants should study grantors’ portals and websites thoroughly

• Collaboration with appropriate partners should be agreed upon as early as possible

• Learner data may be more important than expert data

Day 2 began with a workshop reflection session moderated by Amy Bernard (Medscape) to provide all participants with an insight into the first day’s workshops, with workshop leaders providing a summary of the key discussion points and common themes that had arisen.

Oral poster presentation

The main sessions of Day 2 under section “Listening to ourselves” were introduced and led by Chris Elmitt, an independent moderator, who made a welcome return to the Forum. Elmitt set the scene before vacating the stage to allow Allison Gardner (Med-IQ) to give an oral poster presentation on the use of Bloom’s Taxonomy in instructional design (Figure 9). Gardner explained how the use of such a framework could be incorporated into a backwards planning model for the design of educational activities by selecting action verbs for learning objectives, then mapping these action verbs to cognitive domains. The final phase in this instructional design process draws all the elements together (Figure 10).

Figure 9. Excerpt from the poster “Bloom’s Taxonomy in instructional design: applications beyond learning objectives” by Allison Gardner (Med-IQ).

Figure 10. Oral poster presentation “Bloom’s Taxonomy in instructional design: applications beyond learning objectives” by Allison Gardner (Med-IQ).

Designing and implementing programmes

The first panel discussion of the day, led by Chris Elmitt, considered the topic of “Designing and implementing programmes” with accreditor Reinhard Griebenow (EBAC), Robin Stevenson (Journal of CME, editor-in-chief) and provider Margarita Velcheva (Kenes Group). Their discussion touched on:

• The need to align education to changes in the post-COVID world of HCPs schedules by means of point-of-care and online learning

• Using local and regional data to provide more specific guidance on practice gaps

• Realising that social and cultural issues may contribute to community health gaps and barriers between competence and performance, with an example provided on a shortfall in written plans for diabetes patients due to a lack of printed forms

• Learners want engagement and faculty must be properly prepared and able to facilitate this

The “listening” angle was further addressed after the panel discussion in four concurrent breakout sessions that allowed attendees to talk about a range of topics in a smaller group format.

Making the most of every minute: Mastering short format and microlearning education

Led by Meghan Coulehan and Katie Eustace (Answers in CME). The breakout leaders described the concept of microlearning as the use of “bite size” segments of highly focused content which is adaptable to learners’ pace of engagement and reflective how most content is consumed. They described the mechanism of action based on a combination of theories of limitation of the brain, where cognitive load aligns with multimedia learning in the brain’s capacity to process a limited amount of information at any one time via auditory and visual operations. Attendees participated in a small group exercise to produce content in microlearning style on one of two topics, namely “Total Football” and “How windmills make flour”. In attempting to link content to learning objectives, one group indicated that they had used artificial intelligence (AI) to provide summary material. The breakout leaders argued strongly for greater use of microlearning in CME/CPD, asserting that it can be impactful (Figure 11).

Figure 11. Breakout session “Make the most of every minute: mastering short format and microlearning education” by Meghan Coulehan and Katie Eustace (Answers in CME) highlighting the impact of microlearning education.

Tips and tricks when selecting and managing faculty

Led by Monica Ghidinelli and Jane Thorley Wiedler (AO Foundation). Ghidinelli and Thorley Wiedler shared their experiences in selecting and managing faculty for educational activities, highlighting some important selection criteria and preferred characteristics for faculty as follows:

• Diversity with respect to nationalities and genders

• Timely responsiveness in planning and communication

• Good diction in the language used for presentation

• Digital opinion leaders

• Passionate about their area of expertise

They described some challenging scenarios and suggested ways to overcome obstacles in such situations by setting parameters to guide those who make decisions on the choice of faculty (Figure 12).

Figure 12. Breakout session “Tips and tricks when selecting and managing faculty” by Monica Ghidinelli (AO Foundation) and Jane Thorley Wiedler (AO Foundation) outlining faculty selection criteria.

Tik Tweet Tok: LinkingIn to #effective #education on social media

Led by Sophie Wilson (International Medical Press). This breakout session encouraged attendees to recognise the growing power of social media as a legitimate tool to help HCPs access information that is pertinent to their practice. Examples such as “Tweetorials” were discussed, where threads of comment are interspersed with videos and polls to provide appropriate content. Challenges were also discussed, including issues related to accreditation of such activities, the lack of any framework of best practice in the use of social media, and the need to consider how to measure its impact more effectively (Figure 13). The growing reliance on social media by the so-called “Generation Z” was also mentioned as this demographic group contains the next target audience of HCPs for CME/CPD.

Figure 13. Breakout session “Tik Tweet Tok: LinkingIn to #effective #education on social media” by Sophie Wilson (International Medical Press) highlighting the growing use of social media to provide educational content to HCPs.

Mind the gap: How to identify needs in your learners

Led by Anna Muehlenhaupt (DKBmed). Muehlenhaupt provided the breakout attendees with discussion points to consider some innovative and non-traditional methods of conducting gap analysis to identify learner needs (Figure 14). An exhortation was made to approach the process in a more collaborative manner such as seeking input from different constituent groups simultaneously, for example talking to clinicians and patients at the same time to generate data on learning gaps.

Figure 14. Breakout session “Mind the gap: How to identify needs in your learners” by Anna Muehlenhaupt (DKBmed) outlining an innovative approach to gap analysis.

Lunchtime session: Getting to know our audience

Following the morning breakout sessions, Lisa Sullivan (In Vivo Academy) was in conversation Meindert Sosef, a local surgeon and Dean of Medical Education at the Atrium Medisch Centrum, for an illuminating lunchtime session. Sosef described the process whereby HCPs gain points to maintain their licences through educational and skills training and how points may be obtained through hospital-based education as well as from traditional educational meetings outside the workplace. He indicated that team-based training, some mandatory such as CPR training, was an integral part of the educational offering at his hospital and that a “teaching climate” existed therein with faculty training including role play and the use of actors as patient models in place at the facility. Training takes place along the continuum of medical practice with student trainees able to provide constructive assessments of teaching staff and peer feedback used to help assess professional skills.

Some of the educational techniques in use at Atrium Medisch Centrum are simulation, such as the use of smart glasses and gaming for surgical skills. Sosef acknowledged that more emphasis was being placed on the use of social media in the medical education programme and that shorter formats such as microlearning were becoming more commonplace in the delivery of their training and education. In response to a question on the inclusion of patients in the educational milieu, he noted that a new facility was being built to house an academy for patients, families and carers to help them learn techniques such as self-catheterisation, application of eye drops and self-injection, and to provide opportunities to enhance digital skills. This planned academy is intended to enhance current processes of “mirror conversations”, where patients are able to share their experiences with clinical staff.

Measuring and reporting outcomes

The next panel discussion under the umbrella of “Measuring and reporting outcomes” saw participants from the UK and US share a number of salient points linked to current perspectives on evaluation methods and what is actually being measured. Dean Jenkins (Agnate), Celeste Kolanko (Liberum IME) and Kate Regnier (ACCME) discussed the need for an evaluation process to demonstrate the value of the planning and implementation of CME/CPD educational activities and to assess the performance of educational providers. This was particularly applicable to provider accreditation, with Regnier describing some of the “push and pull” processes used by Accreditation Council for Continuing Medical Education (ACCME) to nudge providers towards achieving higher level outcomes to go beyond measurement of competence and performance. The ACCME Commendation Criteria seeks to provide an incentive to help in this process. Kolanko and Jenkins also highlighted some challenges, such as the cost of measuring outcomes at higher levels, and suggested that some of these challenges could be overcome by focusing clearly on what can and should be measured for a particular activity, such as by designing longitudinal formats that allow measurements within or during an activity. This discussion was followed by a set of breakouts that related to measuring and reporting outcomes.

Evaluating educational activity effectiveness

Led by Steven Kawczak (Cleveland Clinic) and Kurt Snyder (Stanford University). Kawczak and Synder built on their previous workshop session seeking to identify evaluation methods using an established framework and considered the pitfalls that might undermine findings from an evaluation. They described the evaluation implementation process in terms of defining and planning a strategy, assessing the learning, analysing and sharing the results, and making improvements. They compared Kilpatrick’s [3] and Moore’s [4] frameworks and facilitated an exercise using two case scenarios that allowed the participants to design the outline of an evaluation process using Moore’s model. The discussion that followed brought up additional points, including:

• The need for the learners to commit to engage in the evaluation process

• Agreeing on what level of change is acceptable in terms of desired outcomes

• Accepting that measurement of outcomes is a process rather than a one and done situation

Traditional versus non-traditional ways of measuring outcomes

Led by Dean Beals and Stan Pogroszewski (DKBmed). This breakout session presented ideas on ways that providers can move towards higher level outcomes by using a variety of data collection techniques and incorporating qualitative measures into the process. One of their key points echoed one made by Kawczak and Snyder in another breakout session, where the importance of learner engagement and commitment to change was emphasised. It was also noted that access to insurance claims and electronic health record data was much more prevalent in US than elsewhere. The more modern measuring techniques and differences from the more traditional methods are illustrated in Figure 15. The figure also shows the term “outcomes” has been replaced by “impact”, as suggested in the earlier workshop on PI/QI presented by Suzanne Murray (AXDEV Group).

Figure 15. Breakout session “Traditional versus non-traditional ways of measuring outcomes” by Dean Beals and Stan Pogroszewski (DKBmed) outlining different ways of measuring outcomes (or impact).

Are short format outcomes impactful? Development and reporting methodologies

Led by Meghan Coulehan and Katie Eustace and (Answers in CME). This session championed the concept of microlearning and showcased an example of a successful activity that demonstrated the link between content development based on short measurable goals and outcomes success. They described the various components of the process (Figure 16) and a real-life example (Figure 17).

Figure 16. Breakout session “Are short format outcomes impactful? Development and reporting methodologies” by Meghan Coulehan and Katie Eustace and (Answers in CME) summarising the learner experience in short format educational activities.

Figure 17. Breakout session “Are short format outcomes impactful? Development and reporting methodologies” by Meghan Coulehan and Katie Eustace and (Answers in CME) outlining an example of a successful short-format activity.

Mastering the metrics: Enhancing outcomes assessment in independent medical education

Led by Froukje Sosef (COR2ED) and Margarita Velcheva (Kenes Group). Sosef and Velcheva led a discussion on the fundamental questions in educational outcomes (Figure 18) and from this discussion some familiar points emerged. The “why” measure outcomes from the perspective of a provider question elicited responses from participants related to a desire to show improvement in the status of learner, provider and patient, as well as providing feedback to grantors where appropriate. A range of methods such as observation, surveys, polls, tests, interviews and clinical audits were suggested in response to the “how” do you measure outcomes question. However, most of the discussion focused on the “what” outcomes are reasonable and realistic to achieve across various programme formats question, with particular reference to issues associated with implementing educational activities in multiple countries where multiple Institutional Review Board approvals might need to be sought for access to patient level data. The consensus was that a realistic approach to gathering such data was necessary and that benefits could be obtained by aggregating data over time to contribute towards reaching higher level outcomes.

Figure 18. Breakout session “Mastering the metrics: Enhancing outcomes assessment in independent medical education” by Froukje Sosef (COR2ED) and Margarita Velcheva (Kenes Group) asking key questions on educational outcomes.

Valuing and defining independence

The final panel discussion of the day was moderated by Chris Elmitt and focused on “Valuing and defining independence” with UK-based provider Sophie Wilson (International Medical Press), US-based industry representative Sue Ellen Touma (Regeneron) and regulatory body representative Jean-Philippe Natali (Agence Nationale du DPC) from France. The key points that arose from the discussion were:

• Independence does not simply mean separation from the funding source

• Industry finance is not really a factor in France

• It is probably an axiom that “expert” faculty will have the most potential conflicts of interest

• Bias may exist in situations where readily available faculty are selected

• The UK has some specific challenges because of the failure of the Association of the British Pharmaceutical Industry (ABPI) to sign up to Article 16 of the European Federation of Pharmaceutical Industries and Associations (EFPIA) Code of Practice

• There is more ambiguity in accreditation standards outside US

• IME must be considered as distinct and separate from company-led education

This discussion continued in smaller groups where the perspectives of accreditors, providers and industry were shared with participants by working through a case scenario (Figure 19). The breakout session leaders then returned for a final reflection session and shared details of the proceedings from each breakout. There was general consensus in response to the points raised in the breakout case scenario (Figure 19) and further discussion as follows:

Figure 19. “Valuing and defining independence” case scenario in the breakouts exploring stakeholder perspectives.

• The provider seeks funding from a drug company, the supporter’s drug will be mentioned in the educational activity: The prevalent viewpoint was that this is common practice and generally acceptable with the onus on the educational provider to ensure fair balance and content validity

• The provider develops the content and submits it to the supporter for review: This example was regarded as being unacceptable in terms of independence and the situation in UK was cited as problematic due to the stance on Article 16 of the EFPIA Code of Practice adopted by ABPI

• The provider asks the supporter for help with learner recruitment: It was suggested that the role of an industry supporter should be confined to providing activity details in the form of “save the date” materials but that provision of access codes for online content or enduring materials would be unacceptable

• During the event a representative of the drug company attends the session and corrects the inaccuracy of the speaker: It was generally agreed that this would not be acceptable and that the responsibility for making corrections lies with the educational provider

• After the event the drug company supporter requests a list of the learners from the provider: Although the consensus was that this flies in the face of independence, it was noted that in the US under ACCME rules, learners could opt for their contact information to be made available

How CME-CPD is implemented around the world

The Forum continued on Day 3 under the topic “Listening to each other”, with contributions from a panel representing Ukraine (Rostyslav Marushko, MedExpert Group of Companies), Saudia Arabia (Lama Aldakhil, Saudi Commission for Health Specialties), India (Vaibhav Srivastava, Insignia Learning) and Australia (Lisa Sullivan, In Vivo Academy).

Marushko began the session by describing the transition of CME/CPD in Ukraine from what he called an “old Soviet era” to a new regulatory system, set up in 2018 by the Ministry of Health, which provides accreditation with a programme and activity recording system similar to that run by ACCME. The new system is evolving and whilst activity accreditation is still the norm, there is progress towards provider accreditation. Indeed, Marushko’s organisation MedExpert is accredited by the ACCME. Acknowledging that obstacles exist due to the ongoing conflict in Ukraine and difficulties in travel since the COVID-19 pandemic, he was keen to explain that relevant educational content from outside Ukraine is welcomed by that country’s HCP community.

Aldakhil highlighted some parallels with Ukraine in describing the evolving system in Saudi Arabia run by the Commission for Health Specialties, which has responsibility for most aspects of healthcare in her country. She explained that the provision of healthcare is shared among governmental, private and non-profit organisations and the mixed system of activity and provider accreditation that is currently in place, with plans to move towards a provider-only system. Aldakhil also described the requirement for physicians and dentists to obtain credit for 60 CPD hours over a two-year registration period that is recorded via a self-directed learning portfolio. Saudi Arabia is building a national CPD framework and is engaged in a capability development programme that seeks to build a community of practice in CPD, which will include a mentorship programme run by trained CPD champions that may extend to other countries in the region.

The situation in India was described by Srivastava as being highly dependent on funding by industry in a federal system where each of the 28 states has its own system of regulation for the provision of CME by local, national and global sources. Although CME provision is advisory rather than mandatory, there is a plethora of available and high-quality education. Both providers and activities are regulated.

Sullivan described the self-reporting system for specialists’ CME in Australia via a personal development plan. In contrast, an accredited provider system does exist for primary care physicians who are now subject to new requirements of 100 points per year over a three-year cycle, compared with the previous requirement of 130 points every three years, which is subject to a review in performance audit. She went on to compare the provision of CPD in China and Japan, where some advanced type of activities such as microlearning are offered, with South-East Asian countries where much of the provision of CME is highly dependent on funding and implementation by industry.

In response to a question on the provision of CME/CPD from entities outside their countries, all four panellists agreed that, whilst feasible, this tactic should be approached by first ensuring that contacts are established very early in the planning process with suitable partners with a presence in the destination country. In this way collaborations can be established to overcome potential cultural barriers that could hinder the process.

Oral poster presentation

Noleen Turner (PeerVoice) delivered the third and final oral poster presentation on a PeerVoice study that evaluated differences in preferred CME formats, perceived impact and barriers to participation between physicians in the EU5 countries (France, Germany, Italy, Spain and UK) and US (Figure 20). Her presentation described the study analytical plan, results and the key takeaways (Figure 21).

Figure 21. Oral poster presentation “Navigating the learning landscape of global clinicians: a comparative study between EU5 and US” by Noleen Turner (PeerVoice) outlining the key study takeaways.

Figure 20. Excerpt from the poster “Navigating the learning landscape of global clinicians: a comparative study between EU5 and US” by Pan Chen and Yvonne Oliver (PeerVoice).

Ask the accreditors

A plenary Q&A session was held with Reinhard Griebenow (EBAC), Jean-Philippe Natali (Agence Nationale du DPC), Natalie Paulus (EACCME) and Kate Regnier (ACCME) as representatives of regulatory bodies from around the world. The main topics of discussion were:

• The process of evaluation is an area where progress is slow or lacking; a best practice suggestion in the workshop and breakout sessions was that evaluation procedures should be tailored to the learning objectives of learners, as the use of a fairly standard evaluation form is still prevalent. Discussion ensued on how best to embed evaluation into the core of an educational activity and it was suggested that case examples where this does occur might be included in the resources for the 17th Annual European CME Forum

• The recurring topic of a lack of standardisation among accreditation models, particularly in Europe, where there is so much diversity among countries

• There was extended discussion regarding the recently published EACCME 3.0 criteria and Paulus stated that this publication was made in an effort to clarify the accrediting body’s stance on independence in European CME. She went on to provide details of a working group (including providers) established to work through the criteria in an effort to find common ground, with findings from this working group to be presented at a forthcoming European Union of Medical Specialists (UEMS) meeting

• An update was provided on the activities of the International Academy for CPD Accreditation (IACPDA) and their Standards for Substantive Equivalency among CPD/CME Accreditation Systems, with a goal of having accrediting bodies submit themselves for review so that confidence in the legitimacy of accreditors could be achieved through being part of a framework, albeit an informal one

• Short format education and microlearning have become acceptable for accreditation by EACCME and their flexibility in dealing with providers and in the mindset of decision makers, such as the move towards provider accreditation, was also emphasised by Paulus

Journal of CME 2023 update

Robin Stevenson, Editor-in-Chief of the Journal of CME, provided a short update on the journal and welcomed new editorial board members Steven Kawczak, Sophie Wilson and Dean Jenkins. He indicated from the publisher’s report that downloads during 2023 were in excess of 30,000, with the US, UK and China among the top three countries accessing the content. Stevenson introduced the forthcoming special collection theme which will address the use of AI in CME/CPD with Guest Editor Andrew Crim, Director of Education and Professional Development at the American College of Osteopathic Obstetricians and Gynaecologists. Some additional themes for possible future publication were introduced, including other article formats for the journal, performance and quality improvement, interprofessional CME, and education in support of public health priorities. Finally, he encouraged the poster authors to use their posters as the basis for articles to submit to the journal.

What the future holds

A final panel discussion led by Eugene Pozniak convened to consider “What the future holds”. Monica Ghidinelli (AO Foundation), Kate Regnier (ACCME) and David Vodušek (European Academy of Neurology) discussed the growing influence of AI and how its power might be harnessed to speed up content development for more targeted education. They also surmised that AI could help learners in their daily practice and to enhance their digital skills. Regnier saw possibilities for its use in her organisation’s accreditation procedures, such as helping providers to format an outline for a self-study report with the caveat that any such use must ensure the content is valid, accurate and reliable.

The panel also commented on new attitudes and approaches to changes in the format of CME/CPD, including the increased use of microlearning techniques and the use of assessments to drive education, such as the Board Certification process in US labelled as “self-directed learning”.

Some limiting factors, such as the cost of implementing many new technologies, were also outlined and a warning was offered by one attendee to avoid some negative impacts of AI that might lead to automation-induced complacency. The discussion drew to a close with some final thoughts on the need to focus education on areas such as strategies, judgement, teamwork, leadership and skills and not just knowledge transfer. The entry of large technology-driven companies into the healthcare arena, including education, was also raised as a possible area of concern. All stakeholders present were urged to maintain a “seat at the table” to push the agenda forward towards inclusivity in education in order to help the CME/CPD community support the goal of consistency in the delivery of high-quality care.

The CME unsession

In the now familiar role of master of ceremonies, Lawrence Sherman (AXDEV Group) sought to provide answers to unanswered questions from the meeting in an unstructured format. Topics that arose were:

• EACCME 3.0 and the perception that there is an openness and a willingness for dialogue

• Impact and outcomes still need to be covered

• Innovation is not always about the bright shiny thing

• The need to be in dialogue with undergraduate and graduate level medical education to champion the role of CME in the continuum

• The need to do “more with Moore” and update references to relevant publications [5]

• More representation from other regions of the world is needed at the Forum

• Be realistic in terms of the impact of education; relevance is not 100% for 100% of learners 100% of the time

• Unlearning has to occur before relearning and change can take place

Key takeaways from 16ECF

European CME Forum is dedicated to bringing together all stakeholder groups with an interest in European and global CME and CPD, promoting multi-channel discussion in an independent and neutral environment. The key takeaways from the 16th Annual European CME Forum included:

• The recurring theme of learner engagement was echoed in the interactivity of the plenary sessions as well as the smaller group workshops and breakouts

• Suggestions for a move towards measuring “impact” as well as higher level outcomes measurement seemed to resonate with most participants

• Accreditation should not just be about bureaucracy

• Attempts are being made to reduce barriers to success, such as the General Data Protection Regulation (GDPR) restrictions in Europe, as well as easier access to patient information and electronic patient records in US

• Collaboration and dialogue among stakeholders is progressing and the Forum helps to aid this process

• Providers and learners are global and many are keen to move away from time as the main unit of learning

Further information on the annual meetings is available at the European CME Forum website (cmeforum.org). This website has a wealth of information about their past activities. European CME Forum offers the largest resource about CME in Europe, anywhere whether online or in print, covering their 16 years of activities, with reports and presentations from over 272 hours of live presentations and workshops given by over 443 faculty. Participants of 16ECF have access to the latest meeting recordings at the 16ECF “Catch-up” page, while non-participants can register for a free account to access content from previous meetings.

The 17th Annual European CME Forum will be held in 6–8 November 2024 in Madrid, Spain. Visit the website (cmeforum.org/17ecf) for further information.

Disclosure Statement

No potential conflict of interest was reported by the author(s).