Cologne Consensus Conference Standards and Guidelines in Accredited CPD September 13-14, 2019, Cologne, Germany

ABSTRACT

On September 13–14, 2019 the eighth annual Cologne Consensus Conference was held in Cologne, Germany. The two-day educational event was organised by the International Academy of CPD Accreditors, a network of colleagues dedicated to promoting and enhancing continuing professional development (CPD) accreditation systems throughout the world. The conference was planned in cooperation with an impressive group of organisations representing leading European and North American institutions: the European Cardiology Section Foundation (ECSF), the Accreditation Council for CME (ACCME), the Royal College of Physicians and Surgeons of Canada, and Continuing Medical Education–European Accreditors (CME-EA). For the conference’s eighth iteration, Standards and Guidelines in Accredited CPD was chosen as the program topic and educational focus; a choice reflecting increasing international collaborations and an evolution towards consistency and standards across global accreditation systems. A specific list of domains and criteria (developed under a broader initiative already underway by the Academy) would serve as the core content around which the conference was planned. This conference report describes the initiative, the proposed standards to date, highlights of the Cologne Consensus Conference discussions and feedback, and the ongoing process of achieving consensus on the standards yet to be finalised.

KEYWORDS:

• CPD
• accreditation
• standards
• domains
• criteria
• international
• consensus

Introduction

The eighth annual Cologne Consensus Conference (CCC19) was held on September 13–14, 2019 in Cologne, Germany. At its origins, this two-day educational event was organised by the European Cardiology Section Foundation (ECSF) and the European Board for Accreditation in Cardiology (EBAC). However, in recent years, the organising bodies expanded to include the US-based Accreditation Council for CME (ACCME) and the Royal College of Physicians and Surgeons of Canada; with the 2018 conference held in Ottawa, Canada and the 2020 conference planned for Chicago, Illinois, USA.

This year, the Cologne Consensus Conference was back in its namesake city and was officially organised by the International Academy of CPD Accreditors (the Academy) and was planned in cooperation with the ECSF, ACCME, Royal College, and Continuing Medical Education–European Accreditors (CME-EA). Although a seemingly slight change, it is one reflecting the developing role of the Academy and its place within the international CME/CPD community. As described on its website¹

The Academy is a network of colleagues dedicated to promoting and enhancing continuing professional development (CPD) accreditation systems throughout the world. It is also devoted to assisting and supporting the development, implementation and evolution of CME/CPD and continuing medical education (CME) accreditation systems throughout the world. Established in 2013, the Academy serves as a platform that facilitates peer-to-peer support for leaders of CPD-CME accreditation systems and encourages networking, mentoring and interactions about common issues.

It was around this overarching mission that the Cologne Consensus Conference 2019 was organised. Each year, the CCC focuses on a specific content area and dives deep with presentations from faculty members, with substantial attendee interaction complementing the didactic content. Topics of previous conferences include² legal issues, decisions in accreditation, management of conflicts of interests, the role of the provider, assessments, interprofessional education, and the importance of data in CME/CPD. For the CCC’s eighth iteration, Standards and Guidelines in Accredited CPD was chosen as the educational focus; a choice reflecting increasing international collaborations and an evolution towards consistency and standards across global accreditation systems.

This important topic would be addressed in a manner departing strongly from that of previous CCCs. First, the standards in question were not simply conceptual, but rather a specific list of domains and criteria developed under a broader initiative already underway by the Academy. These standards would serve as the core content around which the conference was planned. Secondly, the format would not rely on the traditional structure of expert faculty giving informational presentations, with attendees asking questions and/or discussing the content thereafter. Rather, this year’s conference would comprise almost no formal presentations, but consist nearly entirely of working groups of up to six participants, discussing the various domains and criteria in detail, along with prescribed questions about their intent, interpretation, and acceptance.

As a result of the conference goals, topic, and highly interactive format, CCC19 was an important opportunity for the approximately 50 participants representing a wide range of stakeholders (including international accrediting bodies, regulators, health systems, providers, and medical associations/societies) to collaborate, debate, and provide direct and concrete feedback on the Academy’s proposed standards. Standards that are yet to be finalised, but will eventually serve as important guidelines for the evolving alignment of global CME/CPD accreditation.

Making the Case for Shared Standards

In 2016, a group of international leaders in accreditation, many current members of the Academy,³ co-authored the article Evolving Alignment in International Continuing Professional Development Accreditation⁴ in which they proposed a set of core principles that all CME/CPD accreditation systems must express as the basis for not only focusing educators on meeting the diverse and changing needs of their learners across the world, but also determining substantive equivalency between systems.

• Learning activities are developed to address the needs and professional practice gaps of members of the target audience

• The content is informed by evidence and bias is minimised

• Learning activities are designed efficiently to maximise educational impact

• Learning activities are planned and managed to ensure independence from external interests

• There is a rigorous evaluation of educational outcomes including how education has impacted knowledge, competence, performance, and health outcomes

• The accreditation standards and processes are consistently and fairly applied and continually enhanced

Broad agreement on and adoption of a set of CME/CPD accreditation standards and guidelines that are based on these shared principles and values would not only increase trust between systems, but would provide practical benefit to key stakeholders.

Accrediting Bodies

• Outline a clearer roadmap and objectives to guide, inspire, and teach the emerging and developing systems

• Create stronger collaborations globally, facilitating transference between those already established accreditation systems

• Provide services that respond to educators’ needs

• Create an environment to share best practices

• Leverage the power of education to improve healthcare

Regulators and Health Systems

• Support international regulators in establishing physician educational requirements

• Improve understanding of what is high quality and appropriate CME/CPD

• In recognition of the importance of life-long learning for providing safe and effective patient care, increased trust in accreditation systems to provide appropriate educational activities

• Serve as foundation for seamless and consistent reporting of activities/credits

CPD Educators/Providers

• Opportunity to enhance collaboration across systems to address evolving needs that cross borders

• Focus more on educational outcomes versus varied administrative accreditation requirements

• Advance the role of CME/CPD in the healthcare environment

Industry Supporters

• Clearer guidelines regarding role of industry and what is or is not allowed

• Streamlining across global operations

Learners

• Enhanced mobility, greater flexibility and choice in addressing learning or practice needs

• Potential reduction of regulatory burden as a result of process simplification and standardisation

• Increased focus on education that supports improved patient outcomes

In 2018, during its annual meeting in Ottawa, the Academy agreed to take this value message to the broader CME/CPD community, making the case for and seeking consensus on a set of accreditation standards, comprising specific domains and criteria. These standards would form the basis for substantial equivalency between international accreditation systems, and ultimately, potentially form the basis for credit reciprocity for CME/CPD among and between countries and accreditation systems.

The Draft Standards to Date

Given the international nature, the proposed standards would need to be based on a set of values and principles that offer flexibility in how they are expressed, are respectful of culture and context, and promote innovation and continuing improvement; thus, serving as a guide for both current and future CME/CPD. The Academy’s draft standards, consisting of six domains, as of September 2019 and subject of the Cologne Consensus Conference were:

Domain 1: Eligibility and Administrative Respon-sibilities of an AccreditorThis domain focuses on which organizations are eligible to develop and implement CME/CPD accreditation systems and the criteria that describe their expected administrative roles and responsibilities.

Domain 2: Independence and TransparencyThis domain focuses on polices and processes that ensure educational activities are designed and implemented independent of the influence of commercial interests defined as any entity producing, marketing, re-selling, or distributing healthcare goods or services consumed by, or used on, patients.

Domain 3: Needs AssessmentThis domain focuses on the processes and type of data source that will be used to identify the professional practice needs of individual physicians or health teams.

Domain 4: Content DevelopmentThis domain includes a focus on the process of identifying content that reflects the latest advances in scientific evidence and technological advances continue to enhance the quality and safety of care provided to patients.

Domain 5: Educational QualityThis domain includes a focus on the appropriateness of the design of educational formats in addressing the identified needs of the intended target audience.

Domain 6: Educational OutcomesThis domain includes a focus on assessment of the effectiveness and educational impact of accredited education on learning, competence, or performance of physicians or the health status of patients.

The above draft standards represented a version to date based on reiterations internal to the Academy and its members. Therefore, the next step in the process was to present them to the broader CME/CPD community for feedback on their intention, appropriateness, completeness, and design; with the goal being to achieve consensus on the final standards and detailed domains and criteria.

The Road to Consensus

Consensus methods are defined as a systematic means for measuring and developing consensus.⁵ The International Academy for CPD Accreditation elected to use a modified version of the Delphi technique⁶; an approach commonly used in medical education and comprising a number of stages.

1. Identify the Research Problem

In step one, the Academy identified and formalised the research question: “What are the domains and criteria that define the equivalency of CME/CPD accreditation systems?”.

• (2) Select Participants

Step two consisted of selecting participants among the international leaders in the development and implementation of CME/CPD accreditation systems; including and beyond those current members of the Academy.

• (3) Develop a Questionnaire of Statements

Step three was to develop the questionnaire of statements; namely, the survey comprising the proposed CME/CPD accreditation standards consisting of six domains and associated criteria.

• (4)Conduct Iterative Email-Based Questionnaire Rounds

• (5)Collect Individual and Group Feedback between Rounds and Summarise Findings

The Academy then set out to undertake, and is still in the process of, steps four and five in which they present the standards to members of the CME/CPD community for feedback; with the following key milestones leading up to the Cologne Consensus Conference.

Berlin Meeting Focus Group (16 May 2019)

Organised by Continuing Medical Education – European Accreditors (CME-EU)⁷ this day-long meeting brought together a range of regional, international, and speciality European accreditors.

• CME-EU Members

  • European Board of Accreditation in Haematology (EBAH)

  • European Board for Accreditation in Pneumology (EBAP)

  • European Board for Accreditation in Cardiology (EBAC)

• Austrian Medical Association

• European Board of Urology (EBU)

• German Medical Association

• Medical Council of Ireland

The Berlin meeting participants served as a focus group to review the survey questions and standards, including the six domains and associated criteria. Feedback was summarised and included in materials under consideration during the Cologne Consensus Conference 2019.

Survey to Global Leaders of CME/CPD Accreditation Systems (May 2019)

A detailed survey was administered to international leaders in the development and implementation of CME/CPD accreditation systems for the purpose of obtaining individual feedback, commentary, and consensus on the domains and criteria; with consensus being achieved when 80% of the responses indicated agree or strongly agree with the presented criteria. The survey was emailed to 85 global leaders of CME/CPD accreditation with 15 organisations responding.

1. Accreditation Council for Continuing Medical Education

2. American Academy of Family Physicians

3. Austrian Academy of Physicians

4. Bundesärztekammer (GMA)

5. Centre for Medical Education, NUS

6. College of Family Physicians of Canada

7. Ethiopian Medical Association

8. European Board for Accreditation in Cardiology (EBAC)

9. European Board for Accreditation in Haematology (EBAH)

10. European Board for Accreditation in Pneumology (EBAP)

11. Hong Kong Academy of Medicine

12. Medical and Dental Council of Nigeria

13. Medical Council of Ireland

14. National Agency for CME/CPD (France) – Agence Nationale du DPC

15. Royal College of Physicians and Surgeons of Canada

Survey results achieved 80% consensus for the majority of the domains with an overwhelming degree of responses ranked as agree or strongly agree. Most of the disagreement related to domain 2: Independence and Transparency and domain 3: Needs Assessment. The disagreement on needs assessments is also evident in responses pertaining to domain 5: Educational Quality with questions around using needs assessments. Along with the Berlin meeting, the survey served as a preparatory phase prior to face to face discussions that would take place during the Cologne Consensus Conference.

Cologne Consensus Conference (September 13-14, 2019)

Draft standards for each of the six domains, along with feedback and recommendations from the Berlin meeting and survey, formed the CCC19 core content. As outlined in the event’s program,⁸ each of the standards would be subject to the following five steps.

(1) Presentation

Brief presentation of the domain and criteria, along with feedback from the Academy survey and Berlin meeting.

(2) Discussion

Facilitated small-group discussions, six participants maximum per table. Worksheets with key points or questions for consideration and discussion; including the domain and related criteria, previous feedback received, proposal of additional criteria or suggestions for the larger group to discuss and potentially vote on.

(3) Reporting

Reconvening and reporting back to and discussion with the plenary group.

(4) Voting

Of the feedback reported from the small-group discussions, key points and recommendations were put forth for a group vote.

(5)Summary

Discussion of voting results and final thoughts on domain, criteria, and next steps.

Domain 1: Eligibility, Roles, and Responsibilities of Accreditors

Presentation

The first domain of the conference focused on which organisations are eligible to develop and implement CME/CPD accreditation systems and the criteria that describe their expected administrative roles and responsibilities.

• Accreditor is a legal entity, and not a commercial interest, and is a non-profit organisation or fully controlled by a non-profit organisation(s).

• All decision-makers involved in corporate governance including accreditation need to be independent. Statutes, as well as rules and standard operating procedures of the accreditor, have to ensure that accreditation occurs independently of any third-party influence, in particular of:

  • Professional political organisations (except organisations based on professional law)

  • Scientific organisations

  • Providers

  • Sponsors

  • Other commercial interests

• Accreditors should finance themselves by fees paid by those seeking accreditation. For public bodies (like Medical Chambers) funding may be part of the overall financial plan of the institution. Non-fee financial contributions from providers, sponsors, or other commercial interests have to be excluded by the statutes.

Within the domain of the roles and responsibilities of an accreditor, a number of criteria were presented as relevant to the recognition of the equivalency of CME/CPD accreditation systems. For each of the criteria below, survey respondents rated their level of agreement therewith.

Table

The eligible accreditor must have developed and implemented policies and procedures that require the accreditor to:	Agree and strongly agree
a) Define which activities or organisations are and are not eligible to apply for accreditation	100%
b) Define the process and standards by which eligible providers or activities are reviewed	100%
c) Communicate their requirements and process to applicants	100%
d) Implement a process to oversee the review of and base decision-making on, descriptions of the process and standards that must be demonstrated to achieve compliance (performance-in-practice)	92.8%
e) Have due-process safeguards including a complaint process, reconsideration/appeal process	100%
f) Require accredited providers or organisers of accredited education to improve areas of less than full compliance with requirements	85.7%
g) Require accreditors to retain records for a period of time compliant with applicable national regulations	85.7%

Additional Survey Results and Berlin Meeting Feedback

• Independence of accreditation bodies primarily manifests itself in relation to:

  • Governance issues

  • Finances

  • The accreditation procedure

• 100% of survey respondents agreed that commercial interests (defined as any entity producing, marketing, re-selling, or distributing healthcare goods or services consumed by, or used on, patients) are not eligible to serve as a CME/CPD accreditor.

• Further, 57% of survey respondents felt that there were other types of organisations that should also not be eligible to serve as a CME/CPD accreditor, including: any for-profit or proprietary organisations, subsidiary organisations funded by industry, CME/CPD providers, or other organisations responsible for aligning healthcare with their interests (e.g. for the purposes of cost savings, promotion, or other commercial goals).

• Ultimately, accreditors must be organisations able to ensure that accreditation occurs independently of any third-party influence; whether that be financial, governmental, political, professional, scientific, commercial, or other.

• In addition to the above survey responses to the specific criteria, 31% of survey respondents felt that additional criteria should address: incorporating a peer-review mechanism and expanded record keeping that focused on data management or reporting requirements.

• Accreditors should engage with providers and organisers on a regular basis to improve planning and delivery of CME/CPD as well as accreditation procedures.

• Require both accreditors and providers to retain records for a period of no less than six years or a time compliant with applicable (national) regulations.

• Accreditation decisions should be exclusively identifiable with the accreditor(s).

Discussion

Once presented with the above information, participant groups examined the proposed domain (defining who is eligible to be an accreditor) and criteria (outlining related roles and responsibilities). Eligibility concepts under consideration included accreditor legal status, financial models, and independence from third-party influence. Criteria explored the administrative framework and requirements necessary for an accreditor to fulfill its responsibilities. Participants were also asked whether there were any other types of organisations that should be considered as accreditors, if there are any changes or additions that would enhance the clarity of the criteria, and how these criteria might be measured or adherence demonstrated.

Reporting

Beyond the dynamic discussions around the above prescribed points and questions, this first session generated several concepts and concerns that were of a more wide-ranging nature, applying to all domains. The Academy was encouraged to consider more comprehensively the following.

Diversity

• There was a broad range of participants representing various countries, languages, healthcare systems, specialities, professions, etc. This diversity led to frequently differing views and interpretations of the standards. To date, the standards had been created and reviewed only by accreditors representing relatively established accreditation systems. Given the greater diversity of the CCC19 participants, feedback was rich, and consensus was more problematic.

Terminology and Wordsmithing

• Initially, Academy and CCC19 organisers did not expect significant questions on specific terminology used in the standards. However, vocabulary challenges were a constant brought forth by nearly all groups throughout the conference; illustrating the extent to which accreditation terminology might differ between countries, systems, or languages. For example, “accreditor” and “accrediting body”, seemingly interchangeable terms for most, but decidedly not for all. Or, what defines a “non-profit” in one country may be quite different in another.

• Understandably, terminology in an international context is a challenge and the Academy previously addressed this by creating a CPD accreditation glossary⁹ which defines a common set of terms to facilitate international collaborations. As a result of the CCC19 feedback, the Academy agreed it should review the glossary and ensure the terminology used in the standards aligns accordingly.

• Participants also felt that some of the wording was simply unclear, potentially confusing, or open to subjective interpretation. An example raised by several groups was criterion d): “Implement a process to oversee the review of and base decision-making on, descriptions of the process and standards that must be demonstrated to achieve compliance (performance-in-practice)”. The Academy responded that it would again review all the standards for increasing clarity.

Basic Standards Vs. Best Practice

A question often raised was just how strict or prescriptive the domains and associated criteria should be, given the goal of being as inclusive as possible, while still delineating a quality and compliance standard that all must adhere to. What needs to be included as a basic standard; a level achievable by not only the established or developed accrediting bodies, but also those in emerging or developing systems? Beyond this, lies the concept of best practice; a goal to achieve, a standard to aspire to, but not a minimum requirement? Finding this balance was an underlying concept throughout the conference discussions.

In addition to the general concepts above, group commentary specific to domain 1 addressed both the eligibility requirements for becoming an accreditor, as well as the administrative roles and responsibilities of such.

Domain: Eligibility

• Agreement that an accreditor must be a legal entity. However, there was concern regarding lack of a common definition of a non-profit organisation. More were in favour of ensuring that profit maximisation is not the goal of the accreditation activity and finances should be transparent.

• Consensus with the stated domain and criteria ensuring independence from any third-party influence over the accrediting body itself or the accreditation process at any level. Nevertheless, there was a question regarding the role of government, their potential influence, and whether they can be an accrediting body or not.

• Specifically, a commercial interest, or subsidiary thereof, is not eligible to be an accreditor.

• Many groups also expressed concern that depending on the country or cultural circumstances, bias can take multiple forms: financial, religious, legal, medical speciality interests, political, etc. Should this be taken into consideration?

• Agreement that the accrediting body should be financially independent; financing itself by fees paid by those seeking accreditation. For public bodies (like Medical Chambers) funding may be part of the overall financial plan of the institution. But, financial contributions from providers, sponsors, or other commercial interests are to be excluded.

Criteria: Administrative Roles and Responsibilities

There was general agreement with the proposed criteria, which outline how an accrediting body must define a framework for implementation of a fair, transparent, and concise accreditation procedure. Still, several additional points were presented for consideration.

• Ensure all criteria are applicable to all accreditors, regardless of whether in a provider or activity-based accreditation system.

• Accrediting bodies should be transparent in regard to their governance, activities and finances; proactively providing updates and data regarding the system (number of activities, funding levels, decisions, attendance, etc.).

• Avoidance of arbitrary decisions due to unfair, opaque, and ambiguous accreditation rules or application.

• Quality assurance system to ensure continuing professionalism, compliance with its own accreditation framework, and reproducibility of accreditation decisions.

• Inclusion of the medical profession in the accreditation process via peer review, governance, etc.

• Engagement (education, reporting, communication, etc.) with providers and organisers on a regular basis to improve accreditation compliance and planning and delivery of CME/CPD activities.

• Question of whether accreditors can also be providers of CME/CPD education.

• Some confusion as to what relates to the accreditor and what relates to the CME/CPD provider. Namely, criterion g) where participants are recommended to clarify wording that both accreditors and providers retain records for a period of time compliant with applicable national regulations.

Voting

Accrediting Body Is a Legal Entity with Transparent Finances

• Strongly disagree 0%

• Disagree 3%

• Neither disagree or agree 3%

• Agree 22%

• Strongly agree 72%

Eligibility to Be an Accrediting Body Cannot Be a Subsidiary of a Commercial Interest

• Strongly disagree 0%

• Disagree 3%

• Neither disagree or agree 6%

• Agree 13%

• Strongly agree 78%

Governmental Agencies Cannot Be Accrediting Bodies

• Strongly disagree 28%

• Disagree 19%

• Neither disagree or agree 28%

• Agree 16%

• Strongly agree 9%

The Accrediting Body Has a Responsibility to Report to Their Community on a Regular Basis

• Strongly disagree 0%

• Disagree 3%

• Neither disagree or agree 3%

• Agree 22%

• Strongly agree 72%

Summary

This first session addressing who is eligible to be an accrediting body and what their administrative roles and responsibilities are brought forth several concepts that would be present throughout the conference; challenges posed by the diversity of the international audience, inconsistent or unclear wording in the proposed standards, and how to determine what should be a base standard that must be adhered to versus what is best practice and something to aspire to. The group expressed strong consensus on the role of industry as supporters, but absolutely not as accreditors. Nevertheless, questions remain as to the role of the government in accreditation; especially in regions with emerging or developing systems where governmental agencies are often driving forces in establishing or regulating accreditation systems. Ultimately, the Academy has much to consider as they refine this foundational domain which outlines which organisations may become accreditors and their important role in ensuring quality and independent education, thus ensuring the trust of the medical profession.

Domain 2: Independence and Transparency

Presentation

There are multiple competing interests within and external to the medical profession that may compromise decision-making in healthcare. Therefore, educational activities or interventions developed to inform and enhance the practice of medicine must be designed to ensure independence from commercial or competing interests.¹⁰ Under domain 2: Independence and Transparency participants were asked to review and refine the criteria that would define how a CME/CPD accreditation system can demonstrate the necessary independence from commercial interests or other sources of systematic bias and ensure that accredited education is:

• Responsive to the needs of physicians, patients and communities

• Based on best available scientific evidence and practice-based data

• Designed to achieve improvements in physician practice and patient outcomes

For each of the criteria below, survey respondents rated their level of agreement therewith.

Table

The accreditor must have developed and implemented policies and procedures that:	Agree and strongly agree
a) Ensure the content, as well as persons and organisations in control of developing the content for accredited education are selected independently, with no influence, control or involvement from commercial interests	100%
b) Ensure all individuals involved in the planning and development, presentation (verbal or through creation of a manuscript), or evaluation of the accredited education disclose all relevant conflicts interests	100%
c) Ensure that identified conflicts of interest are resolved by the provider/organiser of the accredited education	92.8%
d) Ensure that all conflicts of interest are disclosed to participants	92.8%
e) Any commercial support accepted for the accredited education, meets the conditions and methods of payment applicable to national legal requirements, in particular tax and anti-corruption law, respectively, as well as professional law	92.9%
f) Inform learners, prior to participating in the accredited education, of the presence (or absence) of all disclosed conflicts of interest from individuals involved in the planning, presentation or evaluation of the accredited education	85.7%
g) Exclude representatives of commercial interests from participating as speakers, moderators or discussants	85.7%
h) Ensures the provision of commercial support never constitutes a relationship between individual learners and the commercial supporter	92.8%
i) Disqualifies individuals who refuse to disclose relevant financial interests from participating in or serving as a planning committee member, presenter of content as a teacher or author or evaluating the accredited educational activity	85.7%
j) Informs learners of the funds or resources provided by a commercial interest	64.3%
k) Excludes any promotional or sales communications to participants relating to the content of the education prior to or during the education	92.8%
l) Prevents the payment of travel, lodging or other personal expenses to participants who are not serving as teachers or authors	71.4%

Table

Excludes members of commercial interests or third-party representatives from controlling or participating in decisions regarding:	Agree and strongly agree
m) Needs assessments	92.8%
n) The selection of individuals or organisations invited to develop, select or present content, or attend	92.9%
o) The selection of educational methods	92.8%
p) Decisions regarding the evaluation of the impact of education on practice	92.4%

Additional Survey Results and Berlin Meeting Feedback

• Non-accreditable education (satellite symposia, social events, meals, etc.) at accredited CME activities cannot compete with or take precedence over the educational events.

• Include something about the provider being responsible for identifying and resolving conflicts.

• Commercial interests can contribute to needs assessments, but can never decide what needs the activity responds to.

• Some accreditors have more detailed mandatory requirements in place regarding communication to participants prior, during, or after the educational activity.

• Teachers should disclose all conflicts to the provider and the provider should be able to choose between disclosing all conflicts or only relevant conflicts to learners.

Discussion

The above Academy survey results reflect a lack of consensus on two criteria required of an accreditor to demonstrate independence and transparency; namely, that the accreditor would:

• Inform learners of the funds or resources provided by a commercial interest, criterion j).

• Prevent the payment of travel, lodging or other personal expenses to participants who are not serving as teachers or authors, criterion l).

Participants were asked to discuss the above and indicate reasons for including or excluding these disputed criteria. Further, groups reviewed the complete list of independence and transparency criteria, discussed their value, as well as any proposed changes or additions.

Reporting

Domain 2: Independence and Transparency generated substantial feedback and heated debate around the criteria outlining requirements to ensure that CME/CPD activities are balanced, independent, and free from commercial bias. Key suggestions and concerns included:

• Criterion c) requiring resolution of conflicts of interest was supported in concept. However, participants felt that doing so in practice is difficult and resource intensive with many stating that they do not actually know how to do this effectively or efficiently. Thus, it was suggested that this may be an aspirational goal, versus a base standard and expectation within all accreditation systems. At a minimum, education and support would be needed for providers/organisers to implement this criterion.

• Agreement that criteria d) and f) have essentially the same purpose of ensuring that all conflicts of interest are disclosed to learners; it was thus recommended to combine into one. Attendees also agreed that some indication of when disclosure is to take place would be helpful.

• Criterion e) was recommended to be removed entirely as it is too detailed and extends into local laws and regulations; beyond the practical scope of accreditation standards.

• Criterion g) flatly excludes representatives of commercial interests from participating as speakers, moderators, or discussants. All agreed that this is too generic and should outline circumstances when an employee may be able to participate.

• Remove criterion i) which requires that anyone refusing to disclose conflicts of interest be eliminated from participation. Participants agreed this is redundant because criterion b) requires all individuals to disclose.

• One of only two criteria lacking consensus going into the conference, criterion j) necessitates that learners be informed of the funds or resources provided by a commercial interest. Generating much discussion, all agreed this needed to be more detailed with guidance on what must be disclosed (company, type of support, amounts, etc.) and when.

• The second criterion lacking consensus was l) which prevents the payment of travel, lodging, or other personal expenses to participants who are not serving as teachers or authors. Easily the most debated criteria, concepts presented for ongoing discussion included:

  • General agreement that commercial interests should not make payments directly to participants.

  • Providers/organisers should be allowed to use commercial support, or other funds, to pay for participant travel, lodging, or other related expenses. However, they must have policies and procedures outlining how they do so in an appropriate, independent, and transparent manner.

  • This is especially relevant in a global context with participants from low/middle-income countries. If banning altogether, this would eliminate many learners, often those with the greatest need, from access to quality education.

  • Consider removing altogether as inclusion is too limiting and may exclude many global organisations.

• Criterion m) excludes members of commercial interests or third-party representatives from controlling or participating in decisions regarding needs assessments. There was a question around compliance with this criterion if responding to an industry-issued request for proposal for CME/CPD activities addressing a specific need identified by a commercial interest.

Voting

Prevents commercial interests from directly providing funding for travel, lodging or other personal expenses to participants who are not serving as teachers or authors.

• Strongly disagree 3%

• Disagree 20%

• Neither disagree or agree 10%

• Agree 30%

• Strongly agree 37%

Informs learners of the amount of the funds or resources provided by a commercial interest or sponsor.

• Strongly disagree 13%

• Disagree 23%

• Neither disagree or agree 17%

• Agree 23%

• Strongly agree 23%

Excludes representatives of commercial interests from participating as speakers/moderators/discussants.

• Strongly disagree 10%

• Disagree 14%

• Neither disagree or agree 10%

• Agree 21%

• Strongly agree 45%

Summary

Participants agreed on the majority of criteria, reflecting the understanding of how important independence and transparency are in CME/CPD. However, as reflected in the discussions and voting results, several key points remained unresolved: how to allow payment for travel/lodging/expenses for some participants (especially important to low or middle-income countries), should commercial support disclosure include actual amounts provided, and are there any circumstances under which a representative of a commercial interest may participate in CME/CPD activities? It was also noted that criteria a)-k) should be reviewed to ensure alignment with concepts included in the Academy’s 2018 Consensus Statement for Independence and Funding of Continuing Medical Education (CME)/Continuing Professional Development (CPD).¹¹

Domain 3: Needs Assessment

Presentation

Traditionally, needs assessment is an essential element to the planning, implementation, and evaluation of accredited education and a requirement for determining the equivalency between CME/CPD accreditation systems. However, the accreditor must support educational planning that is nimble and flexible, allowing for immediate needs to be identified and addressed. During this session on domain 3: Needs Assessment participants examined the criteria requiring that the accreditor has established standards and a process to ensure accredited education is:

• A response to an analysis of the needs of physicians, patients, and communities

• Planned to address needs identified from a variety of data sources including the expressed (perceived) needs of physicians and the health status of patients, populations and measures of knowledge, competence and performance of physicians

• Designed to address needs across a range of competencies relevant to the professional practice of physicians

For each of the criteria below, survey respondents rated their level of agreement therewith.

Table

The accreditor must have developed and implemented policies and procedures that:	agree and strongly agree
a) Describe that, ultimately the responsibility of the profession cannot be influenced or delegated to commercial interests (see domain 2: Independence and Transparency)	85.7%
b) Document the findings and conclusions reached	85.7%
c) Use the identified needs to inform the educational design of the activity	92.9%
d) Share the identified needs with those responsible for content development (see domain 4: Content Development)	85.7%
e) Use the identified needs to assess or evaluate the impact or outcomes of the accredited education (see domain 6: Educational Outcomes)	85.7%

Table

Based on an analysis of at least one of the following data sources including:	agree and strongly agree
f) The expressed needs of participants	92.9%
g) The health status of individual patients, populations or communities	92.8%
h) The gaps in the knowledge base of the profession	92.9%
i) Variation in the clinical competence of physicians	92.8%
j) Variation in the performance of physicians	92.8%

Additional Survey Results and Berlin Meeting Feedback

• These criteria are important for planning a CME/CPD activity, but an accreditor should not be required to measure each of these criteria.

• Within provider accreditation systems, needs assessments should be based on multiple data sources.

• The expectation for measuring gaps in knowledge, competence or performance will be a significant challenge in some countries/regions. Therefore, the criteria for needs assessments should not be so strict.

Discussion

Although the Academy survey resulted in consensus on all of the above criteria, groups were asked to review the complete list, along with the provided comments, discuss the value they bring, as well as propose any changes or additions.

Reporting

Generally speaking, and reflecting traditional accreditation practices, participants supported inclusion of a needs assessment requirement as a basic accreditation standard. However, in opposition to the survey results, there was not consensus on the specific criteria, and several points were presented for the Academy’s consideration.

• Criterion a) outlines that the needs assessment is ultimately the responsibility of the profession and cannot be influenced or delegated to commercial interests. Attendees felt there needed to be greater clarity regarding who the “profession” actually is; what exactly is the intention for including this? Additionally, reference to independence from commercial interests could be removed altogether as this is redundant to the criteria under domain 2: Independence and Transparency.

• Participants also questioned the role of objectives in practically satisfying criteria c)–e) which outline how the educational needs assessment is to be used to inform the design and assessment of the educational activity. Objectives are not currently included in the Academy’s proposed standards; reflecting that although common CME/CPD practice, objectives are not formal requirements imposed by all accrediting bodies. Despite this, the majority of participants agreed that objectives are an effective way of outlining what needs the activity is designed to address and what learners can expect by participating.

• The majority of feedback felt that performing a needs assessment can be difficult, resource intensive, and intimidating for many providers. Therefore, at a minimum, how robust does the process need to be to satisfy the criteria? How many sources are sufficient?

• One area of substantial discussion was whether the expressed needs of participants is sufficient for satisfying the basic standard. Also, must the expressed needs be validated by data? On one hand, it is understandable that learners may not fully recognise and express the actual or full scope of the educational needs (do not know what we do not know). But, on the other hand, some topic areas like communication skills, leadership, problem solving, etc. are more subjective and may not have available data supporting a formal needs assessment.

• Whatever the final criteria are in the end, given the difficulty, all agreed that it is important to offer providers with education and support on how to perform a quality needs assessment.

Voting

The expressed needs of physicians cannot be the only source of needs assessment.

• Strongly disagree 17%

• Disagree 37%

• Neither disagree or agree 7%

• Agree 17%

• Strongly agree 23%

Summary

As reflected in the discussions and voting results, although traditionally accepted, there remains concern regarding the challenges of performing a quality needs assessment. This is often a daunting and resource intensive exercise for any provider, regardless of the maturity of the accreditation system; will inclusion as a basic standard alienate providers in emerging or developing systems? Going even further, a provocative viewpoint was put forth questioning the fundamental basis of the needs assessment itself. Is this an administrative exercise that is becoming somewhat out-of-date? Learning has become more fluid in time, more flexible in regard to educational formats and media. Also, topics and content are going beyond clinical information or knowledge transfer to embrace broader topics like practice management, teaching, problem solving, or leadership. Is a formal needs assessment hindering or guiding quality education under these evolving circumstances?

Domain 4: Content Development and Domain 5: Educational Quality

Presentation

Domains 4 and 5 were presented simultaneously as both address the actual CME/CPD content: how it is developed and how it is delivered. This session focused first on the criteria for content development that will apprise participants of the evidence that should inform practice. The session focused on factors from the research literature that contribute to important educational outcomes, thus outlining the criteria that would define educational quality.

Domain 4: Content Development

Content development is an essential element to the provision of accredited education that requires the accreditor to establish standards and a process to ensure that the content of accredited education:

• Addresses gaps across a range of competencies relevant to the practice of medicine

• Provides recommendations that are based on scientifically valid evidence

• Ensures content that is not supported by evidence or clinical practice guidelines is balanced and informs participants about potential benefits and risks

• Is not influenced by commercial interests

For each of the criteria below, survey respondents rated their level of agreement therewith.

Table

The accreditor must have developed and implemented policies and procedures that ensures the content is:	Agree and strongly agree
a) Relevant and responsive to the needs or practice gaps of the established target audience (see domain 3: Needs Assessment)	100%
b) Based on evidence that is accepted within the profession of medicine as scientifically valid	100%
c) Adequate to justify recommendations related to the care of patients	100%
d) Balanced in the review of all relevant options (prevention, diagnosis, treatment, patient management)	100%
e) Provided by individuals whose conflicts of interest are appropriately managed and resolved (see domain 2: Independence)	85.7%
f) Comprehensive in scope, addressing the range of competencies relevant to the provision of safe, high quality healthcare	92.9%

Additional Survey Results and Berlin Meeting Feedback

• Content should be balanced in the review of all relevant data by use of, for example, Cochrane Library systematic reviews, and meta-analyses.

• Scientific research referred to, reported, or used in CME/CPD in support or justification of a patient care recommendation must conform to the generally accepted standards of experimental design, data collection and analysis, as laid down in WHO, and WMA declarations.

• Inclusion of a broader definition of evidence that is both anchored in the scientific research literature and informed by the “tacit knowledge” that is learned through experience in practice (not described in clinical practice guidelines or systematic reviews, etc.).

• Consider the term evidence-informed to reflect this view of evidence and to recognise that many aspects of clinical practice have limited scientific evidence.

Domain 5: Educational Quality

Educational design is an essential element to the provision of effective education. For this domain the accreditor has established standards and a process to ensure that accredited education is:

• Appropriately designed to address the identified professional practice needs of the targeted audience (see domain 3: Needs Assessment)

• Utilising educational formats that facilitate or enable learning

• Promoting deliberate practice within the educational setting

• Facilitating the translation of new knowledge, skills and competencies into practice

For each of the criteria below, survey respondents rated their level of agreement therewith.

Table

The accreditor must have developed and implemented policies and procedures that:	agree and strongly agree
a) Utilise an assessment of needs to promote or predispose the desire of physicians to learn (see domain 3: Needs Assessment)	78.6%
b) Use educational formats that are appropriate to the objectives and desired results of the activity	100%
c) Incorporate multiple educational methods, alone or in sequence, to provide learners with opportunities, where appropriate, to reflect on, apply or practice what they have learned over time	92.8%

Additional Survey Results and Berlin Meeting Feedback

• Rather than focusing on needs assessment for motivating physicians to attend accredited learning activities we would recommend that the needs assessments are translated to learning objectives that will inform what participants will (or at least should) be able to know or do following the activity, as the primary means of motivation based on alignment of the goals of the educational activity to their perceived needs.

Discussion

Consensus was achieved from survey results for all criteria under domain 4. However, under domain 5: Educational Quality there was a lack of consensus on criterion a) which states that an assessment of needs should be used to promote or predispose the desire of physicians to learn. Groups were asked to discuss if they agreed with this criterion, or whether objectives should be utilised instead for this purpose (as suggested in survey feedback). Groups also reviewed the complete list of criteria and commentary; discussing their value, as well as any proposed changes or additions.

Reporting

From the start, all groups recommended combining domains 4 and 5 into one as both address the delivery of the educational initiative itself. Additionally, as for many of the previous domains, it was suggested that the wording be reviewed to enhance overall clarity. Participants also provided feedback for consideration on the specific criteria, as follows.

• Domain 4 criterion a) was deemed unnecessary and redundant to domain 3: Needs Assessment. Participants thus recommended it be removed.

• It was also suggested to consider removing domain 4 criterion c) which requires that content be adequate to justify recommendations related to the care of patients. Many believed that this is conceptually encompassed within criterion b) which necessitates that content is based on evidence that is accepted within the profession as scientifically valid.

• Most felt that domain 4 criterion e) “Provided by individuals whose conflicts of interest are appropriately managed and resolved” should be removed as redundant to domain 2: Independence and Transparency.

• Questions were put forth whether a criterion should be included which addresses the requirement that content is delivered by qualified educators; to what extent should the standards be prescriptive in regard to faculty?

• There was substantial discord around domain 5 criterion a) and whether promoting/predisposing physicians to learn is a basic standard or best practice. Further, if predisposing physicians to learn remains a basic standard, there was question around the best way of doing so; whether using needs assessment data, objectives, pre-reading, etc.

• Under domain 5, there was debate as to whether there should be an explicit criterion that sets a standard of innovation and interactivity in educational formats.

Voting

Do you agree with the revised recommendations for domain 4, criterion c)?

• Strongly disagree 11%

• Disagree 11%

• Neither disagree or agree 0%

• Agree 33%

• Strongly agree 44%

Do you agree to remove domain 4, criterion e): “conflicts of interest provided by individuals … .”?

• Strongly disagree 0%

• Disagree 0%

• Neither disagree or agree 0%

• Agree 56%

• Strongly agree 44%

Do you agree to eliminate domain 5, criterion a) “assessment of needs to promote the desire to learn … ”?

• Strongly disagree 11%

• Disagree 11%

• Neither disagree or agree 4%

• Agree 37%

• Strongly agree 37%

Summary

All agreed to combine domains 4 and 5 in future iterations of the standards. Also, achieving consensus was removal of domain 4 criterion e) which addresses conflicts of interest, as this is already covered under domain 2: Independence and Transparency. In contrast, revisions are still recommended in regard to wording addressing the adequacy of content to justify recommendations related to the care of patients. Additionally, it was unclear whether a criterion should be included regarding predisposing/promoting the physician’s desire to learn and if so, what the best mechanism for doing so is. Ultimately, there was no question about the importance of these domains for ensuring that CME/CPD activities distribute evidence-based, relevant content that is delivered using quality and appropriate educational formats. However, discussions consistently come back to the fundamental challenge of creating standards that are robust and detailed enough to delineate an achievable accreditation baseline, while not being so restrictive as to stunt innovation and creativity in regard to educational methodologies.

Domain 6: Educational Outcomes

Presentation

The assessment of the impact or outcomes of accredited education is an essential element to the recognition of equivalency between accreditation systems. Within this domain the accreditor has established standards and a process to ensure that accredited education:

• Uses one or more assessment methods appropriate to the educational goals or outcomes

• Measures the impact of learning during and following the completion of an educational activity

• Where applicable, assesses the impact of accredited learning on patients

For each of the criteria below, survey respondents rated their level of agreement therewith.

Table

The accreditor must have developed and implemented policies and procedures ensuring that assessment of accredited education:	agree and strongly agree
a) Is consistently evaluated covering the criteria outlined in domains 2-5	92.8%
b) Use assessment strategies (quantitative and qualitative) to assess the impact of education on the identified needs and established learning goals	92.9%
c) Measure improvements in knowledge, skills and competencies during and/or at the conclusion of the education	85.7%
e) Measure improvements in physician learner performance (where applicable)	92.9%
f) Measure changes in patient health status (where applicable)	71.4%

Note: criteria mis-lettered in original text. No criterion d).

Additional Survey Results and Berlin Meeting Feedback

• We would recommend that a minimal threshold for criterion c) should be the expectation that learners are required to reflect on and describe (self-report) what they learned, or any commitment to make a change in their practice rather than (in all situations) the activity measuring changes in knowledge.

• Performance improvement requires the educational activity to develop a relationship with the clinical environment to support change and demonstrate improvement over time.

• Criteria a) and b) might be redundant from the other criteria in this domain.

• Requirements to measure changes in knowledge, skills, or performance are criteria that are best adapted to provider accreditation models than activity-based accreditation as provider accreditation can hold organisations accountable to assess outcomes using multiple methods. Although we agree with the ultimate goal of improving patient experiences of care or outcomes of care, these would be difficult criteria upon which to base international recognition of equivalency.

Discussion

Survey results reflected a lack of consensus on criterion f) requiring that changes in patient health status (where applicable) be measured. Groups were asked to discuss if they agreed or not with including this criterion. Participants also reviewed the complete list of criteria and feedback; discussing their value, as well as any proposed changes or additions.

Reporting

Overall, participants agreed on the importance of assessing the impact or outcomes of accredited education; especially when discussing international standards for recognition of equivalency between accreditation systems. Nevertheless, concern was also expressed regarding the challenges of properly and efficiently measuring educational outcomes. Specifically, groups brought forth the following for consideration.

• Substantial discussion centred on criterion a) requiring that CME/CPD is consistently evaluated covering the criteria outlined in domains 2–5.

  • Some suggested to remove criterion a) altogether as they did not agree it was necessary to evaluate the basic standards under domains 2–5. If higher outcomes are achieved, as required by domain 6, then the other domains are simply a means to this end and evaluating them would be an unnecessary administrative step.

  • Others argued that the intent of criterion a) is to ensure an evaluation of the process of education itself and obtaining important learner feedback thereon is necessary to improve future educational initiatives.

  • Ultimately, the debate harkened back to the recurring question of whether a criterion should be a mandatory basic standard or a practice left up to the accreditor/provider to implement as deemed necessary.

  • Regardless of the Academy’s final decision to keep criterion a) or not, there was general agreement amongst conference attendees that the wording needs to be clarified as to what exactly should be measured; as currently indicating “criteria outlined in domains 2–5” is perceived as too broad.

• There was a suggestion to combine criteria c) measure improvements in knowledge/skills/competencies, e) measure improvements in performance, and f) measure changes in patient health status into subsections of criterion b) which requires assessing the impact of the education. Whether to combine or not, participants agreed that the various levels of assessment should be included in the general need to assess.

• Under criteria e) and f), it was proposed that instead of measuring changes in performance or patient health status “where applicable”, the wording should be changed to “optional”. This adjustment makes it clearer that it is not mandatory for compliance, but is included as a goal to be achieved based on the expected results of the CME/CPD activity and resources available to the CME/CPD provider.

Voting

The accredited provider will measure changes in patient health status (if possible).

• Strongly disagree 0%

• Disagree 22%

• Neither disagree or agree 19%

• Agree 41%

• Strongly agree 19%

Combine domain 6, criteria b), c), e), f) into one criterion.

• Strongly disagree 7%

• Disagree 11%

• Neither disagree or agree 4%

• Agree 44%

• Strongly agree 33%

It should be required that the learners evaluate the process of education, criterion a).

• Strongly disagree 27%

• Disagree 27%

• Neither disagree or agree 19%

• Agree 19%

• Strongly agree 8%

Summary

Conference participants echoed agreement on the utmost importance of educational outcomes in CME/CPD; measuring the impact of the educational initiatives is key and must be included in the basic standards. Nevertheless, there is an underlying concern about the challenges of properly and efficiently doing so, especially when measuring changes in patient health status. This was reflected in the voting results with conference participants being unable to come to consensus on the proposed changes. As a result, the Academy will need to consider further what they will outline as basic requirements for measuring the impact of CME/CPD and how they might also provide education and support to accrediting bodies and providers to promote proper implementation.

Summary and Next Steps

The 2019 Cologne Consensus Conference set a lofty goal of achieving consensus on the proposed global standards and guidelines in accredited CME/CPD. For those who participated, it was a thought-provoking experience where the detailed examination of the standards and the ensuing discussions and debates underlined just how complex and challenging it is to achieve true agreement. The conference’s highly interactive format generated a diverse mix of viewpoints, personalities, and practices; enriching both the attendee experience and the resulting feedback. In fact, many participants said that they had nuanced or changed some of their previous thoughts on the standards, as well as on their own CME/CPD practices. However, despite this, consensus on many criteria remained elusive.

As outlined in this report, there was a wealth of attendee comments to be considered; reflecting the wide scope of how CME/CPD accreditation is organised and executed in different parts of the world. The conference underlined some of the key challenges and critical considerations influencing the creation and adoption of a set of international accreditation standards.

• Diversity of participants’ experiences and interests based on their number of years in CME/CPD as well as their role therein (accreditor, provider, PCO, government official, etc.)

• Discrepancies related to accreditation system maturity: emerging, developing, or established

• Global contexts with different financial and economic resources to take into consideration; strongly influencing the role industry support plays in a system

• Considerable variation in legal structures of accrediting bodies, as well as the legal contexts they are functioning in

• Strong influence of regional, political, and cultural concepts and experiences

• Linguistic challenges in regard to terminology and subjective interpretations

As the Academy continues down the road to consensus, next steps following the conference are to review the wealth of feedback generated from the Berlin meeting, the survey, as well as CCC19 and continue the in-depth consideration of key fundamental questions and concepts.

• How do you create standards that are as inclusive as possible, while still establishing a minimum level of compliance that clearly delineates unacceptable and/or necessary practices for independent, transparent, and quality CME/CPD?

• Which criteria are essential to the basic standards versus those that are either aspirational best practices or may be better suited for inclusion in later iterations?

• Which points can be combined, simplified, or removed altogether?

• How to ensure terms and definitions used are as concise as possible and broadly understood by a global audience?

• General review of wording and terminology to be consistently employed throughout the various domains and criteria?

• How specific do the criteria need to be? How much information and guidance are enough, but not too much; thus leaving ample space for local adaptation, creativity, and innovation?

• What practical support materials and education are needed to promote broad adoption and implementation?

• Although the standards will have implications for the broader CME/CPD community (providers, societies, industry supporters, etc.), ultimately final decisions on the standards are the responsibility of the accreditors and their regulators (if applicable). As such, how to find balance between viewpoints of all stakeholders?

The next and ninth iteration of the Cologne Consensus Conference will be organised by the International Academy for CPD Accreditation and hosted by the ACCME in Chicago, Illinois, USA on September 10–11, 2020. While the conference’s topic of focus is still to be determined, the event will serve as another important step on the road to final consensus and international roll-out of these important standards and guidelines in international CPD that seek to inspire, guide, and set the bar for accrediting bodies across the globe.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

¹ International Academy for CPD Accreditation; https://academy4cpd-accreditation.org/.

² Cologne Consensus Conference: Future/Past Conferences, Presentations, Reports; http://e-cs-f.org/en/activities-and-projects/.

³ International Academy of CPD Accreditors Current Member Listing; https://academy4cpd-accreditation.org/about-us/current-members/.

⁴ Evolving Alignment in International Continuing Professional Development Accreditation; McMahon, Graham T. MD, MMSc; Aboulsoud, Samar MD, MSc, MEd; Gordon, Jennifer MEd; McKenna, Mindi PhD, MBA; Meuser, James MD; Staz, Mark MA; Campbell, Craig M. MD, FRCPC; Journal of Continuing Education in the Health Professions: Summer 2016 – Volume 36 – Issue – p S22–S26.

⁵ Humprhey-Murto et al. “Using consensus group methods such as Delphi and Nominal Group in medical education research. Medical Teacher 2016.

⁶ De Loe RC et al, Advancing the state of policy delphi practice.: A systematic review evaluating methodological evolution, innovation and opportunities.

⁷ Continuing Medical Education – European Accreditors; http://www.cme-ea.eu/.

⁸ Cologne Consensus Conference 2019 Program and Faculty; http://e-cs-f.org/en/activities-and-projects.

⁹ The International Academy for CPD Accreditation CPD Accreditation Glossary; https://academy4cpd-accreditation.org/cpd-accreditation-glossary/.

¹⁰ Commercial Interest is defined as any entity producing, marketing, re-selling, or distributing healthcare goods or services consumed by, or used on, patients. Commercial Support defined as a monetary or in-kind contribution given by a commercial interest to a CME/CPD provider that is used to pay all or part of the costs of a CME/CPD activity.

¹¹ International Academy of CPD Accreditors Resources; https://academy4cpd-accreditation.org/resources/.