A content analysis of HPV vaccine online continuing medical education purpose statements and learning objectives

ABSTRACT

Numerous online HPV vaccine education interventions for clinicians have been created to improve HPV vaccinations rates. The aims for this study were to (1) assess the content of the purpose statements and learning objectives of online HPV vaccine continuing medical education (CME) activities developed for clinicians and (2) identify themes and gaps in the purpose statements and learning objectives. A content analysis was conducted of the purpose statements and learning objectives for each HPV vaccine online CME activity. Open coding identified the following purpose statements topics: 1) delivering recommendations, 2) HPV epidemiology, 3) HPV vaccine, 4) guidelines, and 5) medical news. The following topics for learning objectives were identified: 1) delivering recommendations, 2) strategies, 3) HPV epidemiology, 4) HPV vaccine, 5) guidelines, 6) prevention services, 7) HPV vaccination advocacy, and 8) disparities. Phrases about guidelines for vaccine administration and vaccine recommendation guidelines were the most common for purpose statements and learning objectives, respectively. One learning objective focused on behavior/skill change, which is concerning considering clinicians do not feel well prepared to provide strong vaccine recommendations. Clear and intentional purpose statements and learning objectives must be used to guide the development of effective CME activities.

KEYWORDS:

• Papillomavirus vaccines
• continuing medical education
• web-based education
• content analysis

1. Introduction

The human papillomavirus (HPV) is the most common sexually transmitted infection affecting 79 million Americans.^1,2 Almost every sexually active person will be infected with HPV at least once in their lifetime.^1,3 Research shows HPV is responsible for nearly all cervical cancers, 69% of vulvar cancers, 75% of vaginal cancers, 91% of anal cancers, and 63% of penile cancers.⁴ Vaccines are available to prevent HPV associated cancers.^2,3,5 Vaccines are recommended for all boys and girls between the ages of 11 and 12 years. Catch-up vaccines are recommended for boys and men through age 21 and for girls and women through age 26, if they did not complete the series when they were younger.^1–3,6 However, despite the high prevalence of HPV and vaccine availability, vaccination rates remain suboptimal.^7,8

Primary reasons for low HPV vaccination rates in the US include missed clinical opportunities for health-care providers to recommend the vaccine and a lack of strong and consistent recommendations by providers.^7,9 Concerns about vaccine safety, discomfort discussing a topic related to sexual behavior, preference for vaccinating older adolescents, lack of time and/or incentives to educate parents about the vaccine, and the providers’ limited knowledge of HPV and the vaccine contribute to suboptimal vaccination rates.^10–19 In addition, health-care providers do not recommend the HPV vaccine as strongly as other adolescent vaccines²⁰ and provide inconsistent, mixed messages about the HPV vaccine.²¹ Clinicians have reported they do not feel well prepared to provide strong HPV vaccine recommendations,^14,22,23 with as many as 75% of clinicians expressing they would benefit from continuing education about the HPV vaccine.²⁴

Numerous online HPV vaccine educational interventions for clinicians have been created to improve vaccination rates. Online educational interventions provide opportunities to quickly update and address health education topics at low cost²⁵ and are becoming a popular delivery method for health-care professionals to obtain continuing medical education (CME).²⁶ Current research suggests that online CME that use strategic health communication design principles can increase clinicians’ knowledge, perceptions, attitudes, and practice behaviors.^27–32

Evaluation of these online HPV vaccine CME activities is critical to the translation of health commutation research into guiding efforts to promote clinician recommendations for the HPV vaccine.^33,34 Evaluation research can successfully identify strengths and weaknesses of educational interventions from the users’ perspectives, determine if interventions are worth the time, resources, and expense for continued implementation, provide the evidence for designing optimally effective interventions,³⁵ provide insights about any negative unintended consequences of the interventions,^36,37 and ensure interventions address audience members’ unique needs, cultures, and expectations.³⁸ A recent evidenced-based evaluation highlights strengths and weaknesses of current online HPV vaccine CME activities,³⁹ and a recent content analysis of online HPV CME activities reveals a lack of content about how to change provider practice.⁴⁰ However, no research on the content of purpose statements and learning objectives has been reported examining online HPV vaccine CME activities.

For a learning activity to be considered competency-based education, the activity must focus on the learners’ performance (learning outcomes) defined by specific purpose statements and learning objectives.⁴¹ Purpose statements are a declarative sentence summarizing the topic and goal of the activity,⁴² while learning objectives are statements regarding the knowledge, attitudes, and skills participants are to acquire by the end of the activity.⁴³ Unlike a list of topics, learning objectives describe the actions the learner can do after successful completion of the activity.⁴⁴ Purpose statements and learning objectives drive online education modules by organizing content for learners, facilitating complex processing,⁴⁵ implying learning methods, enabling evaluation, and specifying the skills participants will gain from the educational activity.⁴³ Purpose statements and learning objectives should align and support each other to achieve specific educational goals⁴⁶ and allow for deliberate rather than generic applications to practice thereby encouraging an effective transfer of knowledge,⁴⁷ resulting in a significant learning experience.⁴⁸ Once purpose statements and learning objectives are written, they should be used as a framework to organize instructional activities⁴⁹ and assessments to ensure instructional time is used effectively.⁴⁸ Medical programs undergo extensive review for learning objectives, curriculum and assessment to ensure quality program design, and CME activities should be no different. Without data evaluating the purpose statements and learning objectives used in online HPV vaccination training and CME activities for clinician, there will be a lack of evidence for ensuring quality course design and refinement of these interventions.³⁹ Therefore, there is a need to examine the purpose statements and learning objectives of online HPV vaccine CME activities for clinicians.

The aims for this study were to: (1) assess the content of the purpose statements and learning objectives of online HPV vaccine CME activities developed for clinicians and (2) identify themes and gaps in the purpose statements and learning objectives.

2. Results

2.1 Purpose statements

For the purpose of this study, we reviewed phrases as the unit of analyses. While there were 21 interventions included in this study, the sample size for purpose statements included 22 phrases in the analysis. See Table 1 for online HPV vaccine CME activities’ characteristics and purpose statement messages.

Table 1. Online continuing medical education characteristics and purpose statement messages.

Online Continuing Medical Education Title	Source	Purpose Statement Messages
		Delivering Recommendations^a (n = 5)	HPV Epidemiology^b (n = 1)	HPV Vaccine^c (n = 2)	Guidelines^d (n = 10)	Medical News^e (n = 4)
You are the key to HPV cancer prevention	CDC	1	0	0	1	0
Adolescent immunizations: Strong Recommending the HPV Vaccine	AAP	2	0	0	0	0
HPV Documentary-Division of Continuing Medical Education	IIC	0	0	0	0	0
You are the key to HPV cancer prevention	AHEC	0	0	0	0	0
Immunization: You call the shots- module eight-HPV, 2016	CDC	0	0	0	0	0
Don’t wait vaccinate! The prevention of HPV cancers	TMA	1	0	2	1	0
Don’t wait vaccinate! The prevention of HPV cancers (Part 2)	TMA	1	0	0	2	0
Human Papillomavirus (HPV) Vaccine Safety	CME University	0	0	0	0	0
HPV Vaccines Updates and Perspective	Medscape	0	0	0	0	0
The story of HPV: yesterday, today and tomorrow	Medscape	0	0	0	2	0
Putting HPV vaccine knowledge into practice	Medscape	0	1	0	1	0
CDC updates guideline recommendations for HPV vaccination	Medscape	0	0	0	0	1
HPV vaccine safety and efficacy	Medscape	0	0	0	1	0
Overcoming gender and socioeconomic disparities in HPV vaccination	Medscape	0	0	0	1	0
ACIP releases pediatric vaccine schedule	Medscape	0	0	0	0	1
AAP provides guidance for parents who refuse vaccination	Medscape	0	0	0	0	1
ACIP releases adult vaccine recommendations	Medscape	0	0	0	0	1
HPV vaccines: updates and clinical perspective	Medscape	0	0	0	1	0
HPV vaccination is cancer prevention (2017 update)	Boston SoM	0	0	0	0	0
Increasing adolescent immunization coverage	CME University	0	0	0	0	0
Immunization: You call the shots- module eighteen-vaccine administration	CDC	0	0	0	0	0

Note. CDC = Centers for Disease Control and Prevention. AAP = American Academy of Pediatrics. IIC = Indiana Immunization Coalition. AHEC = Area Health Education Centers. TMA = Texas Medical Association. SoM = School of Medicine.

^a Delivering recommendations included phrases about: a) delivering recommendations for HPV vaccination to decrease missed clinical opportunities, b) delivering strong recommendations for HPV vaccination, c) delivering effective recommendations for HPV vaccination, d) communicating with and/or education and/or counseling patient about HPV vaccination, e) communicating with and/or education and/or counseling parents/caregivers/family members about HPV vaccination, and f) parents’ concerns or questions about HPV vaccination.

^b HPV epidemiology included phrases about epidemiology terms regarding HPV including incidence and prevalence rates, incidence and prevalence of HPV-related cancers, causative agent and risk, and HPV-related disease.

^c HPV vaccine included phrases about: a) HPV vaccine being used as cancer prevention, b) HPV vaccination rates, and c) HPV vaccination hesitancy, vaccination refusal and barriers to parental consent.

^d Guidelines included phrases about: a) the recommended age, group (adolescents), or sex for HPV vaccination, b) administering (physically giving the HPV vaccine) the HPV vaccine to adolescents, c) locating or identifying resources, avenues, or locations aligned with current immunization guidelines administering the HPV vaccine to adolescents, and d) HPV vaccine safety.

^e Medical news included phrases about providing medical news.

Delivering recommendations

Five (23%) CME online purpose statements centered on delivering recommendations for the vaccine. Purpose statements included messages about missed opportunities (n = 2; 9%), strong recommendations (n = 1; 4%), communicating and educating patients (n = 1; 4%) and communicating regarding parents’ concerns or questions (n = 1; 9%).

HPV epidemiology

One (9%) CME online purpose statement addressed HPV epidemiology.

HPV vaccine

Of the 22 purpose statement phrases, two (9%) included phrases about addressing HPV vaccination. Statements included HPV vaccination rates (n = 1; 4%), and HPV vaccine hesitancy (n = 1; 4%).

Guidelines

Almost half (n = 10) of the purpose statement phrases included a message about guidelines. Specifically, there were purpose statements that included messages about guideline recommendations (n = 1; 4%), and guidelines for vaccine administration (n = 4; 18%), science supporting the guidelines (n = 3; 14%), and guidelines safety (n = 2; 9%).

Medical news

There were four (18%) phrases that included providing medical news to clinicians and health-care providers in the purpose statement.

Four interventions (18%) had the same purpose statement (i.e., “The goal of this activity is to provide perspective on the science behind recent changes to the HPV vaccine dosing schedule.”), as well as three other interventions (14%) had the same purpose statement (i.e., “The goal of this activity is to provide medical news to primary care clinicians and other healthcare professionals in order to enhance patient care.”). Seven interventions (32%) did not include a purpose statement.

2.2 Learning objectives

The sample size for learning objectives included 68 phrases in the analysis. See Table 2 for learning objective messages.

Table 2. Online continuing medical education learning objective messages.

Online Continuing Medical Education Title	Source	Learning Objective Messages
		Delivering Recomm.^a (n = 10)	Strategies^b (n = 1)	HPV Epidemiology^c (n = 6)	HPV Vaccine^d (n = 11)	Guidelines^e (n = 32)	Prevention Services^f (n = 3)	HPV Vaccination Advocacy^g (n = 1)	Disparities^h (n = 4)
You are the key to HPV cancer prevention	CDC	3	0	0	1	1	1	0	0
Adolescent immunizations: Strong Recommending the HPV Vaccine	AAP	3	0	0	0	0	0	0	0
HPV Documentary-Division of Continuing Medical Education	IIC	0	0	0	0	0	0	0	0
You are the key to HPV cancer prevention	AHEC	1	0	0	1	4	0	0	0
Immunization: You call the shots-module eight-HPV, 2016	CDC	0	0	2	0	2	1	0	0
Don’t wait vaccinate! The prevention of HPV cancers	TMA	0	0	2	1	3	0	0	0
Don’t wait vaccinate! The prevention of HPV cancers (Part 2)	TMA	1	0	0	1	2	0	0	0
Human Papillomavirus (HPV) Vaccine Safety	CME University	1	0	0	0	3	0	0	0
HPV Vaccines Updates and Perspective	Medscape	0	0	0	0	1	0	0	0
The story of HPV: yesterday, today and tomorrow	Medscape	0	0	1	0	0	0	1	0
Putting HPV vaccine knowledge into practice	Medscape	0	0	0	0	0	0	0	2
CDC updates guideline recommendations for HPV vaccination	Medscape	0	0	0	0	5	0	0	0
HPV vaccine safety and efficacy	Medscape	0	0	0	0	1	0	0	0
Overcoming gender and socioeconomic disparities in HPV vaccination	Medscape	0	0	0	0	0	0	0	2
ACIP releases pediatric vaccine schedule	Medscape	0	0	0	2	0	0	0	0
AAP provides guidance for parents who refuse vaccination	Medscape	0	0	0	4	0	0	0	0
ACIP releases adult vaccine recommendations	Medscape	0	0	0	1	1	0	0	0
HPV vaccines: updates and clinical perspective	Medscape	0	0	0	0	1	0	0	0
HPV vaccination is cancer prevention (2017 update)	Boston SoM	0	1	1	0	1	0	0	0
Increasing adolescent immunization coverage	CME University	1	0	0	0	2	0	0	0
Immunization: You call the shots-module eighteen- vaccine administration	CDC	0	0	0	0	5	1	0	0

Note. CDC = Centers for Disease Control and Prevention. AAP = American Academy of Pediatrics. IIC = Indiana Immunization Coalition. AHEC = Area Health Education Centers. TMA = Texas Medical Association. SoM = School of Medicine.

^a Delivering recommendations included phrases about: a) delivering recommendations for HPV vaccination to decrease missed clinical opportunities, b) delivering strong recommendations for HPV vaccination, c) delivering effective recommendations for HPV vaccination, d) communicating with and/or education and/or counseling patient about HPV vaccination, e) communicating with and/or education and/or counseling parents/caregivers/family members about HPV vaccination, and f) parents’ concerns or questions about HPV vaccination.

^b Strategies included phrases about using evidence-based strategies to include HPV vaccination.

^c HPV epidemiology included phrases about epidemiology terms regarding HPV including incidence and prevalence rates, incidence and prevalence of HPV-related cancers, causative agent and risk, and HPV-related disease.

^d HPV vaccine included phrases about: a) HPV vaccine being used as cancer prevention, b) HPV vaccination rates, and c) HPV vaccination hesitancy, vaccination refusal and barriers to parental consent.

^e Guidelines included phrases about: a) the recommended age, group (adolescents), or sex for HPV vaccination, b) administering (physically giving the HPV vaccine) the HPV vaccine to adolescents, c) locating or identifying resources, avenues, or locations aligned with current immunization guidelines administering the HPV vaccine to adolescents, and d) HPV vaccine safety.

^f Prevention services included phrases about prevention health services (smoking cessation, weight reduction, diabetes screening, etc.).

^g HPV vaccination advocacy included phrases about advocating for the HPV vaccine.

^h Disparities included phrases about disparities in HPV vaccination.

Delivering recommendations

Of the 68 phrases included in the analysis, 10 (15% focused on clinicians delivering recommendations. Delivering recommendation phrases included delivering effective recommendations (n = 1; 1%), communicating and/or educating parent (n = 6; 9%), and communicating regarding parents’ concerns or questions (n = 3; 4%).

Strategies

Only one learning objective (1%) included a phrase to have audience members implement evidence-based strategies to increase HPV vaccination rates.

HPV epidemiology

Nearly 9% (n = 6) of the learning objectives included phrases that focused on providing information about the epidemiology of HPV (e.g., “describe what is known about HPV disease prevalence”).

HPV vaccine

Eleven total phrases included messages about HPV vaccination within the learning objectives. Specifically, three phrases (4%) focused on HPV vaccine as cancer prevention, one phrase (1%) included HPV vaccination rates, and seven phrases (10%) focused on HPV vaccine hesitancy.

Guidelines

Phrases about guidelines were the most common messages in the interventions’ learning objectives with a total of 32 phrases (47%). Phases about guideline recommendations had the highest count with 17 (25%), followed by phrases about guidelines for vaccine administration resources (n = 6; 9%), guideline for vaccine administration (n = 5; 7%), and guidelines for safety (n = 4; 6%).

Prevention services

There were three (4%) prevention services phrases listed within the learning objectives. These phrases included a statement such as, “Implement disease detection and prevention health care services (e.g., smoking cessation, weight reduction, diabetes screening, blood pressure screening, immunization services) to prevent health problems and maintain health.”

HPV vaccination advocacy

Only one (1%) learning objective included a phrase about HPV vaccination advocacy. This phrase included the following statement: “Have greater competence related to HPV vaccination advocacy.”

Disparities

There were four (6%) phrases that included messages about disparities in HPV vaccination. An example is, “identifying opportunities to decrease racial disparities in human papillomavirus (HPV) immunization.”

Miscellaneous

There were six learning objectives that were not included in the above categories due to lack of clarity and considered outliers. These learning objectives included: a) have increased knowledge related to efficacy data; b) identify the Rights of Medication Administration for Vaccines; c) explain the rationale for early vaccination, d) describe characteristics of the vaccine used to prevent this disease – characteristics may include schedule, contraindications, and/or adverse reactions; e) assess the prevalence of dismissing pediatric patients because of vaccine refusal, f) describe the importance of HPV vaccine; and g) review the latest clinical data regarding the impact of HPV vaccination on relevant clinical endpoints.

2.3 Assessment learning objectives addressed within the content of CME activities

As a salient component of educational activities and evaluation, we determined if the learning objectives were addressed within the content of the CME activities in the study. All but one objective was addressed at least once within the activities. The most addressed learning objectives were “recognize the recommended routes and sites for vaccine administration” (n = 33), “assess current recommendations for adult vaccination from the Advisory Committee on Immunization Practices” (n = 17), and “identify recommended vaccine administration best practices” (n = 15). Table 3 provides each of the CME activities along with the activities’ respective learning objectives and the frequency of which each learning objective was addressed in the activity.

Table 3. Assessment of learning objectives addressed within the content of CME activities.

Online Continuing Medical Education Title	Source	Objectives	Occurrence
You are the key to HPV cancer prevention	CDC	Explain in plain language, the importance of HPV vaccination for cancer prevention and the rationale for vaccinating at ages 11 or 12.	5
		Provide an effective recommendation for HPV vaccination	3
		Describe useful and compelling information about HPV vaccination to succinctly answer questions from parents and aid parents in making the decision to vaccinate their children	5
		Implement disease detection and prevention health care services (e.g., smoking cessation, weight reduction, diabetes screening, blood pressure screening, immunization services) to prevent health problems and maintain health	1
Adolescent immunizations: Strong Recommending the HPV Vaccine	AAP	No longer available for review	--
HPV Documentary-Division of Continuing Medical Education	IIC	No longer available for review	--
You are the key to HPV cancer prevention	AHEC	Define the importance of HPV vaccination for cancer prevention and the rationale for vaccinating at ages 11 or 12	1
		List the recommendations for HPV vaccine for girls and for boys.	3
		Provide useful and compelling information about HPV vaccine to parents to aid in making the decision to vaccinate	4
		Locate resources relevant to current immunization practice.	1
Immunization: You call the shots-	CDC	For HPV, describe this disease, including the causative agent.	1
module eight-HPV, 2016		For HPV, list the groups at highest risk.	2
		For HPV, identify those for whom routine immunization is recommended.	1
		For HPV, describe characteristics of the vaccine used to prevent this disease. (Characteristics may include schedule, contraindications, and/or adverse reactions.)	5
		Locate resources relevant to current immunization practice.	1
		Implement disease detection and prevention health care services (e.g., smoking cessation, weight reduction, diabetes screening, blood pressure screening, immunization services) to prevent health problems and maintain health.	0
Don’t wait vaccinate! The prevention of HPV cancers	TMA	No longer available for review	--
Don’t wait vaccinate! The prevention of HPV cancers (Part 2)	TMA	No longer available for review	--
Human Papillomavirus (HPV) Vaccine Safety	CME University	No longer available for review	--
HPV Vaccines Updates and Perspective	Medscape	Upon completion of this activity, participants will have greater competence related to apply the Advisory Committee on Immunization Practices (ACIP) recommendations for human papillomavirus (HPV) vaccination	2
The story of HPV: yesterday, today and tomorrow	Medscape	Upon completion of this activity, participants will have increased knowledge regarding the long-term data evaluating the impact of vaccination on HPV-related disease	3
		Timely and appropriate HPV vaccination	2
		Have greater competence related to: HPV vaccination advocacy	1
Putting HPV vaccine knowledge into practice	Medscape	Upon completion of this activity, participants will have increased knowledge related to identifying opportunities to decrease racial and ethnic disparities in human papillomavirus (HPV) immunization	2
CDC updates guideline recommendations for HPV vaccination	Medscape	Interpret HPV vaccination recommendations for girls and boys who begin the vaccination series at ages 9 through 14 years, based on updated guidance from the Advisory Committee on Immunization Practices (ACIP	4
		Apply HPV vaccination recommendations for persons beginning the vaccination series at ages 15 through 26 years, for immunocompromised persons, and for other groups, based on updated ACIP guidance	4
HPV vaccine safety and efficacy	Medscape	Have increased knowledge related to HPV safety and efficacy data	14
Overcoming gender and socioeconomic disparities in HPV vaccination	Medscape	Have greater competence related to identifying opportunities to decrease gender and socioeconomic disparities in HPV immunization rates	4
ACIP releases pediatric vaccine schedule	Medscape	Assess the 2017 updated schedule for child and adolescent immunizations, based on an Advisory Committee on Immunization Practices (ACIP) statement	5
		Recognize barriers that need to be overcome to complete adolescent vaccine schedules	7
AAP provides guidance for parents who refuse vaccination	Medscape	Assess the prevalence of vaccine hesitancy and dismissing pediatric patients because of vaccine refusal	1
		Analyze means to reduce vaccine hesitancy and vaccine refusal among caregivers of children and adolescents	12
ACIP releases adult vaccine	Medscape	Evaluate trends in adult vaccination in the United States	1
recommendations		Assess current recommendations for adult vaccination from the Advisory Committee on Immunization Practices	17
HPV vaccines: updates and clinical perspective	Medscape	Have greater competence related to applying the Advisory Committee on Immunization Practices (ACIP) recommendations for human papillomavirus (HPV) vaccination	3
HPV vaccination is cancer	Boston	Describe what is known about HPV disease prevalence	6
prevention (2017 update)	SoM	Explain the rationale for early vaccination	5
		Determine whether a patient should receive two or three doses based on the age of initiating the vaccine	3
		Implement an evidence-based strategy to increase HPV vaccination	6
Increasing adolescent	CME	Identify the Rights of Medication Administration for Vaccines	2
immunization coverage	University	Define the steps for proper vaccine administration	10
		Recognize the recommended routes and sites for vaccine administration	33
		Identify recommended vaccine administration best practices	15
		Describe best practices to prevent vaccine administration errors	5
		Locate resources on current immunization administration practice	2
		Implement disease detection and prevention health care services (e.g., smoking cessation, weight reduction, diabetes screening, blood pressure screening, immunization services) to prevent health problems and maintain health	7
Immunization: You call the shots- module eighteen-vaccine	CDC	Review the latest clinical data regarding the impact of HPV vaccination on relevant clinical endpoints	9
		Assess new strategies to educate and empower adolescents to become engaged in their own health as it relates to HPV vaccination	2
		Evaluate the value of concise and consistent communication to educate patients/caregivers on the importance of competing the HPV vaccination series	4

Note. CDC = Centers for Disease Control and Prevention. AAP = American Academy of Pediatrics. IIC = Indiana Immunization Coalition. AHEC = Area Health Education Centers. TMA = Texas Medical Association. SoM = School of Medicine.

2.4 Learning objective verbs

There were a total of 51 learning objective verbs listed within the 21 interventions (Table 4). There were 17 (33%) knowledge level objective verbs. The most frequent knowledge verb was identify (12%), followed by locate (8%), list (8%), and define (6%). Comprehension verbs totaled 12 (24%) with describe (16%), explain (6%), and discuss (2%). There were a total of eight (16%) application level verbs. These included implement (10%), apply (4%), and employ (2%). Analysis verbs summed 3 (6%) with recognize (4%), and analyze (2%). There were six (12%) evaluation level verbs which included assess (6%), interpret (2%), evaluate (2%), and determine (2%). There were five learning objectives that did not include higher order thinking verbs. These included increase knowledge (4%), answer (2%), introduce (2%), and provide (2%).

Table 4. Online continuing medical education learning objective frequency.

Learning Objective Terms	Frequency N (%)
Knowledge
Identify	6 (12)
Locate	4 (8)
List	4 (8)
Define	3 (6)
Comprehension
Describe	8 (16)
Explain	3 (6)
Discuss	1 (2)
Application
Implement	5 (10)
Apply	2 (4)
Employ	1 (2)
Analysis
Recognize	2 (4)
Analyze	1 (2)
Evaluation
Assess	3 (6)
Interpret	1 (2)
Evaluate	1 (2)
Determine	1 (2)
No Verb Category
Increase Knowledge	2 (4)
Answer	1 (2)
Introduce	1 (2)
Provide	1 (2)

3. Discussion

The purpose of this study was to (1) assess the content of the purpose statements and learning objectives of online HPV vaccine CME activities developed for clinician and (2) identify themes and gaps in the purpose statements and learning objectives. In our analysis of online HPV vaccine CME modules, we found phrases about guidelines for vaccine administration and vaccine recommendation guidelines were the most common for purpose statements and learning objectives, respectively. Phrases about providing medical news to clinicians, science supporting the guidelines, and HPV vaccine hesitance were also frequently mentioned. However, the majority of purpose statements and learning objectives did not contain phrases pertaining to addressing missed clinical opportunities for vaccine recommendation. This finding is consistent with prior research of online HPV vaccine CME modules.⁴⁰ This limitation of online HPV vaccine CME activities is concerning considering the primary reasons for low HPV vaccination rates in the US are missed clinical opportunities and a lack of strong recommendations by providers.^7,9

Our analysis found almost one-third of the activities lacked a purpose statement and several included non-measurable learning objectives. Purpose statements facilitate complex processing, help organize content for learners,⁴⁵ allow for deliberate practice, encourage the effective transfer of knowledge,⁴⁷ provide balance and autonomy in self-directed learning, and help build awareness of one’s own thinking process. Moreover, learning objectives detail what a participant can expect to learn from an educational activity. These statements should describe knowledge, skill, and attitude changes that should occur upon successful completion of activities. In the current analysis, a majority of learning objectives focused on low-level learning – knowledge and comprehension – with very few focused on critical thinking or application skills. Only one learning objective focused on behavior/skill change. The lack of learning objectives focused on behavior/skill change is concerning considering clinicians do not feel well prepared to provide strong vaccine recommendations.^14,22,23 Medical educators can improve future online HPV vaccine CME modules by including more content aimed at behavior or skill change, specifically, skills in vaccine recommendation and addressing parent vaccine hesitancy.

It is important to note there is a possibility that content within the activities may not have been reflected in the learning objectives. For example, while several activities did not include communication training in the learning objectives, there was the potential that the activities included this information in the session. However, a previous content analysis of similar CME activities revealed there was more information about HPV epidemiology and vaccination coverage then delivering recommendation and addressing parents’ concerns,⁴⁰ which aligns with our findings of the learning objectives. In addition, as reflective of educational best-practices, content needs to align with the learning objectives as objectives guide the development of the content. This practice of aligning the content to the objectives ensures participants understand exactly what content will be covered within the activity. Furthermore, since evaluations are concentrated on assessing activity objectives, if learning objectives are not fully reflective of the content, this will lead to an incomplete assessment of the knowledge and skills the activities addressed. There is also the potential that evaluations may not capture unintended consequences or boomerang effects of educational activities. Thus, to guarantee CME activities are centered on best practices in education and evaluation, content must be aligned with learning objectives.

Various professional organizations including the Accreditation Council for Continuing Medical Education (ACCME),⁵⁰ Accreditation Council on Graduate Medical Education (ACGME),⁵¹ and the Liaison Committee on Medical Education (LCME)⁵² require learning objectives for practitioners, residents, and medical students, respectively. While learning objectives are a key component of instructional alignment,⁵³ they are also an educational roadmap critical for alignment with purpose statements. In our analysis however, the purpose statements did not fully align with the stated learning objectives. Out of the 21 CME activities, only two activities contained the same messages in the purpose statement as well as the learning objectives. For example, missed opportunities and strong recommendations are included in various purpose statements but not in any of the learning objectives. In addition, the learning objectives had an extensive focus on guideline recommendations; this focus is not reflected in the purpose statements. The lack of alignment between purpose statements and learning objectives reflects a lack of consistency and understanding regarding how to increase clinicians’ recommendation behaviors. This finding further emphasizes the needs for purpose statements and learning objectives to be aligned in order to achieve specific educational goals and allowing for deliberate applications related to behavior and skill change.⁴⁷

Another notable finding from this study was that 44 out of 45 (98%) learning objectives listed for the CME activities were addressed in the activities. This alignment with learning objectives and content is critical as the first step in developing competency-based education providing the learner with a consistent and effective learning experience.^41,48 Evidence-based educational practices require learning objectives to be used as the organizational framework to assemble educational activities.⁴⁹ Our results demonstrate a strength of the online HPV vaccine CME activities regarding the use of evidence-based educational practices to utilize the learning objectives to guide the organization and development of the activities to begin providing the learner with a practical and fruitful learning experience.

While this analysis provides insightful information regarding the direction and quality of web-based HPV vaccine educational interventions’ purpose statements and learning objectives, there are several limitations to consider. First, only online content was analyzed. Therefore, these results cannot be generalized to other types of interventions such as in-person lectures, workshops, seminars, or grand rounds. These alternative delivery methods could differ in terms of overall purpose statements and learning objectives, content, and the quality of intervention.³⁹ The second limitation is the timeline of sample selection. Modules created before or after the study timeframe could have varying purpose statements and learning objectives. Third, this analysis had an exclusive focus on purpose statements and learning objectives. Additional studies are needed to compare purpose statements and learning objectives to the content within the educational modules for evaluation of learning methods, the curriculum’s alignment with the purpose and learning objectives (instructional alignment), as well as themes, concepts, quality, and clarity. Despite these limitations, the findings from this analysis provide valuable information for those who develop online continuing medical education activities regarding HPV vaccines, by identifying strengths and weaknesses of the HPV vaccination online educational activities’ instructional output.

Many of the purpose statements and learning objectives were challenging to comprehend and determine what knowledge, attitudes, or skills would be assessed, which is needed to conduct assessment of instructional alignment, to clearly communicate what an intervention addresses, and communicate its intended outcomes for the participant (e.g., increased knowledge and changed behavior). In addition, many interventions’ purpose and learning objectives focused on providing information about HPV, the vaccine and guidelines with limited information about how to decrease missed opportunities for vaccination and how to provide strong and consistent recommendations for the vaccine. Due to their economic and geographical flexibility, online activities will continue to be used to provide CME for clinicians. Thus, there is a need for the purpose statements and learning objectives to effectively orient the learner to the type of knowledge and skills addressed in the continuing education activity for clinicians. Clear and intentional purpose statements and learning objectives must be used to guide the development of quality interventions to allow for evaluation which at the minimum should include an increase in knowledge and best-case scenario a positive change in behavior.

4. Methods

4.1 Sample selection

We conducted a content analysis of the purposes and learning objectives of online HPV vaccine CME activities with available and current CME credits any time during the study period starting in August 2016 up to August 2017. Systematic methods utilized to identify the online HPV vaccine CME activities included using general search engines such as Google, CME search engines such as Medscape, health department websites, and professional organizations websites such as American Academy of Pediatrics. Search terms and variations of search terms used in each search engine and website included “clinicians,” “healthcare providers,” “HPV webinars,” “HPV vaccination webinars,” “HPV online education,” and “HPV continuing medical education.”

The initial search identified 178 online HPV vaccine CME activities. Of the 178 online CME activities, 21 met the following inclusion criteria: (1) created for clinicians (defined as individuals qualified to deliver health-care services – such as physicians, nurse practitioners, physician assistants, clinical nurses, and school nurses – to patients aged between 9 and 26 years); (2) delivered information about the HPV vaccine; and (3) provided current CME credits. Online CME activities were excluded if they were created for patients, adolescents, parents, or focused on vaccines that did not include the HPV vaccine.

4.2 Analysis

We conducted a content analysis by analyzing the purpose statements and learning objectives for each online HPV vaccine CME activity. The unit of analysis used was a phrase and was considered to end with the term “and”.⁵⁴ To identify key messages, one author (BLR) independently conducted open coding of the purpose statements and learning objectives. Topics for purpose statements included: 1) delivering recommendations, 2) HPV epidemiology, 3) HPV vaccine, 4) guidelines, and 5) medical news. Topics for learning objectives included: 1) delivering recommendations, 2) strategies, 3) HPV epidemiology, 4) HPV vaccine, 5) guidelines, 6) prevention services, 7) HPV advocacy, and 8) disparities. Two authors (JMB and RA) independently coded each CME activity’s purpose statement and learning objectives using the codebook. The first round of coding for the purpose statements resulted in 100% agreement between reviewers and no revisions were made to the codebook. The first round of coding for the learning objectives resulted in 55.6% agreement between reviewers, Krippendorff’s alpha = 0.478, and eight disagreements. The disagreements were reviewed by a third author (BLR) and revisions were made to the codebook based on discussion about the disagreements. After the second round of coding for the learning objectives, there was 100% agreement with Krippendorff’s alpha = 1.00, and no disagreements. To determine if the activities learning objectives were addressed within the activity, JMB and RA independently coded each activity using the learning objectives as the codes. To analyze the learning objective verbs, we examined each learning objective for a verb. Once the verb was identified, we used Bloom’s Taxonomy – a framework for categorizing educational goals and learning objectives – as the method to delineate types of learning objective verbs.⁵⁵ These verbs were categorized by knowledge, comprehension, application, analysis, synthesis, and evaluation.⁵⁵

Disclosure of potential conflicts of interest

No potential conflicts of interest were disclosed.

Additional information

Funding

No funding was received to conduct this study.