Breaking Down the Skills: Designing and Pilot-Testing a Video-Based Microskills^® Training for Sexual Health Educators

Abstract

Effective sexual health education plays a critical role in adolescent pregnancy and HIV and STI prevention. This article describes the formative research conducted to design, and then pilot-test, SkillFlix^®, a streaming video training created to improve educators’ skills in delivering sexual health education to youth by modeling Microskills^®. Results from pilot testing of the SkillFlix training exceeded expectations, with educators demonstrating significant improvement in integrating and successfully using two prototype Microskills: (1) correct implementation of LGBTQ inclusivity; and (2) answering of sensitive questions. Educators’ perceived skill levels also increased significantly after using SkillFlix. Skill acquisition occurred even though the educators employed content (vs. skills acquisition) focus when using the resource.

Keywords:

• Sexual health education
• video modeling
• training
• online learning
• inclusivity
• SkillFlix^®
• Microskills^®

Introduction

Recent statistics concerning adolescent sexual risk behaviors in the U.S. present a mixed picture. On the one hand, the country’s adolescent pregnancy rate (births per 1000 females ages 15–19 in a given year) has decreased each year since 2009 (Martin et al., 2019). As of 2018, the birth rate for females aged 15–19 in the United States was 17.4 births per 1,000, down 7% from 2017 (18.8) and down 58% since 2007 (41.5), the most recent high (Martin et al., 2019). On the other hand, the teen birth rate in the United States (18.8%) remains higher than that in many developed countries, including Canada and the United Kingdom (CDC, 2018a; Martin et al., 2019; Sedgh et al., 2015). Moreover, young people in the United States continue to engage in sexual risk behaviors that can result in unintended health outcomes, including pregnancy, HIV, and other sexually transmitted infections (STIs) (CDC, 2018a, 2018b; Martin et al., 2019). An estimated 21% of all new HIV diagnoses in 2016 were among those ages 13–24 (HIV Surveillance Report, 2016), and half of the 20 million new STIs reported each year are among those ages 15 to 24 (CDC, 2017; Martin et al., 2019). In 2017, 40% of high school students were engaging in sexual activities, with 10% reporting four or more sexual partners (CDC, 2018b). Condom use among this age group has also declined nearly 8% in the last decade (CDC, 2018b). Just 53.8% of sexually active teens reported using a condom during their previous sexual activity (CDC, 2018b).

The year 2015 marked the first time that the CDC analysis of national data gathered from the 2015 Youth Risk Behavior Survey included information on the health risks of LGBTQ high school students. The findings showed that LGBTQ youth report a higher incidence of physical and sexual dating violence, as well as bullying, drug and alcohol use (Kann et al., 2016; Puckett et al., 2015). Relative to their heterosexual peers, LGBTQ youth are also disproportionately affected by suicide-related thoughts and behaviors (O’Brien et al., 2016). The suicide rate remains 2–7 times higher among the LGBTQ community than among heterosexuals (Marshall, 2016; Haas et al., 2010), and recent research indicates a dramatically high suicide attempt rate among transgender youth of 40% (Haas et al., 2015). All of these statistics point to the ongoing need for effective and inclusive sexual health education for, and prevention efforts with, our nation’s youth.

School-based sexual health education and prevention programs remain critical

The U.S. Department of Health and Human Services recognizes the benefits of school-based sexual health education. It has set a national goal of increasing comprehensive sexual health education in middle and high schools (U.S. Department of Health & Human Services [US DHSS], 2017). The National HIV/AIDS Strategy also recognizes that schools can play a “fundamental role in providing current and accurate information about the biological and scientific aspects of health education” to adolescents, easing STI prevention and management barriers (CDC, 2018a, 2018c; Whitehouse Office of AIDS Policy, 2015). The American Academy of Pediatrics, American College of Obstetricians & Gynecologists, American Medical Association, and American Public Health Association all recommend implementing comprehensive school-based sexual health curricula (due to the strong correlation between increased student knowledge and delayed sexual activity, fewer sexual partners, and consistent condom usage) (Alford, 2012; Elliott, 2009; Kirby, 2008; Sexuality Information & Education Council of the United States [SIECUS], 2014; Zief et al., 2013).

Despite these well-recognized benefits, the percentage of schools providing sexual health education has declined over the last 15 years, an outcome of complex factors at play, including federal policy and budgeting shifts (e.g., periods of federal funding of abstinence-only sexual health education curricula), state and local school budgeting constraints, and ongoing social and cultural debates about the appropriate content for sexual health education in schools (CDC, 2018d; Hall et al., 2016; HHS & CDC, 2015; Strasburger & Brown, 2014). Schools nevertheless remain the primary source of this education—80% of those aged 15–19 reported receiving some formal sexual health education at school—and more youth receive sexual health education at school than from their parents (Lindberg et al., 2016; National Association of School Nurses, 2016).

Notably, schools as a primary source for sexual health education can present a problematic reality for LGBTQ students. This diverse population experiences multiple sexual health inequities driven, in part, by deficits in exclusionary school-based sex education programs (Mustanski et al., 2015). In contrast, research has documented that when LGBTQ youth perceive greater inclusivity in sexual health education, this has been associated with lower levels of depression, anxiety, depression, and suicidality (but not associated with sexual risk-taking and substance use) (Keiser et al., 2019). When asked how to make exclusive sexual health education more inclusive, LGBTQ youth have offered strategies such as directly discussing LGBTQ issues, recognizing healthy relationships among LGBTQ youth, and emphasizing sexually transmitted infection (STI) prevention over pregnancy prevention (Gowen & Winges-Yanez, 2014). Ultimately, comprehensive sexual health education is needed not just for reducing STI/HIV rates and unintended pregnancies, but also for inclusively supporting adolescents of diverse gender/sexual identities as they navigate a range of sexual choices and experiences. Educators need support in achieving these ends.

Many educators lack the skills to teach sexual health education

While there are numerous, evidence-based sexual health curricula, these curricula must be delivered effectively. Lessons are most effective when implemented by skilled educators (schools and other settings). They are comfortable leading the particular types of instructional activities that are integral to sexual health education (e.g., group discussions, role-plays, condom demonstrations). However, variability in educator preparedness, self-efficacy, comfort levels, and enthusiasm can significantly impact both curriculum implementation and outcomes (Dickson et al., 2019; Jaycox et al., 2006; Kam et al., 2003; Wandersman & Florin, 2003).

Researchers have found that instructional support is warranted for effective implementation of sexual health programs (Dickson et al., 2019; Eisenberg et al., 2013; Fagan et al., 2008). Yet no states regulate the professional training or development of sexual health educators and only 13 states require such instruction to be “medically accurate” (Guttmacher Institute, 2017). While outside trained sexual health educators may be hired and paid by school districts to teach this content, in many U.S. schools, teachers are commonly assigned to teach sexual health education with insufficient training (Barr et al., 2014; Eisenberg et al., 2013). This training deficit is evident. Educators report discomfort when teaching sexual health course material, uncertainty when answering student questions, and a lack of confidence and skill when implementing unfamiliar instructional activities, such as role-plays and group discussions on sensitive topics (Wang et al., 2015). Teachers also face structural barriers in teaching sexual health education. For example, in one study of 368 middle and high school teachers with sexuality education assignments in Minnesota, almost two-thirds of participants cited restrictive policies over what could be taught (e.g., teen parenting, abortion, contraception) and 45% expressed concerns about parent, student, and administrator responses to particular topical areas being taught, such as sexual violence, sexual orientation, and abortion (Eisenberg et al., 2013).

Educators also often lack the skills needed to create an inclusive environment for all students. For example, less than 5% of students report that lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ) topics are represented positively in their health courses (Kosciw et al., 2014). Research has shown that current sexuality education materials are generally not tailored to meet the needs of LGBTQ youth, and “many have been critiqued for disenfranchising these populations” (Gowen & Winges-Yanez, 2014). Notably, even among educators working to promote inclusivity in sexual education, heteronormative assumptions and discussion may prevail (Abbott et al., 2015; Elia & Eliason, 2010).

Students note other educator shortcomings. While youth occasionally praise their sexual and reproductive health teachers (Allen, 2009; Levin, 2010; MacDonald et al., 2011), they generally regard their teachers as unsuitable for teaching the subject, citing lack of training and reliance on passive teaching methods (e.g., showing outdated films) (Javadnoori et al., 2012). Other shortcomings noted include teacher embarrassment and inability to discuss sex frankly (Bourton, 2006; Lester & Allan, 2006) as well as inability to satisfactorily respond to student questions (Forrest et al., 2002; Reeves et al., 2006). These teacher shortcomings affect student perceptions about the quality of their sexual health education (Bleakley et al., 2009; Lester & Allan, 2006). For example, the inability to talk frankly about sex affects student perceptions of teacher credibility (Hilton, 2003; Kimmel et al., 2013). These types of perceptions may also encourage adolescents' to seek out and reply upon critical sexual health information from peers (Guttmacher Institute, 2016; Helleve et al., 2011).

Current professional development resources often do not meet educator needs

Professional development and training can effectively address these challenges by building educators’ knowledge, skills, and confidence so they can teach sexual health curriculum effectively, engage students, and skillfully implement instructional strategies (Kalafat et al., 2007; Kirby et al., 2014; Rohrbach et al., 2010). Unfortunately, traditional training and development options often fall short for multiple reasons. First, if educators receive any sexual health training, it usually occurs after their career has begun, when they face significant time and resource constraints. While some in-person sexual health educator trainings are affordable or even free (e.g., the California Prevention Training Center¹ offers free sexual health trainings to health professionals and educators), many in-person face-to-face trainings are expensive, with some training organizations charging up to $20,000 for 20 participants (E. Blanke, Personal communication, ETR, 2018). These trainings can require participants to take time off of work for travel to a distant location. Second, trainings usually focus on a specific curriculum, rather than building the universal skills educators need to implement sexual health curricula effectively. Third, even highly rated, face-to-face trainings rarely provide continuous implementation support.

Existing internet-based training resources have several shortcomings

Online training resources for educators have the potential to address cost and scheduling challenges. They can be used nearly anytime and anywhere, allowing for repeat usage, reinforcement of information, and practice. They can also be configured to provide ongoing support (Killion, 2001; 2002; Means et al., 2009). However, online resources frequently suffer from two major shortcomings, the first being low teacher completion rates (Firpo-Triplett et al., 2012, 2014; Tyler-Smith, 2006; Welsh et al., 2003). Online trainings typically present information in lengthy blocks totaling many hours, which generally does not fit the time constraints faced by in-service teachers. Second, most online training is curriculum-specific. The average high school curriculum only accommodates 6.2 h of instruction in human sexuality, which precludes implementing a full evidence-based program (Welsh et al., 2003). Consequently, many teachers are forced to put together their own non-standardized curriculum (CDC, 2012).

Streaming video holds significant promise for educator training

The uses and benefits of video for learning and training are well documented (Bayram, 2012; Lindgren et al., 2007; Yousef & Chatti, 2013; Zhang et al., 2006). Videos have been found to be the most useful component of online training studies (Firpo-Triplett et al., 2012, 2014). They offer a particularly valuable tool for modeling skills—a fundamental construct in numerous behavior change theories (Rosales et al., 2015)—and also employ constructive learning processes (Kurz & Batarelo, 2010). Research has found that video-based e-learning is superior to illustrated, text-based e-learning for teaching practical skills to health practitioners (Buch et al., 2014). Video training designed to increase sex educators’ knowledge about HIV risks has also been found to outperform in-person training (Chao et al., 2010). Similarly, a study of teacher training on classroom management found video presentations to be more effective than standard presentations at increasing procedural knowledge of classroom management strategies (Hylton, 2000). Such strong results for video learning should come as no surprise, given that 80–90% of information transmitted to the brain is visual (Hyerle, 2000). This type of visual learning also decreases learning time by 40% and is more attractive to learners (Clark & Feldon, 2005). The explosive growth of streaming video services with short clips for entertainment and learning (e.g., YouTube, Lynda.com) is further evidence that users find this format engaging.

SkillFlix^®: using a Microskills^® approach for educator skills training

SkillFlix^® is a streaming video training platform designed to develop and support educator skills in delivering sexual health education to youth aged 14–18. Specifically, SkillFlix focuses on the development of teacher Microskills^® using a training method where educators view very short videos modeling the desired Microskills. Each larger skill (i.e., a skillset) to be learned is broken into a series of smaller steps (Microskills) that comprise the larger skill. For example, boiling water is a skill, but that skill encompasses several smaller skills, such as filling a pot with water, putting it on a heating element, turning on a heating element, etc. Through learning each Microskill, the learner builds the knowledge and confidence to implement the larger skillset. With SkillFlix, many Microskills are repeated across skill sets, providing several opportunities for learning and reinforcement regardless of the order or path the learner chooses to proceed through the material. SkillFlix terms and examples are described in Table 1.

Table 1. Key SkillFlix terms with examples.

Term with definition	Skill (Skill set) A set of Microskills that comprise a larger skill.	Microskill The natural “bite-sized” sub-skills that combine to create a skill. Microskills are the ideal size for quick learning via observation—they can be modeled in only 2–6 steps.	Microskill steps The specific steps (like steps in a recipe) that comprise a Microskill. Steps are modeled in each video with synchronized supporting text edited into the video.	Topic The content addressed in the Microskill video. "Topics are not inherent in the Microskill, but are a way to frame the skills of interest to educators.
Example	Answering Sensitive Questions	Answering an “Is it Normal?” Question Example: the anonymously submitted question, “Is it normal for my penis to be hard in the morning?” (See Table 3 for more examples.)	Provide definition of terms in the question. Address facts or values embedded in question. Match amount of detail to amount of time. Acknowledge range of experiences.	Anatomy and Physiology Erection Penis

Each SkillFlix video is 30–120 s long and models and trains the user on a Microskill™ beneficial for teaching youth sexual health education. Each Microskill is grouped within skill clusters and modeled using a dual video approach (i.e., community versus classroom settings, differences in level of detail offered, the depiction of talkative versus quieter students, etc.) to meet the varying needs of sexual health educators across the United States. When complete, the SkillFlix resource will contain 25 skill clusters for close to 500 videos.

The idea for SkillFlix was based on the developer’s many years spent developing skills-based training for sex educators both face-to-face and online. This led to an understanding of the skills sex educators need to perform well in the classroom and how best to build those skills. The resource—intended to be both time and cost-effective compared to in-person trainings—is a professional development tool that can be widely implemented at the state, district, school, organization, and individual educator level. The content of SkillFlix is informed by the new National Professional Learning Standards for Sexuality Education,² the National Sexuality Education Standards,³ and the Professional Learning Standards for Sex Education produced by the Sex Education Collaborative (2018).⁴ The content also reflects the sensibilities of research by Braun-Harvey (2019), emphasizing the importance of integrating six principles of sexual health—including both risk reduction and sex positivity messages—when facilitating sexual health conversations (i.e., a focus on STI and unintended pregnancy prevention, non-exploitation, consent, honesty, shared values, and sexual pleasure) (Braun-Harvey, 2019; Schnarrs et al., 2019).

SkillFlix is grounded in accepted learning theories

SkillFlix is grounded in well-accepted learning theories, primarily Social Learning Theory (Bandura, 1977) and Adult Learning Theory (Knowles, 1975; 1984). Social Learning Theory posits that people learn from one another via methods such as observation, imitation, and modeling. However, Bandura theorized that behaviors are not learned automatically simply by observing a model, then responding. Rather, cognitive or meditational processes intervene between the observation or stimulus, and the actual learning or response. Observational learning and skill modeling are central to skill acquisition (Bandura, 1977). A key feature of SkillFlix is learning through repeated observation and cognitive processing of videos that model Microskills in realistic scenarios. This approach is also supported by the literature on mirror neurons (the group of neurons that activate when we perform an action or see an action being performed) that are essential for imitation, a key to the learning process (Bonini & Ferrari, 2011; Jeon & Lee, 2018).

Adult Learning Theory posits four principles (Knowles, 1975, 1984): (1) that adults want to determine their own learning path as opposed to being led through a linear process; (2) that adult learning is problem-centered rather than content-oriented; (3) that adults are most interested in learning subjects of practical importance and immediate relevance; and (4) that adults’ existing experience (including mistakes and challenges) provides the basis for learning activities (Kearsley, 2010). SkillFlix facilitates adult learning self-determination by giving users multiple methods of accessing the material (e.g., tailored recommendations, selection of skills and Microskills) and engaging with the material (e.g., ability to select the content of interest, next video of interest). SkillFlix also teaches adult learners information of practical importance and relevance to their work (i.e., mastery of Microskills for communicating effectively with students on sexual health topics) while building on their existing experiences with these efforts. In sum, both of these theories were used to inform the design of the SkillFlix application.

SkillFlix’s features

SkillFlix includes several other innovations besides its focus on Microskills. It provides “bite-sized” videos that have proven popular with educators at large and meet the time constraints reported by in-service educators. It gives educators a “grab and go,” fast-pasted learning method they prefer by allowing them to focus on videos that most appeal to them. And while it is primarily aimed at novice sexual health educators with limited sexual health education training, it has the flexibility to serve educators with a broader range of experience and skills. After answering a series of questions, SkillFlix will be able to identify the user’s particular skill areas that need improvement and provide customized learning recommendations or “video pathways.” SkillFlix will also offer an online learning community where educators can upload their videos and receive peer observation feedback. This skill observation and critique invitation function is a valuable training method commonly used for student teachers but typically unavailable to in-service teachers due to logistical constraints. Table 7 more fully describes Skillflix’s features developed in this first phase and those planned for a robust second phase R&D effort.

Pilot study

The pilot study of the SkillFlix project was conducted in November and December of 2017 and focused on establishing the technical merit and feasibility of a subscription-based Microskills video training library for educators offering sexual health education to youth 14–18 years old. Specifically, the study explored whether watching short videos focused on Microskills modeling would have a significant and rapid effect on educator skills acquisition. The pilot study was completed on time and within budget, with all specific aims and benchmarks met (see Table 2).

Table 2. Summary of Phase I benchmarks of success (additional details in Tables 6 and 7).

Benchmarks	Phase I results
(1) Successful launch of SkillFlix website to prepare and support sex education teachers to use best practice Microskills.	The SkillFlix portal was successfully launched on October 28, 2017 with best practice video models for sex education educators in the pilot. The site continues to be available to key educators and trainers in the field.
(2) Satisfaction and usefulness results with mean ease-of-use scores of 5 or better on 7-point scales.	Mean scores from usability survey: Satisfaction = 6.9 Usefulness of Information = 6.36 Ease of Use = 6.27
(3) Increased knowledge and perceived skill level of educators using Microskills.	Reaction interviews revealed perceived increases in knowledge (in 100% of educators) and better skills in leading the lessons (83% teachers). Survey results showed a significant increase in perceived level of skill (p = .01).
(4) Increased demonstration of best practice Skills/Microskills as measured by comparing pre- and post-intervention assessment videos.	Observation data revealed significant and very large increases in Critical Skills Score in pre/post measures (p < .001) and the All Skills Scale (p < .001).

Table 3. Example of Microskills.

Answering Sensitive Questions is a skill set that includes 12+ Microskills, including:

Establishing group agreements Responding to “shock” questions (personal, angry, offensive language, technique questions) Responding to “am I normal?” questions Responding to “information-seeking” questions Responding to “permission-seeking” questions Introducing the anonymous question box Answering anonymous questions

Conducting the formative evaluation

Expert advisors

Project staff convened multiple meetings with four national expert advisors in the area of adolescent reproductive health to guide the overall conceptualization and training direction of SkillFlix.⁵ Two of the advisors also conduct national trainings on sexual health education. Among other topics, project staff sought input from the expert advisors on practical applications of learning theory approaches to the application, and whether any planned strategies needed to be reframed. Useful discussions took place with the advisory panel about how to balance theoretical approaches with some of the practical issues educators might face using the application. For example, it was decided we should not: (a) require SkillFlix users to watch videos in a particular order since this goes against the adult learning theory principle that learners want to choose their own learning pathways; (b) add reflection question exercises at the end of the MicroSkills videos since advisors felt that among target busy educators, “no one is going to do those.”

Focus groups

The advisory team also helped with the recruitment of sexual health education teachers to participate in four 90-min virtual focus groups. The goal of the focus groups was to understand the training needs and interests of those who teach sexual health topics to youth, including what skills and Microskills they felt were universally needed. Educator input was also solicited on resource features, the look and feel of the videos, and the overall website. Calls for participants were disseminated through the advisors’ networks and list serves. Project staff also disseminated calls for participants through our own national contacts in the SRH field as well as local school districts and community-based organizations. Four focus group members from our local region (two teachers and two CBO sexual health educators) agreed to act as a secondary advisory group, offering input on which Microskills we chose and feedback on drafts of the video scripts. Focus groups were conducted virtually to allow us to hear from a broader national audience of educators and sexual health education trainers. Project staff facilitated the virtual focus group; these were audio recorded, with an additional project staff member taking notes. Using a series of open-ended prompt questions, focus group participants were asked to provide their reactions, suggestions, and requests in response to a description of the project goals, proposed content and design features, and the learning theories guiding our approach. Focus group data was subjected to a thematic analysis by project staff, with key themes and recommendations highlighted for use in guiding the application’s design.

The video and website decisions were further developed through targeted phone meetings and online content reviews with both the national expert advisors and the additional local advisors. This expert input was instrumental in guiding the website development, as well as the skill set content of the videos for both the prototype and the planned second phase of R&D for this SkillFlix product. Advisors reviewed all prototype content before script development began.

Developing the content plan

Working closely with the national and local advisors, the project team identified high-priority skill sets needed to implement basic sexual education topics in comprehensive, evidence-based sexual and reproductive health (SRH) interventions. An initial 50-item plus priority list was first created with input from both sets of advisors based on their expertise and knowledge of the field. Project staff then honed the list down to 22 skill sets deemed the highest priority to include, with a focus on skillsets that are program agnostic (i.e., applicable to every classroom regardless of curriculum used) and reflecting known trends and funder interests in the training field. These included: setting group agreements, managing role plays, inviting diverse cultural perspectives, leading condom demonstrations, and teaching about cyber safety.

Next, staff performed the following tasks: (1) reviewed 28 evidence-based SRH interventions for teens identified by the Office of Population Affairs and analyzed them for needed instructional skills (Office of Adolescent Health, 2015); (2) ensured the Microskills were informed by the National Professional Development Standards for Sexuality Education developed as part of the National Sex Education Collaborative; and (3) reviewed other standards and requirements that inform sexual health educator skill needs, such as the National Sexuality Education Standards, Future of Sex Education (FOSE), California Healthy Youth Act, Sexuality Information and Education Council of the U.S. (SIECUS) Standards, and World Health Organization/European Union Standards for Sexuality Education. Two skillsets, Answering Sensitive Questions and LGBTQ Inclusivity, were identified as high priority and universally needed by the expert advisors and educators in the focus groups. They were chosen as the skillsets to be developed for the prototype SkillFlix application.

Developing the pilot study Microskills videos

Forty-eight video clips were developed to model Microskills in the two chosen pilot study skill sets. The project team conceptualized and defined the Microskill steps and characteristics that were then reviewed by the expert advisors and sexual health educators from the field. Then scripts for each Microskill were written. While the videos focus on effective modeling of a Microskill, each also contains common sexual health content to facilitate a demonstration of that skill in realistic contexts.

Microskills were modeled using a dual video approach. In addition to depicting school versus community-based settings, we also designed one set of videos to be more progressive in content, the other more conservative. In general, sexual health education offered in schools is more commonly monitored and subjected to extensive approvals than sexual health content offered in community-based settings. For example, a school or school district might only allow abstinence-based versus comprehensive sexual health programs to be taught. They may also place more conservative limitations on discussions of particular topics such as LGBTQ+ sexuality, sexual/gender spectrums, reproductive choice, etc. For these reasons, we created content in dual formats, broadly differentiated as “progressive” vs. “conservative” to address the different content approaches that might be found in community versus school-based settings. As an example, if we were modeling ways that an educator might respond to a student question asking “how can I please my partner?” a conservative answer would be: “We’re not going to get into discussing sexual techniques, but I can tell you that being able to communicate about what feels good to your partner is important.” For the more progressive answer, we modeled a more extensive response: “We’re not going to get into discussing sexual techniques, but communication about these issues is really important. For example, don’t assume that vaginal intercourse is going to be the most pleasing. Talk to your partner. Remember, too, that the more a person understands about self-pleasuring, the more they can communicate with their partner.”

After sexual health experts from dfusion, the expert advisory group, and the training advisory group conducted a final script review, video production began. The production company recruited talent for the adult educator actors (2 male and two female actors), and dfusion staff recruited local youth for the student actors (24 teens ages 14–18 from diverse ethnic backgrounds). After script clarification, prop assembly, and location preparation, two-camera video recording sessions took place over the course of two days. The video clips were strategically edited for heightened engagement. Microskill titles and explanatory bullet-point text summarizing the Microskill steps were edited into the video clips. The lower part of each video was left blank for the addition of closed captioning.

Examples of videos created for the LGBTQ Inclusivity skill set included the following five titles: Group Agreement: Respect Diversity (a video showing a teacher including an agreement for respecting diversity, explaining that includes sexual diversity; introducing sexual diversity early in the program); Inclusive Language: Relationships (teachers demonstrate how to use inclusive terms, address diversity in both classroom and community based settings); Inclusive Language: Anatomy and Physiology (teachers are taught to refer to body parts without gender terms; introduces inclusive terms and refers to sexual activity and behavior without assigning gender); Inclusivity: Lesbians & Condoms (teachers remind youth of the related agreement, acknowledge honesty in sharing orientation, acknowledge people may have sexual activity outside of their identified orientation, remind them to share the information with others); and Inclusive Language: Families (both videos demonstrate how to acknowledge diverse types of families; acknowledgement that one’s most trusted adult may not be a parent; not assuming youth have a mother or father).

Building the SkillFlix prototype website

dfusion’s technology team developed the prototype SkillFlix website, including the Drupal⁶-based infrastructure, visual presentation, and site layout. The infrastructure was designed from the outset to make the video clips easily accessible and searchable. Using input from the expert advisors and educator focus groups, the team created wireframes for the site that outlined each function, navigational flow, and placement of key navigation buttons. Staff met to discuss the overall site look and logo. Wireframes and initial visual concepts were provided to the graphic designer, who designed the key web pages. A library scientist created a comprehensive structured lexicon of the website’s taxonomy that ensures the content is well organized, and that the most relevant searchable content shows automatically, as needed, in the correct place. The taxonomy currently includes three classification areas: Skill Set, Microskills, and Topics. Examples of taxonomy tags include Masturbation, Oral Sex, PrEP, Porn, and Transgender.

Figure 1 Sample SkillFlix Video Web Page.

Videos were uploaded to Vimeo, an ad-free video-sharing platform, with selected specific properties and permissions specified for each. A web page was created for each video on the SkillFlix site with the following content: a title; Vimeo ID; thumbnail image; Skill Set PDF; shorter summary (for search results); steps shown in the video; taxonomy and tag selections; and next video in Skill Set (see Figure 1). After technology development was completed, the SkillFlix resource was subjected to internal system testing; changes based on these test findings were made, and a final alpha version was produced. Educators and trainers at ETR, a partner health organization then reviewed the site, along with dfusion staff that were not involved with the technology development. A small number of issues were found during alpha testing and addressed by the team, at which point the SkillFlix prototype was readied for the evaluation study. The SkillFlix orientation video can be viewed here: https://vimeo.com/317095031/ed8b1ddf5a

Method

Participants

Educator recruitment

The pilot evaluation was designed to assess the impact of SkillFlix with 10–12 local educators (see Table 4 for protocol). Staff recruited local educators within a 60-mile radius of dfusion’s Scotts Valley, California offices by connecting with professional contacts, key administrators, school educators, and youth-serving organizations. Of the 38 educators who completed the online interest survey, the first 12 who matched the following inclusion criteria were included in the study: (1) had experience as a classroom teacher/youth educator; (2) had three or fewer years’ experience teaching teens sexual health topics; and (3) had received minimal training on teaching sexual health topics. The selected educators represented three groups: school teachers, sexual health education teachers from CBOs that teach in schools, and sexual health education teachers that teach in community-based programs (e.g., Boys and Girls Club, after school community center programs, etc.) Educator participants completed informed consent protocols and received $400 for completing all study activities over an eight-day period.

Table 4. Pilot evaluation protocol.

Pilot Session One: Educators come to site and:

Lead 30-min sex ed lesson with 10 teens while being videotaped (pre-assessment video) Complete brief in-person reaction interview Complete pre-survey (self-perception survey) Receive instructions (including private passwords) on how to use SkillFlix. Next 7 Days: Educators used SkillFlix at their own pace, with number, order and combination of videos/features entirely determined by the user.

Pilot Session Two: Educators come to site and: Lead a different 30-min sex education lesson with the same 10 teens while being videotaped (post-assessment video) Complete post-survey (self perception-survey) Complete brief in-person reaction and usability interviews (and receive stipend) Site use tracking data was retrieved and analyzed.

“Student” actor recruitment

It was not feasible to have educators teach the lessons to their own students because it would have required the educators to have the curriculum approved by their school or district administration. Therefore, dfusion staff recruited 24 diverse youth ages 14–18 to play students. The youth were instructed to participate in the classes as they normally would, with seven instructed to make specific “planted” comments or questions at key points in each lesson. The planted questions and comments guaranteed that the educators would have the opportunity to use the Microskills in question so their use of the skills could be evaluated. Based on past online training research projects, project staff has found that, unless opportunities are expressly provided for certain skills or Microskills to be used, it can take many hours of testing for such opportunities to naturally occur.

Procedure

Lesson teaching

For the pretest, the educators were asked to teach one 30-min lesson, the first session, on a weekend. For the post-test, they prepared a different lesson the following weekend. The lessons were conducted in dfusion conference rooms staged to look like school classrooms. Each “class” contained ten diverse students aged 14–18. To reduce the expected practice effect, half of the educators led Lesson A and half led Lesson B during the pretest session, and each participant taught the other lesson during the post-test session. Lesson A was the HIV Risk Continuum Activity from the Making Proud Choices intervention (Jemmott et al., 2016); and Lesson B was the Protection Myths and Truths Activity from the Reducing The Risk intervention (Barth, 2015). Both lessons allow youth to learn and discuss a variety of information related to pregnancy and STI/HIV prevention. The lessons are self-contained, so participant educators did not need to do any additional research or content preparation. All materials needed for leading the lessons were provided two days before each lesson. Educators were instructed not to consume any outside training materials at this time so we could focus on what they learned from watching the videos and seeing the skills modeled.

Educator use of SkillFlix site between lessons 1 and 2

After teaching the first (pretest) lesson, each educator was provided access to the SkillFlix site, given instructions regarding its use, and instructed to view all 48 prototype videos. Each participant’s site usage was tracked to verify that they viewed each video at least once. Educators spent approximately 1 h in total to complete viewing all of the videos. One week elapsed between their initial access to the SkillFlix site and their second (post-test) lesson.

Materials

Data collection instruments included as follows.

Observation/coding form

A trained project staff member and an external consultant twice observed and rated each educator’s use of 7 key Microskills and applicable skill steps, once during the pretest and once during the post-test classroom sessions as captured via discrete GoPro video camera on tripods. Before coding, the project team reviewed all of the Microskills in each of the two skill sets (Asking Sensitive Questions, LGBTQ inclusivity), identifying which were deemed the “critical skills” in each set. Approximately 50% of the skills in each of the two sets were designated as critical. Both “Critical Skills Steps” and “Total Skills Steps” could then be tallied.

Interview protocol guide

To assess the educators’ experience, in-person interviews lasting 30–40 min were conducted immediately following the pretest and post-test classroom sessions. Two sexual and reproductive health trainers, each with over 20 years of experience in the field, conducted the interviews. Educators were asked a series of open-ended questions about what they felt went right during the lesson, what they thought could have been improved, and how they prepared. They were also asked what additional resources, if any, would have been helpful. After the second lesson, the educators were asked to describe how using SkillFlix impacted their leading of the lesson (e.g., how they used SkillFlix to prepare, what aspects of the training they applied in their sessions, and what additional resources they would have found useful in SkillFlix). Educators were also asked a series of usability questions for improving the SkillFlix videos and site features (e.g., look and feel of the site; ease of navigation and search functions, utility of ratings and comments features).

Self-perception survey

Immediately before leading the pretest lesson and post-test sessions, each participant educator completed a 13-item tablet-based survey to self-assess their comfort, confidence, and knowledge related to teaching sex education. These three measures were analyzed separately. The survey was developed by the authors and included varied question formats (Likert scale, multiple-choice, True/False and fill-in). It took 10–20 min to complete. Sample items included a focus on the educator’s knowledge of sexual health discussion strategies (e.g., True or False? It’s important to encourage all teens to talk to their parents about sex and protection); strategies for answering sensitive questions (e.g., A student asks: “I heard it’s possible to give a girl multiple orgasms; how do I do that?” How would you respond?) as well as self-perceived skill levels discussing LGBTQ topics and answering sensitive questions.

Tracking data

User data was collected from the SkillFlix interface that documented user logins, session duration, specific videos viewed, number of minutes spent viewing videos, and the total number of minutes on the website.

Results

As noted above, each lesson taught by a participant educator was video recorded and coded/scored for skills demonstration by two coders⁷ (a project staff member and external consultant), both with substantial experience in providing sexual health education to adolescents. To prevent bias, both coders were blind to (a) which videos were pre- or post- conditions, and (b) the order in which the educators taught the two skillset lessons. Coders watched the videos together, recorded their scores independently, then discussed and addressed any notable scoring discrepancies (inter-rater reliability scores were not established, however). The educators’ skill demonstration scores were analyzed to determine differences in both critical skills and all skills demonstrated during Lessons 1 (pretest) and 2 (post-test). For example, educators were taught that if someone makes a derogatory comment about LGBTQ issues, a critical skill step would be “state that the derogatory comments are disrespectful/hurtful.” A non-critical skill-step would be “acknowledge that you heard the comment.”

Specifically, a scaled score was created for critical skills in which participants received one point for each demonstration of a critical skill (e.g., enforcing group agreements, reframing the question, generalizing the answer). The total critical skill score was calculated as the total number of times the educator demonstrated the critical skills in each session. A second scaled total score was created for all skills in the prototype by again, awarding a point for each time the participant demonstrated any skills in the prototype. For example, with the skillset LGBTQ inclusivity, there were 30 total skill steps, 17 of which were deemed critical prior to coding.

Educators showed significant improvements in rated skills

T-tests were used to determine changes in both critical and total skill scores between the pretest and post-test. As shown in Table 5, participants demonstrated significant improvement (with a very large effect size) in both critical skills and total skills. Moreover, total skills t-tests showed significant improvement in the educators’ integration of each prototype skill (Answering Sensitive Questions, LGBTQ Inclusivity). Overall, participants more than doubled the number of times they were able to correctly implement LGBTQ inclusivity skills, and there was a more than threefold increase in the number of times they were able to correctly answer a sensitive question.

Table 5. Pilot study results (n = 11).

Scale	p	Effect size Cohen’s d
Critical Skills Scale	<.001	3.65
All Skills Scale	<.001	2.69
Answering Sensitive Questions, # of times correctly included	<.001	2.41
LGBTQ Inclusivity, # of times correctly included	<.001	1.65
Perceived Level of Skill	.01	1.19

Educators’ perceptions of their own skills

After using SkillFlix, both school and community educators’ self-perceived skill levels also increased significantly (p = .01). As noted above, all participants viewed each video at least once, with tracking data verifying educators’ self-reported SkillFlix usage time. There was no significant correlation between each educator’s total time spent on the SkillFlix website and their skill scores, suggesting that viewing each video only once was enough to improve both their expert-rated skills and their perceived skills significantly. The results of the in-person interviews with the educator participants immediately following the two lessons were also promising. After using SkillFlix, 100% of the participant educators perceived an increase in their knowledge of how to lead the lessons, and 83% perceived that their skills in leading the lessons had improved, as demonstrated by the following quotes:

SkillFlix helped me think about the importance of inclusive language, correcting slang like the “boner questions” (and) correcting derogatory language.

I felt bad last week (during pre-assessment lesson) like I had failed the kids (teacher shame). Today I feel great. SkillFlix really did help. It made me feel more confident – because I knew I was doing it right. I did some things right last time, but I just wasn’t sure. SkillFlix helped me feel more confident in what I know and in what I am learning.

Even though I have 20 years of experience being involved in sexual health, I learned a lot and it was very valuable.

Nearly all of the educators (91%) believed they did a better job implementing the second lesson, after using SkillFlix, than the first lesson. When asked what they did differently the second time, 83% believed they answered the challenging questions better, and the majority mentioned that they more skillfully included an LGBTQ youth focus in their lesson. For example:

I did a better job framing the responses. Before I just answered the questions, but today I did the whole process of affirming the question, looking for the underlying question/issue, creating a safe space, and following up to make sure the students were clear.

(The) videos are a nice reminder to be intentional about language, particularly about inclusive language—regarding gender, family, etc.

When asked how they could have better prepared for the second lesson, the majority mentioned that they could have watched the SkillFlix videos additional times:

Maybe watching the LGBTQ videos 2 or 3 more times would have been helpful. They were spot on. Even if the questions are different, the language and the approach is the same. Videos did a good job at showing how to do it. It just isn’t second nature yet.

SkillFlix site usability results

To assess educators’ experiences using SkillFlix, experienced sex educators conducted in-person interviews with the study participants. Each participant also completed the short, tablet-based self-perception survey immediately after the second pilot session. Overall, the website was very well received by the educators. On average, they used the site 88 min during the week between Lessons 1 and 2, and all participants viewed each video at least once (again, viewing all videos once takes approximately 60 min). On a 7-point Likert Scale, mean scores for participant satisfaction with SkillFlix was 6.9, and 6.27 for ease of use (see Table 6). All participants felt the video length was “about right,” and there was near-universal consensus on the appeal and utility of the videos for teachers.

Table 6. Usability survey results (n = 11) based on Likert scales with 1 = worst and 7 = best.

Question	Mean
Appearance of web site	5.18
Ease of use	6.27
How professional was the site	6.27
How informative was the site	6.18
How useful was the information presented	6.36
How much did you like the videos	6.09
How much did you like the educators in the videos	6.91
How much did you like the youth in the videos	7.00
Ease of navigation	6.11
Satisfaction, would recommend to colleague	6.90

Of the few usability concerns identified by the participating educators, the most common was the way some search result subsets were displayed. There was an inconsistent display for some search types, which caused confusion over which video to begin with when the user was new to the site. Approximately half of the educators wanted a more obvious home button. About 20% of participants found the dual video approach for each topic (i.e., two videos show either a classroom vs. community setting, a conservative vs. progressive approach, and more vs. less detail) to be redundant, while others found it reinforcing. The two most significant usability concerns related to topical content. First, some users did not fully understand the distinctions between skill sets, Microskills, and topics (see Table 1 for definitions). Second, some users—who do not necessarily distinguish content from skills—requested more content materials or links to content-related materials. Others characterized the topical content as easy-to-understand, intuitive, and reflecting an interesting approach to the material. In sum, the strength of this pilot data has provided strong preliminary evidence of the effectiveness of the Microskill^TM video models to improve skills, regardless of whether users approached the modeling videos with the intent to learn content or skills.

Discussion

The pilot study results demonstrated that short videos focused on Microskills modeling have a significant and rapid effect on educator skills acquisition, and that these effects can occur even when users approach the videos with the intent to learn content rather than acquire skills.

Educator participants demonstrated significant improvement (with a very large effect size) in both critical skills and total skills as well as significant improvement in their integration of each prototype skill. After using SkillFlix, both school and community educators’ self-perceived skill and comfort levels teaching this particular content also increased significantly. Educator’s noted, in particular, that by learning and practicing critical microskills, they felt more adept at both answering sensitive questions as well as employing inclusive language that is responsive to LGBTQ students’ sexual health education needs. This finding bodes well for the possibility that SkillFlix can support the countering of heteronormative frameworks in sexual health education.

Traditional in-person methods of skill instruction normally include several cognitive and behavioral strategies, including skill practice with observational feedback (Joyce et al., 2004). This classic approach to skill acquisition is also supported by social learning theory (Bandura, 1977). Thus, in this study, where participants were not provided a forum for practicing skills before the second (post-test) class session, staff expected more modest results. Staff were surprised and pleased to learn that this Microskills video training resource could significantly improve educator skills with remote delivery, in a short timeframe, and with no opportunity for practice combined with observational feedback.

The pilot study generated several other interesting findings. First, in the authors’ experience, educators often seek training primarily to improve their content knowledge. That was true for this study as well, where many of the users were motivated by the desire to learn content. Yet at the same time, viewing the videos also significantly improved the users’ skills. This improvement suggests that, while users may have consciously employed a content orientation to the material, they subconsciously developed their skills. Therefore, creating learning applications that bundle relevant content with skills training materials and strategies holds significant promise.

Second, the authors were pleased that approximately 80% of users found the dual video approach useful. This approach—where each Microskill was modeled twice and in slightly different ways for reinforcement—was used to better reflect the varying needs of sexual health educators across the United States, including different educational venues, a wide range of settings, and diverse SRH approaches. While we had concerns the dual videos might be seen as repetitive, in fact, educators in the pilot study mentioned that they could have achieved more skill improvement if they had watched the videos additional times. This type of skill acquisition support—support that is readily available, when it is particularly needed—is consistent with distributed learning theory (Knowles, 1975, 1984). SkillFlix could serve as a valuable form of ongoing implementation support for educators to use on an as-needed basis while teaching sexual health lessons.

There were a number of limitations to this pilot study worth noting. First, the use of selective sampling to identify the small number of local educators who participated could have yielded results that are not as generalizable as studies using a larger, randomized nationwide cohort of educators. Second, positive findings from the in-person evaluations with the educator participants may have been influenced by the need to please the interviewer, and/or the stipends they were paid (though the latter is a routine research practice).

Third, in the pilot study, one coder was a consultant and the other a dfusion staff member. While they were blind to both the pre- and post-conditions and the lesson order they were evaluating, there might have still been some unconscious bias at play. For the scope of the pilot study, this was our most practical option, but for our planned future full-scale rigorous evaluation of the application of a more robust product, we will be working with outside evaluators.

Fourth, our use of “student actors” raises some limitations. For reasons noted earlier, we used an experimental pilot design with actors portraying students in a recreated classroom, coached to use pre-determined prompts with the educators. These prompts were needed to ensure that the educators were exposed to the types of student questions and scenarios needed to evaluate the educators’ Skillflix training. The teachers did not know these students. The “class size” was smaller than in a normal classroom, and the actors ranged in age from 14 to 18, representing 9th–12th graders. This helped us to achieve a diverse group of student actors, but we acknowledge that in “real world” settings, educators would be teaching in classrooms with less age diversity. As such, we cannot guarantee similar results would be found in actual classroom settings with educator’s regular students. Also, given the pilot study design, we also did not collect data from “students” concerning their perceptions of educators and their lessons, but will do so in a more robust evaluation in a second phase study.

Fifth, we recognize that the large effects size found in our pilot study may not be generalizable to educators with more expertise in teaching sexual health education. Our focus in the pilot, however, was on those at a more novice level of teaching sexual health education. Finally, while we were interested in whether educator skill acquisition continued over time, this type of follow-up research was beyond the scope of this first phase pilot study.

Conclusion

Encouraged by these results, we are seeking more robust research and development funding to fully develop and evaluate the effectiveness of this Microskills video training library for educators who teach sexual health to youth aged 14–18. This effort would include development of approximately 500 new video clips covering an additional 20 Microskill sets, including Guiding Large Group Discussions, Leading Condom Demonstrations, Teaching About Cyber Safety, and Using Trauma-Informed Strategies. Design efforts will also incorporate feedback requests from our usability testing efforts, including (a) adding additional links for more SRH information; (b) minor reprogramming of navigational button locations and some user participation features in order to require less scrolling from the user. The second phase of our research will also involve adding an online peer-support learning community to the platform for educators offering sexual health education. The community will provide a unique opportunity for educators to upload their videos and receive an objective review, observation, and feedback options from their peers (see Table 7).

Table 7. SkillFlix features.

First phase features

Robust search by skills or content Watch history allows users to track progress Trending videos allow users to access universally most watched videos Video ratings and comments Favorites button for user-created list of favorite videos Written Microskill steps. The Microskills modeled in the video are also described in text below the video for increased comprehension Certificates of completion are created for each skill set. Post viewing printable skill snapshots of all Microskill steps within a skill set User account settings including photo and profile Help functions include a video tutorial and frequently asked questions (FAQs) Optimized for use on mobile devices

Additional features for Phase II

500+ short video clips modeling Microskills User assessment tool to identify a customized learning path that addresses the user’s unique skill-building needs Links to relevant topical information presented in videos (as requested by educators in Phase I) A learning community that includes optional participation in:

Teacher Submitted Skill Videos (vetted before posting), with observation checklists for feedback; and Learning Groups for online discourse eCommerce functionality to charge users for subscriptions to the site

As this form of training is program-agnostic, it could, therefore, be applied to educators teaching any number of evidence-based sexual health curricula as well as their own personally developed sexual health lessons. Finally, building on these promising pilot study findings, dfusion is also working to develop SkillFlix resources for use with diverse audiences, including parents of youth on the autism spectrum, parents of youth with cognitive disabilities, special education teachers, and pharmacists who provide information to customers in the context of non-prescription syringe sales.⁸ The content for these other applications is currently in different stages of development.

Acknowledgements

National training and SRH experts who advised on the project included Karin Coyle, PhD, Deborah Christopher, MSM, Jutta Dotterwiech, MPH, Lori A. Rolleri, MSW, MPH. We also thank the many teachers, sexual health educators and trainers who participated in the formative research and pilot evaluation. A special thanks to the actors and video production company, SecondPeak that brought the videos to life, as well as Katy Casselman for prop work and Erica Marsh for content management.

Notes

1 California Prevention Training Center. Sexual Health Educator Training Program. [Online]. https://californiaptc.com/sexual-health-educator-training-program/

2 https://www.etr.org/blog/new-professional-learning-standards-for-sex-education-plsse/

3 https://siecus.org/wp-content/uploads/2018/07/National-Sexuality-Education-Standards.pdf

4 https://siecus.org/wp-content/uploads/2019/04/PLSSE-FINAL.pdf

5 Expert advisors for this project were as follows: Jutta Dotterweich, Director of Training and Teaching Assistant, Cornell University ACT Center for Youth Excellence; Deborah Christopher, Director of Professional and Organizational Development, ETR; Lori Rolleri, Principle of Rolleri Consulting; and Karin Coyle, Chief Science Officer, ETR).

6 Drupal is content management software used for creating websites and other integrated digital frameworks. See: https://www.drupal.org/about

7 While having the developer pilot the product in a Phase I grant is customary given the scope and size of Phase I pilots, for our continued research with Phase II funding, we will have independent evaluators conduct the formal RCT, removing any potential conflict of interest.

8 We recently concluded a federally funded project in which pharmacy staff participated in SkillFlix video training with Microskills designed to improve their customer service and compliance with regulations when participating in non-prescription syringe sales. In a real-world evaluation of skills demonstrated during interaction with customers before and after the SkillFlix training, pharmacy staff significantly improved their total skill score (p = .001) and improved in nearly all individual skills.