Development of Long Live Love+, a school-based online sexual health programme for young adults. An intervention mapping approach

Abstract

This paper describes the development of a Dutch online programme called Long Live Love+ focusing on positive, coercion-free relationships, contraception use, and the prevention of STIs, using the Intervention Mapping (IM) approach. All six steps of the approach were followed. Step 1 confirmed the need for a sexual health programme targeting young people aged 15 and over enrolled in higher level secondary education. Step 2 resulted in the production of a series of matrices-of-changes, including detailed programme objectives at the behavioural and the psycho-social level. Step 3 involved the selection of relevant methods and applications. Step 4 consisted of programme development, resulting in a sexual health programme with online and offline components, and including interactive exercises. Step 5 focused on adoption and implementation and included the production of a detailed teacher manual. Step 6 involved detailed planning for the process and effect evaluation and included interviews with teachers and focus group discussions with students to evaluate their experiences of the programme. The inclusion of a linkage group – and especially the inclusion of teachers in the development of the programme – turned out to be essential in terms of developing a programme in line with their context and needs.

Keywords:

• Schools
• comprehensive sexuality education
• intervention mapping
• pregnancy prevention
• STI/HIV prevention
• sexual harassment prevention
• The Netherlands

Introduction

Although The Netherlands is known for the relatively good sexual health of its youth (Ferguson, Vanwesenbeeck, and Knijn 2008; Sedgh et al. 2015; Weaver, Smith, and Kippax 2005), there is still room for improvement. A recent Dutch study focusing on pregnancy experiences and contraception use among a representative sample of the Dutch population showed that 7.4% of the girls aged 15–19 indicated to have ever experienced a pregnancy that was not intended (Picavet 2012) and that Chlamydia rates in particular are high among young adults; 60% of all Chlamydia infections are reported among young people aged below 25 years (Van Oeffelen et al. 2015). In addition, in one study, almost 40% of girls aged 15–17 years reported being subjected to sexual-related behaviours (e.g. comments, touching) against her will (Dukers-Muijrers et al. 2015). Moreover, 1 out of 20 young adults reported that they could never be friends with a homosexual peer (Kuyper 2015).

Over the years, multiple Dutch sex education programmes, covering one or more of the above-described sexual health-related issues, have been developed. Among those, Long Live Love (LLL) is one of the most widely used school-based sex education programmes for young people aged 13–14, and has been proven to be effective in positively influencing students’ attitudes, beliefs and intentions regarding condom use and safer sex (Schaalma et al. 1996). Based on an observed need to revise and update the original programme, as well as the need for sex education materials targeting older youth aged 15–20 years (De Graaf et al. 2012), the original LLL programme has been revised and expanded. Since 2012, therefore, there have been three versions of LLL available: the original, revised LLL targeting those aged 13–14 and two additional (online) versions for youth 15+; the first for students enrolled in lower level vocational education (called Long Live Love – MBO¹) and the other for students enrolled in higher level general secondary education (called Long Live Love – BB) in this paper referred to as LLL+. The content of the three different versions of LLL is targetted to different levels of sexual experience (less for 13–14-year olds and students enrolled in higher level education as compared to the 15+ students enrolled in lower level vocational education) and the literacy levels of the students (higher for high level education 15+ students). In addition, the delivery of the LLL versions is slightly different to comply with demands of teachers and the different school/class contexts (i.e. LLL+ is fully online delivered while LLL for 13–14-year olds also has offline materials that need to be ordered via the website).

All three versions of LLL were developed in line with the Intervention Mapping protocol (Bartholomew et al. 2016). Intervention Mapping (IM) provides a systematic framework for the development and evaluation of theory- and evidence-based interventions. According to the protocol, six steps should be followed: an initial needs assessment (step 1), the formulation of programme goals and objectives (step 2), the selection of theory-based methods and applications (step 3), programme development (step 4), the development of an adoption and implementation plan (step 5) and the development of an evaluation plan (step 6). Each step includes several core processes, for example, brainstorming about potential answers and explanations before searching the literature for evidence. The results, or data, gathered in each step provide input for the next steps. For a more extensive description of the IM protocol, see Kok (2014) or Bartholomew et al. (2016).

Each LLL version addresses the same programme goals, namely that: (1) students have healthy, happy (sexual) relationships, free of coercion; (2) students prevent pregnancy and (3) STIs and that (4) students are not prejudiced towards sexual minorities. However, the programmes differ with regard to the specific performance – and change objectives that are included (step 2 of IM), and the methods used (step 3 of IM), so as to address the different needs of the target audience due to age, sexual experience and educational level. In line with increasing calls among scientists and health workers for more detailed programme descriptions to enhance replicability (Peters, De Bruin, and Crutzen 2015; Sugg 2016), this paper provides a detailed description of the development and evaluation plan of one of the three LLL versions, namely LLL+ for students aged 15+ enrolled in higher level general secondary education.

All six steps of IM as described above are described. In addition, we hope that a precise and systematic description may guide and enhance the development of future (online) programmes in the field of sexual health (Sugg 2016). The paper focuses on the first three programme goals of LLL+ (i.e. healthy relationships and the prevention of pregnancies and STIs). The development of the programme component focusing on the fourth goal (the reduction of prejudice towards sexual minorities) is described elsewhere (Mevissen et al., 2017).

Methods

We followed the six steps of the Intervention Mapping approach in order to systematically develop and evaluate a theory- and evidence-based online sexual health programme for students aged 15+ in general secondary education (in Dutch, HAVO/VWO).

Step 1: needs assessment

The needs assessment focuses on gaining a clear understanding of the health problems, factors underlying these problems and the needs of those the programme or intervention aims to meet. A multidisciplinary linkage group should be formed that includes potential programme participants and implementers. Data are collected by means of a brainstorming session with members of the linkage group and a review of the literature in order to identify the specific needs of the target group and the capacity of the community to respond to these needs.

Step 2: programme goals, performance objectives and matrices of change objectives

In step 2, it is specified who and what will change as a result of the intervention. Programme goals (i.e. expected programme outcomes for health-related behaviour) are formulated. These programme goals are then further differentiated into objectives at the sub-behavioural level (performance objectives, or POs). For each PO, determinants are selected that are significant and changeable. Matrices are developed for each programme goal by linking performance objectives with determinants to identify the changes needed at the social-cognitive and affective/emotional levels (change objectives, or COs). Objectives should be formulated using the SMART (Specific, Measurable, Attainable, Realistic, Timely) mnemonic and are guided by the brainstorm and the literature review as well as by the results of step 1.

Step 3: selection of theory-based methods and applications

Next, theory-based methods are selected from the literature and translated into practical applications that will work within the intervention context to meet the objectives as formulated in step 2. Currently available programme materials can be reviewed for potential use within the programme and, if necessary, adjustments are made to ensure that the application is in line with the programme objectives as well as with the theoretical parameters for use (Peters, De Bruin, and Crutzen 2015). The selection of methods and applications is discussed with members of the linkage group.

Step 4: programme development and production

The methods and applications selected in step 3 are combined together into the final programme. Programme themes, topics, scope and sequence are created together with programme materials. Members of the linkage group are asked for feedback on the final programme structure. Preferably, programme components should be pre-tested with members of the target group.

Step 5: adoption and implementation plan

Potential issues concerning the adoption and implementation of the programme should be considered during all steps of programme development. Expected adopters and implementers of the programme are consulted at each step. An adoption and implementation plan should be developed with the same steps as described above: programme use outcomes and objectives (POs and COs) for programme adoption, implementation and sustainability should be formulated; methods and applications selected; and an intervention to promote programme adoption, implementation and sustainability is developed.

Step 6: programme evaluation

To evaluate the effectiveness of the programme in changing the determinants of the selected health behaviours, an evaluation plan is developed. The evaluation plan should include a process evaluation focusing on (factors influencing) the level of programme implementation as well as the experiences of the implementers and the target group. The development of the effect evaluation plan is guided by the objectives formulated in step 2; steps 3 and 5 guide the development of the process evaluation plan.

Findings

Needs assessment

The needs assessment focused on gaining a clear understanding of reproductive and sexuality-related health problems, factors underlying these problems, and the needs of the students. A multidisciplinary linkage group was formed which included experts on young people’s sexual health (four people, including the PI), experts in Intervention Mapping (three people), a social worker, two public health workers and 14 teachers. We specifically aimed at not only including biology teachers but also the teachers of social studies.² The linkage group also included the developers of the two other versions of LLL, in order to ensure that all three versions of LLL would be geared towards one another.

The teachers in the linkage group (12 biology teachers, 2 teachers of social studies) clearly identified a need for sex education materials for young people aged 15+ enrolled in higher level secondary education. In The Netherlands, most young people receive their first sex education in school at around 12–14 years of age. However, in general, higher educated young people in the Netherlands become sexually active at a later age as compared to lower educated youth (De Graaf et al. 2012). This finding is in line with the perceptions of teachers involved in the linkage group. Because of this, it is important to provide sex education not only when students are young and before they become sexually active, but also at a later stage when it becomes most relevant (Kreuter and Wray 2003). Research has also shown that message-repetition can enhance the impact of the message (Keller and Lehmann 2008). Both of these findings support the need for targeting students with sex health education programmes multiple times and at different ages. In addition, teachers in the linkage group expressed the need for a programme which includes activities that are more in line with the intellectual capacities of higher educated students. Finally, all teachers agreed on implementing the programme in year 4 (i.e. with students age 15–16) and not in the final examination years (year 5 for HAVO students and year 6 for VWO students) because at this latter time both students and teachers would be too busy for work with additional health programmes.

The teachers unanimously expressed their doubts about our idea of creating a programme that would be delivered entirely online and based on a problem-based learning (PBL) approach. Teachers expected that students would need training before being able to work according to PBL, and they thought that such training would be too time-consuming. Teachers were very positive, however, about the idea of developing an online programme. However it was stressed that this should not fully replace real-life interactions between students, and between students and teachers. Teachers highly valued direct communication with their students, especially in relation to topics such as sexuality, as ‘sometimes you just need the entire class, to have the interaction’. Moreover, teachers reported large differences in computer and Internet (i.e. Information and Communication Technology, ICT) experience, ICT-related self-efficacy and skills and feasibility of ICT use (in terms of implementing the program) between schools and between teachers. These observations are backed up by Dutch reports on ICT use in (higher) education (Giling and Van Der Laan 2005; Stichting Kennisnet 2015; Van Rooij and Van Den Eijnden 2007), as well as by findings from an online survey among high school teachers on the determinants of using ICT in education carried out by the PI and a master thesis student (Van Schoonhoven 2012). Based on these comments and findings, we decided to abandon the PBL approach as well as the idea of a fully online delivered programme and instead focus on a flexible programme that would include online as well as offline components.

The literature review we carried out showed that among higher educated students in the Netherlands, as compared to their lower educated peers, a lower number of sex partners, a higher age at first sex, a lower prevalence of teenage pregnancy and less experience with sexual harassment are reported (De Graaf, Vanwesenbeeck, and Meijer 2014; De Graaf et al. 2005, 2012; Kuyper et al. 2013). However, the fact that such students have better sexual health outcomes in general does not mean that harassment, teenage pregnancy and STIs are completely absent; e.g. 9% of the higher educated girls in one study reported that they had been persuaded into sex against their will, and this rate increases to 13% with age (De Graaf et al. 2005). Other studies have shown that up to 53% of girls, regardless of educational level, have experienced some kind of non-physical form of sexual harassment (Timmerman 2005). In addition, self-indicated suffering from sexual problems such as not being in the mood for sex, lack of arousal and difficulties achieving orgasm are more frequently reported by higher educated girls as compared to lower educated girls (De Graaf et al. 2005). Moreover, lower rates of always using condoms with the most recent sex partner have been reported among higher educated boys (29%) as compared to lower educated boys (40%) while perceptions on the kind of relationship had with the last sex partner did not differ substantially; 65% of higher educated boys and 61% of lower educated boys evaluated the most recent relationship as ‘stable and monogamous’ (De Graaf et al. 2012). In addition, higher educated girls reported lower rates of STI-testing as compared to lower educated girls. STI rates do not seem to differ between educational levels (De Graaf et al. 2005). On the other hand, contraception use rates (including hormonal contraception) with the most recent partner are higher among higher educated girls (88%) and boys (84%) as compared to lower educated girls (77%) and boys (74%).

Studies have further shown that persuasion is more accepted from a regular partner (‘it’s part of the deal’) as opposed to casual partners (Kuyper et al. 2013). Relationship quality (both positive and negative aspects) has been found to be related to unprotected sex (Brady, Gruber, and Wolfson 2016). Positive sexual experiences are related to intimacy, trust, being at ease, taking initiative and having control over the situation (Cense and Van Dijk 2010). Risky, unpleasant situations are related to unclear (non-verbal) communication, not recognising the other person’s boundaries and taking a passive stance (Kuyper et al. 2013). Factors related to not using condoms include perceiving a relationship as serious and trusting one’s partner, negative attitudes towards condom use (e.g. ‘it disrupts a sexual encounter’ and ‘less pleasurable sex’), low risk perceptions, and being unprepared (De Graaf et al. 2005, 2012; Fortenberry et al. 2002). Reasons for not getting tested for STIs/HIV include low risk perceptions and lack of awareness (‘I don’t have any symptoms’), negative attitudes towards testing location and/or procedure, fear (of the testing procedure or testing outcome) and perceived STI-related stigma (Blake et al. 2003; Cunningham et al. 2009; De Graaf et al. 2005; Pavlin et al. 2006; Shepherd and Harwood 2016). The majority of young women and young adult girls in the Netherlands, regardless of educational background, take the pill as a form of contraception (De Graaf et al. 2005, 2012). However, the contraceptive pill has several disadvantages: it is (far) less effective for protection against pregnancy when adherence is not strictly followed or after vomiting. Reasons for not (consistently) using contraception include forgetting to take the pill regularly, a lack of awareness with regard to the importance of adherence, worries regarding potential side effects, low risk perception and lack of skills in organising adherence (Kirby 2002; Trussell and Wynn 2008; Xu and Shtarkshall 2004).

All the information gathered in step 1 was summarised and informed the following IM steps.

Programme goals, performance objectives and matrices of change objectives

Guided by the observed needs of the target group as established and described in step 1, the PI formulated three³ programme goals which were approved by the members of the linkage group: higher educated students aged 15+(1) have happy and healthy (sexual) relationships free of coercion; (2) prevent pregnancies; and (3) prevent STIs/HIV. These programme goals were then each subdivided into performance objectives (POs) – related to and expected to support the programme goals. For example, with regard to the programme goal ‘students have happy and healthy (sexual) relationships’, POs were formulated as ‘students clearly communicate their own sexual/relational wishes & boundaries’ or ‘students respect the sexual/relationship wishes and boundaries of their partner’ (for an overview of all POs see Table 1). Support for the POs formulated was found in the literature, and the list of POs was discussed and fine-tuned with members of the linkage group until consensus was reached. The PI then held a brainstorming session with members of the linkage group to ascertain determinants for each of the POs. Additionally, a review of the literature was carried out to select empirically and theoretically supported determinants of each PO. This resulted in a list of determinants for each PO. The list was fine-tuned taking into account the importance and changeability of each determinant-PO combination (Peters, De Bruin, and Crutzen 2015). Determinants that turned out not to be that strongly related to the PO and/or were (too) difficult to change were omitted. For example, social norms were expected to influence several POs but literature shows that social norms are relatively difficult to change (Mollen, Ruiter, and Kok 2010).

Table 1. Examples of performance objectives for each of the three modules (relationships, (un)safe sex and pregnancy and (un)safe sex and STIs) in the sexual health programme for students aged 15+ in higher level secondary education.

Module	Performance objectives (summary)
Relationships	R.1 Formulate their own relationship-related as well as their sexual wishes & boundaries
	R.2 Clearly communicate their own sexual/relationship wishes & boundaries
	R.3 Respect sexual/relationship wishes & boundaries of partner
(Un)safe sex and pregnancy	P.1 Anticipate having sex at some point
	P.2 Decide on using contraception to prevent pregnancy when (becoming) sexually active
	P.3 Buy/organise contraception
	P.4 Use contraception correctly and consistently
(Un)safe sex and STIs	S.1 Decide to use condoms to prevent STIs when having sex
	S.2 Buy condoms
	S.3 Have condoms available
	S.4 Use condoms correctly and consistently for as long as own and partners’ STI status is unknown
	S.5 Go for STI testing and communicate STI test results with sexual partner when necessary:
	(1) After sex with a partner of unknown STI status (2) Before quitting condom use in a steady relationship

Subsequently, matrices were created in order to formulate specific change objectives (COs) for each determinant-PO combination. For example, for the PO ‘students clearly communicate their own sexual/relational wishes and boundaries’, one of the selected determinants was ‘knowledge’. In terms of behavioural change, knowledge is not usually the most important determinant; however, knowledge is easy to change (Bartholomew et al. 2016). The CO formulated for this determinant-PO combination was ‘students differentiate between clear and unclear (as well as direct and indirect) ways of communicating wishes and boundaries’. In the end, three matrices were created; one for each programme goal. The matrices were discussed and fine-tuned with members of the linkage group. The content of these matrices (i.e. all the COs) guided the final content of the intervention (step 3 and 4). See Table 2 for a summary of the matrices.⁴

Table 2. Examples of performance objectives (PO), determinants, and change objectives (CO) from the matrices for the three main modules in the sexual health program (relationships, (un)safe sex and pregnancy, and (un)safe sex and STIs).

Module & PO	Determinants and COs
	Knowledge & Awareness	Attitude	Affective (emotions)	Self efficacy and skills	Norm
Relationships R.2 Clearly communicate their own sexual/relationship wishes and boundaries	Mention differences and examples of clear and unclear – as well as direct and indirect – communication of wishes and boundaries	Acknowledge the importance of the correct timing of communication of wishes and boundaries	Acknowledge emotions that can be involved when communicating wishes and boundaries	Show how one can deal with emotions being involved with communication of wishes and boundaries	Feel supported by important others in communicating sexual wishes and boundaries
(Un)safe sex and pregnancy P.4 Use contraception^a correctly and consistently	Describe important factors involved in the correct and consistent use of different types of contraception (adherence, timing, etc.)	Express positive attitudes towards the correct and consistent use of contraception in addition to condoms for pregnancy prevention (‘double Dutch’)	Mention positive emotions involved in having the security of correct and consistent contraception use	Mention ways to use contraception correctly and consistently (e.g. reminders for timing)	Boys acknowledge their responsibility in correct and consistent contraception use
(Un)safe sex and STIs S.5 Go for STI testing and communicate STI testing results with sexual partner when necessary	Mention when it is necessary to go for STI testing	Acknowledge the importance of early testing and treatment	Mention emotions that may be part of going for STI testing and discussing it with partner	Show how one can deal with emotions involved in STI testing and discussing it with partner	Acknowledge that STI testing is not related to distrust but to taking responsibility for each others health
	Describe the STI testing procedure

^a In this paper, we differentiate between condoms and other forms of contraception by using the term contraception for all medical forms of contraception except condoms.

Selection of theory-based methods and applications

Now that the objectives of the programme had been clearly formulated up to the social-cognitive and affective/emotional (determinants) level in step 2, the developers could brainstorm and select theory- and evidence-based methods for changing the determinants. Methods were selected following a review of the literature and based on their fit with the intervention context. The goal now was to translate these methods into applications that can be used online – but that also work within a school- and classroom-based context and are in line with the preferences of teachers (see step 1). Thus, online as well as offline practical applications were identified, first by means of a brainstorming session, and then by looking for examples in the literature. At the same time, available sex education materials were reviewed for their potential applicability by evaluating whether their objectives, methods and applications matched with those of our programme. We carefully considered the conditions for use (i.e. parameters for use i.e. moderators of the effectiveness of the method) for all selected methods and applications (Kok et al. 2016).

For example, the CO ‘students recognise situations that can result in crossing their (or their partners’) sexual boundaries’ relates to the determinant awareness, which belongs to the PO ‘students clearly communicate their own sexual/relational wishes and boundaries’. A theory- and evidence-based method for changing awareness is ‘scenario-based risk information’ (Bartholomew et al. 2016; Mevissen, Ruiter, et al. 2010). Two important conditions for use that must be taken into account when considering this method are the plausibility and imaginability of the scenario. It is also worth bearing in mind that the method is most effective when people generate their own scenario, or when multiple scenarios are provided (Mevissen, Ruiter, et al. 2010; Mevissen et al. 2012). The practical application that was developed therefore included two scenario-stories describing two different situations in which sexual boundaries are crossed. The application included a short list of questions to encourage students to elaborate on the scenarios and generate and imagine their own scenarios. The stories and the questions were designed to be downloadable from the intervention website. A final list of potential methods and applications to be included in the programme was discussed with members of the linkage group. See Table 3 for more examples of methods and applications linked to determinants and change objectives included in the programme.

Table 3. Overview of the Sexual Health program including scope and sequencing related to performance objectives (PO) and change objectives (CO), methods, parameters, applications and materials.

Module	Assignment	PO, determinants & CO (examples)	Methods	Parameters	Applications	Materials
Relationships	1. Introduction to relationships	PO: R1. Formulate their own relationship-related as well as their sexual wishes & boundaries	Active Learning (all determinants)	Needs time, information and skills.	All students write down on sticky notes three aspects they like or appreciate in (sexual) relationships. The teacher collects all the notes and sticks them to the backboard. The notes of boys are placed on one side, those of girls on the other side. Based on the sticky notes, the following statement will be discussed by the students: 1) Girls appreciate ‘being together’ as the most important aspect of a relationship, for boys it’s mainly the sex.	Blackboard and sticky notes (via school/teacher).
			Discussion (knowledge)	Listening to the learner to ensure that the correct schemas are activated.
						Teacher manual with instructions and background information to guide the discussion (available via the LLL website).
		Determinants: Knowledge, awareness, attitude, empathy, norms
			Elaboration (knowledge, attitude)	Individuals with high motivation and cognitive ability; messages that are personally relevant, surprising, repeated, self-pacing, not distracting, easily understandable.
		CO: Mention differences and similarities between men and women regarding (expressions of) wishes and boundaries in (sexual) relationships
			Shifting perspective (attitude)	Initiation from the perspective of the learner; needs imaginary competence
					Students then write down on sticky notes what kind of information or ideas about sex they can collect from the internet or media. Again, this will be followed by a statement to be discussed: 2) From the internet you can learn how to have nice and good sex
					The teacher will then guide the (group) discussion. In the majority of cases, it will show that boys and girls are not that different from each other. In addition, they will conclude that both ‘safe’ as well as ‘unsafe’ information can be found on internet. The teacher will provide the students with a list of trustworthy websites.
	2. Desires	PO: R1. Formulate their own relationship-related as well as their sexual wishes & boundaries	Persuasive communication (all determinants)	Messages need to be relevant and not too discrepant from beliefs of the individual. Can be stimulated by surprise and repetition. Will include arguments.	An (online) video shows - by way of a cartoon-like graph - how the sexual arousal curve differs for boys vs. girls but also between boys/girls. The cartoon alternates with clips of a nurse further explaining arousal as well as factors influencing arousal such as stress. Different graphs are shown to visualize and clarify that several factors are involved in sexual arousal.	(Online) video delivered via the LLL website.
			Elaboration (knowledge, attitude)
		Determinants: Knowledge, awareness, attitude, norms
		CO: Mention that desire and sexual arousal differs between genders and people.
				See above
	3. What do you want?	PO: R.2 Clearly communicate their own sexual/relationship wishes & boundaries	Scenario-based risk information (awareness)	Plausible scenario with a cause and an outcome. Should be imaginable. Most effective when people generate their own scenario or when multiple scenarios are provided.	Two scenarios describing different situations about ‘crossing sexual boundaries’ are available for students to read and discuss in small groups or in a class setting.	Two stories on paper with questions, downloadable via the LLL website.
			Imagery (knowledge)
		Determinants: Knowledge, awareness, attitude, empathy, self-efficacy, skills, norms
						Teacher manual with instructions and background information to guide the discussion, downloadable via LLL website.
			Arguments (attitude)
					One scenario describes a young couple who do not communicate clearly. The boy tries to become more intimate than the girl would like - but the girl is not giving a clear message and doesn’t dare to say no, and the boy is not checking with her. In the end, the girl has sex against her will and both are left feeling unhappy.
				Familiar physical or verbal images as analogies to a less familiar process
			Planning coping responses (self-efficacy, skills)
				For central processing of arguments, they need to be new to the message receiver.
			Empathy training (empathy)
			Active learning (all determinants)
				Identification of high-risk situations and practice of coping responseBeing able and willing to identify with the person. Imagine how the other person would feel.
					In the second scenario, a boy and a girl are chatting via webcam. They are schoolmates. The girl convinces the boy to show his ‘six pack’. The boy doesn’t feel comfortable to ‘strip’ for the webcam but gives in. He then hears giggling and it turns out the girls’ friends are also watching.
				See above
					After reading the stories, students discuss items like ‘how would both people feel?’, ‘what went wrong?’, ‘how could the boy/girl have acted differently to avoid this?’ and ‘what if the situation was reversed (boy asking girl to undress or girl wanting more intimacy than boy?).
				See above
				See above
			Discussion (knowledge)
			Elaboration (knowledge, attitude)
		CO: Mention differences and examples of clear and unclear - as well as direct and indirect - communication of wishes and boundaries
(Un)safe sex and contraception	1 – 4: Reproductive body parts, menstrual cycle and conception	PO: P.1 Anticipate having sex at some point	Chunking (knowledge)	Labels or acronyms are assigned to material to aid memory	Short video animations show schematic representations of the male and the female sexual body parts, the different components and their names and how they function. A third and fourth video animation explains the menstrual cycle and conception. Information includes hygienic advice, facts and myths regarding the hymen, and stresses that shapes and looks of reproductive body parts can be very diverse.	Video animations delivered via the LLL website.
			Advanced organizers (knowledge)
				Schematic representations of the content.
				See above
			Persuasive communication (all determinants)
		Determinants:Knowledge, awareness, attitudes
		CO: Mention the male and female reproductive body parts and their functions
	5. Safe vs. unsafe sex	PO:P.2 Decide on using contraception to prevent pregnancy when (becoming) sexually active	Consciousness raising (Awareness)	Can use feedback and confrontation but must be quickly followed by increase in problem-solving ability and self-efficacy.	Different sexual behaviors (kissing, touching, penetration etc.) are discussed in light of what is (un)safe for pregnancy vs. STI prevention.	A matrix will be filled in using blackboard.
						Teacher manual with instructions and background information to guide the discussion, downloadable via the LLL website.
				See above
			Active learning (all determinants) Discussion (knowledge)
				See above
				See above
			Elaboration (knowledge)
		Determinants:Knowledge, awareness, attitude
		CO: Express positive attitudes regarding the use of contraception other than and next to condom use (referred to as ‘double Dutch’).
	6. Contraception	PO: P.4 Use contraception correctly and consistently	Self-reevaluation (attitude, afffect)	Can use feedback and confrontation. However, raising awareness must be quickly followed by an increase in problem-solving ability and self-efficacy.	Students (in pairs, groups or individually) collect information on one or more types of contraception and focus on how it works, but also on what the (dis)advantages of certain types are, how to deal with disadvantages, how to get and how to use the different types, and how to deal with potential emotions (e.g. embarrassment) involved in discussing and organizing contraception use.	Leaflets with information focusing on different types of contraception, downloadable from the LLL website.
			Guided practice (Self-efficacy, skills)
		Determinants: Knowledge, awareness, attitudes, affect, self-efficacy, skills. CO: Mention ways to use contraception correctly and consistently (e.g. reminders for timing)
						Teacher manual with instructions and background information to guide the discussion, downloadable via the LLL website. Manual includes links to additional information provided on other websites as well as information about the option to order a ‘contraception suitcase’; a small plastic suitcase that contains real examples of the different types of contraception.
			Active learning (See above)
				Subskill demonstration, instruction, and enactment with individual feedback. Requires supervision from an experienced person.
			Discussion (See above)
			Elaboration (See above)
					At the end of the session, the teacher will discuss and compare the different types together with the students and show them a website where students (girls) can get tailored advice on which type of contraception may be best for them personally.
				See above
			Arguments (See above)
				See above
				See above
				See above
	7. Hormonal contraception	PO: P.4 Use contraception correctly and consistently	Chunking (See above)	See above	Short video animation that shows how hormonal contraception works in preventing conception.	Video animations delivered via the LLL website.
				See above
			Advanced organizers (See above)
		Determinant: Knowledge
		CO: Describe how hormonal contraception prevents pregnancy
(Un)safe sex and STI	1. Quiz	PO: S.1 Decide to use condoms to prevent STIs when having sex	Reinforcement (all determinants)	Reinforcement needs to be tailored to the individual, to follow the behavior in time, and to be seen as a consequence of the behavior.	Online quiz on STIs and (un)safe sex including 10 questions with multiple choice answer options. After answering each question, the student will receive tailored advice depending on the answer selected. More general advice is given tailored to the total score for the quiz. Good answers and scores are praised.	Online quiz is accessible via a link on the LLL website.
			Feedback (all determinants)
		Determinants: Knowledge, awareness CO: Mention that STIs often do not have symptoms
				Feedback needs to be individual, follow the behavior in time, and be specific.
			Active learning (See above)
				See above
	2. STIs	PO: S.5 Go for STI testing and communicate STI test results with sexual partner when necessary	Active learning (See above)	See above	Students (in pairs, groups or individually) collect information on one or more types of STI. They answer questions about what it is, potential symptoms, consequences, and how to cope with negative consequences.	Suggestions for websites with reliable information on STIs are provided in the teacher manual. In addition, the LLL website includes an online dictionary regarding words related to sex and sexuality.
					After all information is collected, the teacher will summarize the results with the class.
						Power point presentations that can be used by teachers for the post-discussion are provided via the teacher environment on the LLL website.
				See above
			Advanced organizers (See above)
				See above
			Guided practice (See above)
		Determinants: Knowledge, awareness, risk perception, attitude, affect, self-efficacy, skills
		CO: Acknowledge the importance of early diagnoses and treatment of STI.
	3. Chlamydia	PO: S.5 Go for STI testing and communicate STI test results with sexual partner when necessary	Chunking (See above)	See above	Short video animation shows how chlamydia infects your body and influences your reproductive system.	Video animations delivered via the LLL website.
				See above
			Advanced organizers (See above)
		Determinants: Knowledge, awareness
		CO: Mention which symptoms are related to STI.
	4. STI testing	PO: S.5 Go for STI testing and communicate STI test results with sexual partner when necessary	Modeling (all determinants)	Attention, remembrance, self-efficacy, and skills, reinforcement of model, identification with model, coping model instead of mastery model.	Video shows a young couple chatting. The boy would like to stop using condoms and comes up with reasons that are often used as to why it would be safe. The girl proposes to go for an STI test before quitting condom use, which results in a bit of a discussion about the relevance of STI-testing (including standard excuses that are tackled by the girl). In the end, the boy agrees, and both visit the STI clinic.	Video is delivered via the LLL website. A leaflet including the questions for the post-discussion (one version with and one version without potential answers included) is also provided via the LLL website.
			Active learning (See above)
		Determinants: Knowledge, awareness, attitude, affect, self-efficacy, skills, normCO: Show how one can deal with emotions involved in STI testing and discussing it with partner
			Discussion (See above)
				See above
				See above
			Guided practice (See above)
				See above
						Teacher manual with instructions and background information to guide the discussion, downloadable via the LLL website.
				See above
			Planning coping response (See above)
					The video shows the type of conversation they have with the nurses, the questions asked, and the tests carried out (urine test for the boy, vaginal swap for the girl, and blood-tests for both). A few days later, they receive the results. The girl is fine but the boy has tested positive for Chlamydia and receives treatment for this. They are happy they went for the test.
					After the video, the students receive several questions for a more in-depth discussion of topics such as how you would deal with a resistant partner.
	5. What’s your personal STI risk?	PO: S.1 Decide to use condoms to prevent STIs when having sex	Personalized risk communication (awareness)	Present messages as individual and undeniable.	Online survey on sexual history of the person. A safe sex recommendation will be provided tailored to the answers provided.	Online survey is accessible via a link on the LLL website.
				See above
			Feedback (See above)
		Determinants: Knowledge,
		awareness CO: Acknowledge their own susceptibility for getting infected with an STI

Programme development and production

The selection of theory-based methods and practical applications (step 3) coincided with the development and design of programme components, the pre-testing of components in the classrooms of teachers participating in the linkage group to check their feasibility and the fine-tuning of components for final production. The first step needed for programme production was the development of a programme website that would include all programme components, including a teacher manual. It was agreed that the website should provide both online and (downloadable) offline components in order to meet the diverse needs of teachers in terms of preferences and the technical support available. In addition, the website should enable the provision of videos, search tools and surveys.

The LLL programme previously developed for young people aged 13–14 years (Schaalma et al. 1996) is very well known among teachers in the Netherlands. It was therefore decided that just one LLL website would be developed that could provide teachers with clear, identifiable and easily distinguishable information and materials for all three versions of LLL. The website for LLL+ therefore had to include sections for the other two versions of LLL (LLL-OB and LLL-MBO). Finally, to ensure as much independence from ICT and website development companies as possible, we aimed to create a website based on a content management system (CMS).

The structure and sequence of the LLL+ programme was guided by the module topic, the type of assignment and the teachers’ time schedules: the selected activities should fit into the 40-minute teaching schedule of secondary schools and the total programme should be limited to a maximum of six teaching hours. Together with the members of the linkage group, it was decided that the programme would be organised in such a way that it begins with less sensitive and controversial topics and then moves to more sensitive matters. It was considered important to first create a safe atmosphere in the classroom, so that students will feel comfortable and safe enough to openly discuss the topic of sex and sexuality. Hence, the programme starts with the module ‘happy and healthy relationships’ after which the module ‘pregnancies and STIs’ is introduced.⁵

Due to the limited time available to teachers to provide sex education, the biggest challenge turned out to be the inclusion of as many COs and POs as possible within the final programme, while at the same time ensuring that there would be sufficient time for each programme component to be implemented by the teacher in a safe and comfortable context, without the need to rush. In the end, this meant that several COs and POs had to be excluded from the final programme. Our decision-making process with regard to which objectives to exclude was guided by theory and evidence regarding the importance and changeability of each objective. Moreover, our goal was to include a logical and comprehensive selection of topics in the final programme.

To design culturally relevant programme materials, both deep-structure and surface-structure factors were considered (Resnicow et al. 1999): The website includes pictures of young people from different ethnic backgrounds and mixed ethnic couples and the printable documents (work sheets and teacher manual) include pictures of youth that are multi-ethnic in appearance. In addition, ethnic-specific relevant topics such as the hymen and (female) genital mutilation are included in the modules and the teacher manual provides suggestions for discussing topics from different ethnic or religious perspectives.

Members of the linkage group were asked for feedback on the final structure of the programme. The final programme includes three modules, with a total of 15 assignments designed to be implemented in 6 teaching hours (2 hours per module). For an overview of all components included in the final programme, see Table 3. The programme is available via the Internet (see www.langlevedeliefde.nl – Bovenbouw section). The programme website includes a section for teachers (providing the teacher manual, additional programme materials and background information on (the optimal delivery of) the programme, see also step 5), and a section for students explaining the assignments and including programme materials such as videos and animations. For screenshots of the different webpages of LLL+ please refer to Figures A1–A6 in Appendix 1.

The programme is available free of charge as the costs of the development of programme materials (e.g. website, videos, PowerPoint and pdf documents) was covered by a grant from ZonMw. By delivering these materials via the Internet there are no additional postages of copy-costs for the developers. Using a content management system for developing the website also prevented ongoing hosting costs for the Long Live Love website for the programme developers. The only costs left for teachers are the costs of printing the teacher manuals and other pdf materials (work sheets etc.). The materials were designed for requiring a minimum of ink or toner when printed. We intentionally decided to deliver the program for free as widespread adoption and implementation of LLL+ was considered important.

Adoption and implementation plan

In order for programmes to be effective, they need to be adopted and implemented with completeness and fidelity (Durlak and DuPre 2008; Lendrum and Humphrey 2012). In the context of LLL+, the people adopting and implementing the programme were to be high school teachers. As of December 2012, and by law, all schools in the Netherlands are required to incorporate sex education into the curriculum. However, teachers themselves decide which teaching materials they want to use and how they will structure their lessons. They do not need approval from parents, school board members, school principals or other authorities. This generally means that teachers feel free to use a programme such as LLL+.

Teachers were consulted at each step. Our aim was to create ownership among them by closely involving them in the programme development. Although there were limited means (financially and time-wise) to really develop adoption and implementation matrices or an intervention as advised by IM, we did review the literature on factors influencing programme implementation and this, together with teachers’ feedback in each step, guided the development of a detailed adoption and implementation plan. The aim of this was to: (1) increase the likelihood of acceptance of the programme by closely involving a group of teachers in the decision-making process at each step of the developmental process; and (2) acquire feedback from teachers during the developmental process on the outcomes of each step in order to adjust the programme as much as possible to fit the context and needs of the teachers while at the same time keeping the content of the programme theory-based. By doing so, we hoped to reduce potential implementation barriers. We aimed also to (3) develop a teacher manual based on teacher feedback and comments on LLL+ and/or in relation to the school setting as well as the scientific literature (Paulussen, Kok, and Schaalma 1994; Schutte et al. 2014; Wiefferink et al. 2005); (4) include a process evaluation plan for gathering additional information on actual implementation barriers (see step 6); (5) use the information gathered in the process evaluation to further optimise the programme and teacher manual; and (6) develop a theory- and evidence-based information leaflet about LLL+ and participate in teacher conferences for further programme dissemination.

The needs of potential adopters and implementers of the LLL+ programme (i.e. teachers) were considered from the very first step of programme development. As indicated earlier, teachers were included in the linkage group and their reflections on the outcomes of each step were taken into account. In particular, teachers’ feedback was essential to the development of a programme that would suit their needs and requirements and we expected this, in turn, to enhance adoption and implementation.

The most important adjustments to the programme made, based on teachers’ feedback, were excluding problem-based learning, including not only online but also offline applications, and keeping the programme short (six lessons). The teachers’ manual provided clear guidelines on how to implement the different exercises. To increase implementation completeness, the manual included alternative suggestions for implementing an exercise. The aim of this was to provide as much flexibility as possible for the teacher to adjust a programme component to his/her specific classroom situation/context while at the same time maintaining fidelity. For example, suggestions were given for working individually, as part of a group, or as an entire class. Suggestions were also provided for students for preparing parts of an exercise at home. For most programme components, with the exception of the videos and animations, an offline as well as an online option was provided.

Alongside the development of the different versions of LLL (LLL-OB, LLL-MBO and LLL+), an e-coaching website was designed using the Intervention Mapping protocol, to support teachers in implementing LLL–OB (Schutte et al. 2016). The LLL+ teacher manual and the LLL+ teacher web-environment included links to the e-coach. Although the e-coach targeted teachers using the LLL-OB programme, we still expected that it could also provide relevant support for teachers working with the LLL+ programme.

Programme evaluation

To evaluate programme effectiveness in changing (determinants of) sexual health, as well as teachers’ experiences of programme implementation, we developed an extensive programme evaluation plan. To this end, we planned an RCT that would include an intervention group and a waiting-list control group. Effects on (determinants of) condom use, contraception use and clear communication of sexual wishes and boundaries would be evaluated in a pre- and post-test design. The survey was guided by the programme objectives as formulated in step 2. Additionally, we planned a process evaluation that would include interviews with all teachers who implemented the programme as well as focus group discussions with students from the intervention schools. The process evaluation plan focused on completeness and fidelity of programme implementation, factors influencing programme implementation, as well as student and teacher evaluations of the programme. In addition, Google analytics was recommended as a way to analyse actual website use and user behaviour (Crutzen, Roosjen, and Poelman 2013).

Discussion

In this paper, we have described the systematic development of a school-based sexual health programme called Long Live Love+ targeting young people enrolled in higher level secondary education in the Netherlands. The programme aimed for young people aged 15+ to (1) have healthy, happy (sexual) relationships, free of coercion; (2) prevent pregnancies and (3) prevent STIs. A fourth goal, that students ‘are not prejudiced towards sexual minorities’, was also included but is separately described in a different article (Mevissen et al., 2017). Long Live Love+ is delivered via the Internet and includes both online and offline exercises, as well as a detailed teacher manual with suggestions for alternative ways to implement an exercise. In line with previous findings (see e.g. Riley 2014), teachers in this study strongly stressed the need for a programme that would be flexible enough to adjust to different classroom circumstances. By providing a relatively flexible programme as requested, our aim was to stimulate implementation of all programme components with a high level of fidelity (Durlak and DuPre 2008). Methods are mostly interactive and include components such as active learning, discussion, argument and guided practice, in line with the recommendations of Albarracín et al. (2005) as well as Kirby, Laris, and Rolleri (2007).

The involvement of teachers in all steps of the developmental process turned out to be invaluable and often even more valuable than the involvement of young people themselves. Teachers were very positive and enthusiastic about the idea of having a newly developed sexual health programme for their students, had very valuable suggestions and ideas and did not show any reservation in delivering such a programme themselves. Students, in contrast, did not show very strong or clear opinions on what they liked or did not like or what they thought would be important. The only thing they were clear about was that they would like to watch movies or do quizzes. The PI’s impression was that students were not used to have an opinion about something they likely consider as ‘school material’; school is just school, the content is prescribed by teachers, it is just taken for granted and not something one could have an opinion about beyond liking or disliking a subject. This finding contrasts somewhat with the experience of other colleagues (see for example Jacquez, Vaughn, and Wagner 2013; Riley 2014) who reported the input of young people as being valuable and the attitudes of teachers as rather reserved when it came to delivering sex education. These different findings may be explained by differences between cultures and traditions.

Intervention Mapping turned out to be a valuable tool for designing the intervention and developing an implementation and evaluation plan. Although there are various other helpful suggestions available in the literature for developing (sexual) health interventions (for example, reviews that provide overviews of core components of effective sex education programmes such as those by Kirby and colleagues (Kirby, Laris, and Rolleri 2007) and the UNESCO guidelines (UNESCO 2009), the integrated framework proposed by DiClemente and Jackson (2014) and planning models like the precede-proceed model of Green and Kreuter (2005) or the Behavioural Change Wheel (Michie, van Stralen, and West 2011), none of these provided assistance on how to make theory- and evidence-based decisions concerning each step of intervention development, implementation and evaluation. In addition, IM helps provide a better understanding of the complexity of a behavior by delineating behaviours in terms of performance objectives and underlying change objectives. This in turn helps to define SMART intervention objectives (Kirby, Laris, and Rolleri 2007) and to ensure that important factors influencing the ultimate behavioural outcome are outlined.

Finally, IM clearly addresses how theory can be used at each step of the process and is particularly useful in providing guidance in deciding what methods to use and how to apply them correctly based on theoretical assumptions and evidence in order to achieve change. IM stresses the importance of considering the conditions-for-use of a method when designing the practical application. By using IM, there will likely be a clear, comprehensive and detailed programme description as well as a plan for implementation and evaluation that can easily be used or replicated by others. Using IM to develop interventions may be perceived as labour intensive; however, we feel that the effort is fully justified. Human behaviour is complex and changing human behaviour is even more. Simplifying the intervention development process by, for example, skipping steps or making decisions based on gut feelings may result in the more rapid development of a new programme, but at the cost of an increased likelihood for mistakes or having spent time and resources on ineffective interventions and in the worst case creating programmes with deleterious effects (Mevissen, Meertens, et al. 2010; Peters, Ruiter, and Kok 2013).

Limitations

A few limitations of this study are worth mentioning. First, IM stresses the importance of considering the environmental causes of the health problem (Bartholomew et al. 2016). In other words, a health problem may be more effectively solved when factors in the environment that influence the health problem are also the target of intervention. In the case of improving sexual health of youth, parents and teachers are important environmental factors. However, due to time constraints and financial limitations we had to limit our focus on the students only.

Second, our aim was to target teachers from several different backgrounds (i.e. not only those teaching biology, but also the teachers of social studies) in order to enable more widespread implementation. It turned out to be difficult however to find teachers of social studies willing to participate in the linkage group. This hesitancy may reflect these teachers’ limited intentions to implement the final programme as well. In addition, although sex education in the Netherlands is officially part of the school curriculum, teachers from orthodox religious schools (a small minority) may decide to only very limitedly address sex education and use their own materials to teach the topic. Additional research is recommended to explore the needs and willingness to teach sexuality education among both teachers of social studies and teachers from orthodox religious schools in order to reach a higher number of 15+ HAVO/VWO students. Unfortunately, due to a limited project budget, we could not perform such a study ourselves.

In line with findings of Cushman and colleagues (Cushman et al. 2014), time and especially financial constraints in our project had implications for our implementation plan. Although we managed to come up with various ways to (potentially) enhance widespread implementation with completeness and fidelity, and the study benefitted from a new law obliging all schools in the Netherlands to include sex education in the curriculum, we had neither the means to develop a detailed needs assessment study among teachers, nor to develop a teacher-specific training or coaching programme. In addition, although a small scale effect- and process evaluation of the try-out implementation of LLL+ has been planned, studying the implementation and sustained use of LLL+ (including monitoring the extent of online access and use) in real-life would provide valuable information but is not included in current plans.

As programme planners, we had to balance our goal of addressing as many objectives as possible on the one hand, with practical considerations such as teacher time constraints on the other hand. As a result, certain concessions had to be made, and this inevitably resulted in a programme that could not cover all the objectives formulated in step 2. Finally, it is important to note that LLL+ was designed to be implemented within the Dutch secondary school context. Future work could address how best to modify the programme for use in other countries and settings.

Conclusion

Our findings suggest that Intervention Mapping is a useful tool for the systematic development of a multi-component and multi-module school-based online sex education programme. The final programme is well-structured, with clear and measurable objectives. The programme is theory- and evidence-based, includes online as well as offline components and is in line with the needs of programme adopters and implementers. It includes a clear adoption and implementation plan as well as a concise plan for programme evaluation.

In addition, our study highlights the importance of working with a multidisciplinary group of people as members of the linkage group (see also Riley 2014). Experience shows that their feedback can enhance decision-making at each step. Finally, the inclusion of teachers as part of the linkage group also proved to be an essential part of programme success.

It is important to stress that LLL+ should not be implemented in contexts other than Dutch secondary schools without first exploring whether the programme fits local needs. In other words, adjustments may be needed, for example, regarding the specific content of the change objectives (young people in different settings may need different knowledge or skills or attitudes) and not all methods used in LLL+ may be suitable in other contexts. However, we hope that our extended description of the development of LLL+ may inspire other sexual health programme developers and provide them with the ideas and tools for developing programmes that fit the situation they work in.

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

This work was supported by ZonMw [grant number 124290002].

Acknowledgements

The paper is written in memory of the late Herman P. Schaalma, for his extensive contribution to (research in) youth sexual health. Professor Schaalma was the main applicant of the grant proposal that resulted in the funding for the development of LLL+. We would like to thank all teachers and students for their feedback and advice on LLL+.

Notes

1. MBO stands for Middelbaar Beroeps Onderwijs. MBO is equivalent to vocational training. BB stands for Bovenbouw and refers to HAVO (senior general secondary education) and VWO (university preparatory education or ‘pre-university training’). For an overview of the Dutch educational system see https://www.epnuffic.nl/en/publications/find-a-publication/education-system-the-netherlands.pdf

2. Traditionally, teachers of biology provide sexuality education in The Netherlands. However, biology is not a compulsory subject for 15+ HAVO/VWO students, while social studies (or its equivalent) is.

3. The actual programme covered four programme goals. In addition to the three goals described in this paper, a fourth goal was ‘Adolescents are supportive towards sexual minorities’ (i.e. acceptance of sexual diversity). For more information regarding the development of the programme-component covering the fourth goal, see Mevissen et al., 2017.

4. The full matrices are available at https://www.researchgate.net/project/Long-Live-Love-A-Dutch-School-Based-Online-Sexual-Health-Program-for-Adolescents-aged-15

5. The fourth module included in the programme is ‘sexual diversity’. This programme component is described elsewhere. See Mevissen et al., 2017.

Appendix 1

^{Figure A1}

Figure A1. Screenshot of the introductory page of the Long Live Love programme. The three different-coloured rectangular buttons at the bottom of the page (purple, green and blue) direct visitors to the version-specific web-environments, including both a section for students (‘leerlingen’) and a section for teachers (‘docenten’).

^{Figure A2}

Figure A2. Screenshot of the introductory page for the LLL+ students section. The four main themes can be selected from the menu on the left of the screen.

^{Figure A3}

Figure A3. Screenshot of the webpage for theme 3 of the LLL+ programme (students section). The four exercises related to this theme (are shown and) can be selected from the menu on the left.

^{Figure A4}

Figure A4. Screenshot of the webpage for exercise 4 (the STI clinic) of the LLL+ programme (students section).

^{Figure A5}

Figure A5. Screenshot of the introductory page for the LLL+ teacher section. The four main themes of the programme can be selected from the menu on the left, as well as a general introduction to the LLL+ programme, information for supporting implementation and background information on sexuality among students.

^{Figure A6}

Figure A6. Screenshot of the webpage for theme 2 of the LLL+ programme (teachers section).