Teen Well Check: an e-health prevention program for substance use, sexual assault, and sexual risk behaviors for adolescents in primary care

Figures & data

Figure 1. Study flow.

Table 1. Template for Intervention Description and Replication (TIDieR) checklist.

	Teen Well Check TIDierR checklist
Why:	A brief, technology-based integrated prevention for adolescents focused on substance use, sexual assault, and sexual risk in primary care settings has the potential to prevent related risk behaviours in a brief format that can be delivered as part of a wellness visit that does not add burden to primary care providers.
What (material):	See Table 2 for a description of intervention components and Figure 2 for examples of the content included in the technology-based intervention.
What (procedures):	Teen Well Check can be delivered to adolescents aged 14–18 visiting a primary care clinic and can be delivered in the waiting room, while waiting in a clinic room, or after the visit.
Who provided:	Teen well check is a self-led programme and does not require any particular training or person to deliver the programme. However, it was developed to be provided in the context of a primary care visit.
How (mode of delivery):	Teen Well Check is a technology-based programme that is delivered through a web-app. The web-app can be accessed via a tablet, smart phone, or computer.
Where:	Pediatric primary care clinics. If delivered in the waiting room or a clinic room, the clinics would need a tablet, smart phone, or computer that is connected to internet. If delivered after the primary care visit, the patient would need access to a tablet, smart phone, or computer that is connected to internet.
When and how much:	Teen Well Check is a single session programme. It takes approximately 5–15 min to complete.
Tailoring:	The current iteration of Teen Well Check is tailored by age, gender, and participant report of substance use. Participants are provided with age and gender tailored normative data throughout the intervention. Further, if participants report no substance use, they are able to choose which substance they want to learn about. If they report using one substance, they are given information about that substance. If they report using multiple substances, they are able to choose which substance they want to learn about out of those they have used. Given that it is a technology-based programme it could be tailored more specifically in future iterations.
Modification:	Yes, the purpose of this study was to modify the intervention based on feedback from providers and adolescents. Please see Figure 2 for examples of how the sample content changed after feedback.
How well (planned):	In the current study, participants reviewed the entire programme because an interview took place after viewing the intervention content. Future work needs to assess fidelity to the programme when delivered as part of a feasibility trial. Therefore, adherence was not assessed as an outcome in the current study.
How well (actual):	Adherence was not assessed as an outcome in the current study.

Figure 2. Sample content from Teen Well Check across each phase of the study.

Table 2. Theory- and evidence-based components in Teen Well Check.

Target outcome	Theoretical framework	Prevention components
Substance use	Motivational Interviewing (Naar-King & Suarez, 2011)	Screening, reinforcement for non-use, self-identified consequences of use (Dimeff et al., 1999; Naar-King & Suarez, 2011; National Institute on Alcohol Abuse and Alcoholism, 2011)
	Social Norms Theory (Borsari & Carey, 2001; Collins et al., 1985; Lewis & Neighbors, 2004; Perkins, 2002)	Correcting misperceptions of perceived and actual peer norms of alcohol and drug use
	Ecological Framework (Bronfenbrenner, 1979)	Increase communication skills with peers (including romantic and sexual partners), parents, and medical providers regarding use
	Negative Reinforcement Theory (Skinner, 1958)	Psychoeducation on traumatic stress symptoms and use of substances to cope with traumatic stress symptoms, encouraging active coping strategies
Substance use before sex	Sex-Related Substance Use Expectancies (e.g. Alcohol and Marijuana Expectancy Theory) (Hendershot et al., 2010)	Interactive feedback regarding sex-related substance use expectancies (Gilmore et al., 2022)
	Impairment Theories (e.g. Alcohol Myopia Theory) (Steele & Josephs, 1990)	Interactive feedback applying alcohol myopia theory to sexual decision-making (Gilmore et al., 2022)
Sexual risk behaviours	Social Norms Theory	Correcting misperceptions of perceived and actual peer norms of sexual risk behaviours (Lewis et al., 2014)
	Skills Deficit Model	Interactive education to increase condom use, condom use self-efficacy, and sexual communication skills
Sexual assault victimization	Ecological Framework/Cognitive Mediational Model (Gidycz et al., 2006; Nurius & Norris, 1996; Ullman, 2014)	Increase sexual communication skills, sexual assault risk perception skills, and effective use of resistance strategies through scenarios (Gilmore et al., 2022)
Sexual assault bystander intervention	Social Norms Theory/Bystander Education (Fischer et al., 2011; Jouriles et al., 2018; Labhardt et al., 2017; Latané & Darley, 1970; Latané & Nida, 1981; Latané & Rodin, 1969)	Interactive bystander education, psychoeducation on substance use and bystander intervention (Gilmore et al., 2022)

Table 3. Characteristics of primary care providers (n = 11) and adolescents 14–18 years (n = 35).

Characteristic	Providers		Adolescents (Intervention Development)		Adolescents (Intervention Refinement)
	N	%	N	%	N	%
Gender
Female	11	100%	14	56%	8	80%
Male	0	0%	9	36%	2	20%
Other (Genderfluid, Two Spirit)	0	0%	2	8%	0	0%
Age			16.3 (M)	1.37 (SD)	16.7 (M)	1.49 (SD)
Race
White/Caucasian			9	36%	6	60%
Black/African American			11	44%	1	10%
Asian American/Pacific Islander			0	0%	1	10%
Biracial			2	8%	1	10%
Other/Mixed			3	12%	1	10%
Ethnicity
Hispanic/Latinx			2	8%	1	10%
Not Hispanic/Latinx			23	92%	9	90%
Sexuality
Straight/Heterosexual			15	60%	5	50%
Bisexual			4	16%	3	40%
Gay/Lesbian			4	16%	0	0%
Questioning					2	20%
Other (Pansexual)			2	8%	0	0%
Tablet Access			24	96%	9	90%
Past year alcohol use			3*	12%*	4	40%
Past year tobacco use			N/A*	N/A*	5	50%
Past year drug use			1*	4%*	7	70%

*Note: During Aim One, participants were not asked about smoking behaviours or past year of alcohol or drug use. These results for Aim One reflect past month alcohol and drug use.

Table 4. Sample feedback quotes on phase of Teen Well Check.

Adolescent interviews				Provider interviews
Intervention development results	Sample quotes	Intervention refinement results	Sample quotes	Intervention refinement results	Sample quotes
Aesthetics	(1) ‘Well, color-wise, there are like, or from what I’ve heard, there are colors that attract the human eye more. So, like color-wise, making it, a little more colorful, maybe even the font wise. It's some, it's trying to reach each teenage group, so it's like, if I see color, like if see the love right there and it's in red, it's like a pop out.’	Engagement and Interaction	(5) ‘I actually really liked that because it actually helps people learn about your brain. And I didn't even know that some things are certain things. So, I think that's really cool … I like how you added a video too. That's like, I like videos.’	Engagement and Interaction	(6) ‘I like the way you guys asked questions that they had to kind of guess on and then you gave them the correct response. So I like that it was interactive.’
Alcohol and Substance Use	(2) ‘Some people don't really know about the brain but, this just tells you about it basically and how you could stop doing that because this is how it could affect you like you’re just thinking about it, like oh people told me about lungs, smoking and that but then your brain is also slowly functioning because of you doing that. Yeah, I like that.’	Language and Tone	(7) ‘I feel like it's good. I don't think it's very criticizing of people. So, it's very open-ended. And I like how you already included the alcohol at the party. So, it's not like, oh, you should just avoid all risky situations to begin with and like – I like how it automatically – I mean, they’ll understands that we do have – people have sex. And it's not just, oh, you just use abstinence, like it welcomes the idea that you are allowed to do things that like you’re allowed to drink and you’re allowed to, like, have fun and like, engage in sexual activity as long as you’re safe. So, it's kind of acknowledging maybe like, yes, people do instead of ignoring like, oh, you should just not do it until later.’	Language and Tone	(8) ‘It sounds harsh. I’m having trouble articulating but like almost like judgy … So anything that has somewhat like all or nothing language, I think unconsciously can turn some kids off and … sort of missed an opportunity. So just subtle things like the word “forever,” which is true, but … You know, maybe that's too darkening … When you do these kind of screening things. The more that you feel you can be asked without judgment. I think the more it lands.’
Healthy Relationships (Sexual Risk)	(3) ‘When I took sex ed and like, I think it was eighth grade maybe, I saw this video about like how to put on a condom and it was like this cartoon thingy –– it was teaching us how to put condoms on and he was like putting it on himself and like sliding it down, it was kind of weird but it was like funny and education at the same time. So, maybe [include] something like that.’	Inclusivity	(9) ‘There's a lot more identities coming out now. So, just to be a little more inclusive, because a lot of people [are] like non-binary.’	Inclusivity	(10) ‘I feel like you guys have tried to have graphics that are like representing diverse individuals. But I think you could just, like, make sure you’re looking at those again because like this page, it looks like a lot of Caucasian. Like five of the six people, at least from the view that I can see what Caucasian and teams. Want to see themselves represented … yeah but being sure to have like diverse body types and skin colors, etc … Not same sex couples, but I think all those are important considerations.’
Consent (Sexual Assault)	(4) ‘Like, I think with a girl you could say … Like, I don't want you to do that or like, no you can't do that, sorry. Or don't even apologize because it's your body. You could say something like, that's making me uncomfortable, can you stop that. I like you, but I don't want to do that.’	Aesthetics and Images	(11) ‘Don't use people pictures.’	Aesthetics and Images	(12) ‘You know, thinking teenager mindset, “hokey” stock pictures are their least favorite in the world. There's no way to avoid that.’
		Personal Background and Use of Stories	(13) ‘I think I know most about drugs because, my mom did drugs with my dad, so that's why I live with my grandparents. And I’ve seen what it does to you, so I stay away … I’ve seen what other kinds don't and they don't know what actually happens. And they can't feel what I’ve gone through … it's not real to them, they just don't understand … If they’re doing it, they aren't going to stop because we’ve already been taught not to do it. Unless something dramatic happens … I don't know maybe put in some scary pictures … what it looks like after you use this type of stuff. And what I think really gets to people are like having other people's stories be heard. Like in middle school we had this guy come in and he was telling us about his son and like I don't know – I feel like you can't really get personal with just some slides and stuff.’	Personal Background and Use of Stories	(14) ‘Some of this content is like a lot of fact-based content. I don't know if there's any role of like … More of a like personal story or anecdote. Like that's good content, maybe just might feel a little more applicable if it wasn't Just numbers and facts, but like maybe what happened to someone who use this or something like that. So it felt like maybe a little bit more relatable.’
				Logistics and Information Sharing	(15) ‘If you were to maybe expand into take-home because I do understand time is an issue so that would be very good. But if it was like a take-it-home and do it separately, I would probably include all of them [modules] just for educational purposes.’
		Parent/guardian-related Topics	(16) ‘So having some important resources like this, we just don't want it to be something that's going to cause the person holding this tablet to like, click out of it or shut down and be like, “I don't want to see this” or even something that would make them uncomfortable with it if they’re in the waiting room … Especially like, if it's an underage person, their parent is going to probably be like, right there and watching … So that's also a big issue, especially for me. So like, if I was a kid and say, I’m like, 14, and my mom's right there, so many questions I’m not going to … I wouldn't answer honestly if my parents in, they are sitting right there.’	Parent/guardian-related Topics	(17) ‘I think most of our parents will be pretty positive about it. I think a lot of my parents are not naive to think that their kids are exposed or engaging and some of these behaviors … We have a smaller section of patients and families that would be very uncomfortable. They’re very uncomfortable with just alone time with us in the room. And they get very nervous and want to know exactly what we’re talking about. So I think those families would have difficulty with it … But I think for the majority of our families … they would welcome any chance to educate their kids and try and save them from risky behavior.’

Table 5. Usability and acceptability of the Teen Well Check tablet-based intervention among primary care providers (n = 11) of adolescents 14–18 years in the Intervention Refinement process (n = 10).

Survey Item	Provider Average Score; Mean (SD)	Adolescent Average Score; Mean (SD)	Scale
Post-Study System Usability Questionnaire	1.84 (1.10)	1.24 (0.35)	1=Strongly Agree (Usable)	5=Strongly Disagree (Not Usable)
How likely would you be to use this tablet-based intervention in your clinical practice?	5.09 (1.76)		1=Very Unlikely	7 =Very Likely
How likely would you be to recommend that teens aged 14–18 use this tablet-based intervention?	5.36 (1.50)		1=Very Unlikely	7 =Very Likely

Naar-King, S., & Suarez, M. (2011). Motivational interviewing with adolescents and young adults. Guilford Press.

 Google Scholar

Dimeff, L. A., Baer, J. S., Kivlahan, D. R., & Marlatt, G. A. (1999). Brief alcohol screening and intervention for college students: A harm reduction approach. Guilford Press.

 Google Scholar

National Institute on Alcohol Abuse and Alcoholism. (2011). Alcohol screening and brief intervention: A practitioner’s guide. http://pubs.niaaa.nih.gov/publications/Practitioner/YouthGuide/YouthGuide.pdf

 Google Scholar

Borsari, B., & Carey, K. B. (2001). Peer influences on college drinking: A review of the research. Journal of Substance Abuse, 13(4), 391–424. https://doi.org/10.1016/s0899-3289(01)00098-0

 PubMedGoogle Scholar

Collins, R. L., Parks, G. A., & Marlatt, G. A. (1985). Social determinants of alcohol consumption: The effects of social interaction and model status on the self-administration of alcohol. Journal of Consulting and Clinical Psychology, 53(2), 189–200. https://doi.org/10.1037//0022-006x.53.2.189

 PubMed Web of Science ®Google Scholar

Lewis, M. A., & Neighbors, C. (2004). Gender-specific misperceptions of college student drinking norms. Psychology of Addictive Behaviors, 18(4), 334–339. https://doi.org/10.1037/0893-164X.18.4.334

 PubMed Web of Science ®Google Scholar

Perkins, H. W. (2002). Social norms and the prevention of alcohol misuse in collegiate contexts. Journal of Studies on Alcohol, Supplement, 14(s14), 164–172. https://doi.org/10.15288/jsas.2002.s14.164

 Google Scholar

Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Harvard University Press.

 Google Scholar

Skinner, B. F. (1958). Reinforcement today. American Psychologist, 13(3), 94–99. https://doi.org/10.1037/h0049039

 Web of Science ®Google Scholar

Hendershot, C. S., Magnan, R. E., & Bryan, A. D. (2010). Associations of marijuana use and sex-related marijuana expectancies with HIV/STD risk behavior in high-risk adolescents. Psychology of Addictive Behaviors, 24(3), 404–414. https://doi.org/10.1037/a0019844

 PubMed Web of Science ®Google Scholar

Gilmore, A. K., Leone, R. M., Oesterle, D., Davis, K. C., Orchowski, L. M., Ramakrishnan, V., & Kaysen, D. (2022). Web-based alcohol and sexual assault prevention program with tailored content based on gender and sexual orientation: Preliminary outcomes and usability of Positive Change (+change). JMIR Formative Research, 6(7), e23823. https://doi.org/10.2196/23823

 PubMedGoogle Scholar

Steele, C. S., & Josephs, R. A. (1990). Alcohol myopia: Its prized and dangerous effects. American Psychologist, 45(8), 921–933. https://doi.org/10.1037/0003-066X.45.8.921

 PubMed Web of Science ®Google Scholar

Lewis, M. A., Litt, D., Cronce, J., Blayney, J. A., & Gilmore, A. K. (2014). Underestimating protection and overestimating risk: Examining descriptive normative perceptions and their association with drinking and sexual behaviors. The Journal of Sex Research, 51(1), 86–96. https://doi.org/10.1080/00224499.2012.710664

 PubMed Web of Science ®Google Scholar

Gidycz, C. A., McNamara, J. R., & Edwards, K. M. (2006). Women’s risk perception and sexual victimization: A review of the literature. Aggression and Violent Behavior, 11(5), 441–456. https://doi.org/10.1016/j.avb.2006.01.004

 Web of Science ®Google Scholar

Nurius, P. S., & Norris, J. (1996). A cognitive ecological model of women’s response to male sexual coercion in dating. Journal of Psychology & Human Sexuality, 8(1-2), 117–139. https://doi.org/10.1300/J056v08n01_09

 PubMedGoogle Scholar

Ullman, S. E. (2014). Reflections on researching rape resistance. Violence Against Women, 20(3), 343–350. https://doi.org/10.1177/1077801214526308

 PubMed Web of Science ®Google Scholar

Fischer, P., Krueger, J. I., Greitemeyer, T., Vogrincic, C., Kastenmüller, A., Frey, D., Heene, M., Wicher, M., & Kainbacher, M. (2011). The bystander-effect: A meta-analytic review on bystander intervention in dangerous and non-dangerous emergencies. Psychological Bulletin, 137(4), 517–537. https://doi.org/10.1037/a0023304

 PubMed Web of Science ®Google Scholar

Jouriles, E. N., Krauss, A., Vu, N. L., Banyard, V. L., & McDonald, R. (2018). Bystander programs addressing sexual violence on college campuses: A systematic review and meta-analysis of program outcomes and delivery methods. Journal of American College Health, 66(6), 457–466. https://doi.org/10.1080/07448481.2018.1431906

 PubMed Web of Science ®Google Scholar

Labhardt, D., Holdsworth, E., Brown, S., & Howat, D. (2017). You see but you do not observe: A review of bystander intervention and sexual assault on university campuses. Aggression and Violent Behavior, 35, 13–25. https://doi.org/10.1016/j.avb.2017.05.005

 Web of Science ®Google Scholar

Latané, B., & Darley, J. (1970). The unresponsive bystander: Why doesn’t he help? Appleton-Century Crofts.

 Google Scholar

Latané, B., & Nida, S. (1981). Ten years of research on group size and helping. Psychological Bulletin, 89(2), 308–324. https://doi.org/10.1037/0033-2909.89.2.308

 Web of Science ®Google Scholar

Latané, B., & Rodin, J. (1969). A lady in distress: Inhibiting effects of friends and strangers on bystander intervention. Journal of Experimental Social Psychology, 5(2), 189–202. https://doi.org/10.1016/0022-1031(69)90046-8

 Web of Science ®Google Scholar

Data availability statement

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available.