Effects on secondary outcomes of the Brazilian version of the European unplugged drug use prevention program: drug knowledge, intention predictors, and life skill competencies

Abstract

The drug use prevention program #Tamojunto2.0, a Brazilian version of the European Unplugged, showed effectiveness in preventing the alcohol use onset at short term. However, we aimed to evaluate the program effects on its secondary outcomes, such as drug knowledge, behavioral beliefs, attitudes, decision-making skills, and refusal skills. A cluster-randomized controlled trial (registration: RBR-8cnkwq) was conducted in 73 public middle schools in three Brazilian cities (N = 5,208 students; 49.4% girls; M_age= 13.2 years). The intervention group attended twelve #Tamojunto2.0 lessons conducted by their own teachers previously trained. The control group received no intervention. Data were collected pre-intervention (February/March 2019) and at nine-month follow-up (November/December 2019). We used multiple imputation to handle missing data and performed multilevel mixed-effect regression models, adjusted for sex, age, socioeconomic status, and city. The #Tamojunto2.0 program seems to have increased drug knowledge (Coef.= 0.26, 95%CI = 0.17–0.36), and negative and non-positive alcohol beliefs (Coef.= 0.24, 95%CI= 0.05–0.42), according to its logical framework. Nevertheless, it was found no evidence regarding the program’s effect on marijuana beliefs, attitudes, decision-making skills, and refusal skills. These findings could explain the effectiveness of the program in preventing the onset of alcohol consumption in adolescent participants. More research is needed to observe the long-term effects of the program on primary and secondary outcomes.

Keywords:

• Prevention
• adolescents
• drug knowledge
• intention predictors
• life skill competencies

1. Introduction

In the field of prevention, the interplay between science and practice is crucial, with schools being indicated as strategic places for the implementation of prevention programs (Coie et al., 1993). Regarding drug use prevention in adolescents, there is a large body of scientific literature on the key characteristics of school-based programs that have shown effectiveness on preventing drug use (Armitage & Conner, 2001; Cuijpers, 2002; Faggiano et al., 2008b; 2014; Guo et al., 2015; Huang et al., 2012; Nation et al., 2003; NIDA, 2021; Peters et al., 2009; Skeen et al., 2019). Therefore, the evaluation of school-based programs is essential, because its execution in countries such as Brazil, habitually has an inconsistent design, sporadic implementation, and rarely subjected to evaluation (Pereira & Sanchez, 2020).

In 2013, the Ministry of Health in Brazil (BMH) implemented the Unplugged program, an European school-based intervention for drug use prevention in adolescents (van der Kreeft et al., 2009). It integrates the ‘Preventive Principles’ (NIDA, 2021) and conceptual framework of many psychosocial theories, as Social Learning theory, Social Norms theory, Health Belief theory, theory of Reasoned Action-Attitude, I-Change Model, and Problem Behaviour theory (Vadrucci et al., 2016). These theories underlie the program’s psychosocial constructs (such as behavioral beliefs and attitudes), life skills competencies (critical thinking, creative thinking, relationship skills, communication skills, assertiveness, refusal skills, managing emotions, coping, empathy, problem solving, and decision making), and drug knowledge, which are its ‘secondary outcomes’ and serve as ‘intermediate variables’ (hereinafter abbreviated as ‘IV’) on the primary outcomes of the program (drug use prevention). Hence, it was expected that the program would improve the adolescents’ personal and interpersonal skills to control social influences, through which adolescents develop erroneous perceptions of the frequency and acceptability of drug consumption (Giannotta et al., 2014). The short-term primary goal of this program was to reduce the number of adolescents who used alcohol and other drugs (Faggiano et al., 2008a).

Therefore, the IV are considered predictors of future substance use, which play a crucial role in the efficacy of school-based programs by determining their success in achieving the drug use prevention outcomes (Cuijpers, 2002; Sussman et al., 2004). The association between IV and self-reported drug use has been studied (Garcia-Cerde et al., 2021; Healy et al., 2020; Mehanović et al., 2020; Sanchez et al., 2019a; Stephens et al., 2009), along with changes in behavior through changing behavioral intentions (Webb & Sheeran, 2006). Meanwhile, few studies have focused on the effect of programs on IV (Cashin & Lee, 2021; Giannotta et al., 2014).

Unplugged has demonstrated effectiveness on preventing alcohol and marijuana use (Agabio et al., 2015; Faggiano et al., 2008a; 2010; Vigna-Taglianti et al., 2014) among adolescents between 12 to 14 years old, and shown positive effects in endorsing fewer positive attitudes toward all drugs; fewer positive beliefs about cigarettes, alcohol, and cannabis; correcting normative misperceptions of peers using tobacco and cannabis; and increasing knowledge of all substances and refusal skills toward tobacco (Giannotta et al., 2014). In Czech Republic, Unplugged was effective in preventing any smoking and any cannabis use (Gabrhelik et al., 2012). In a recent evaluation of Unplugged implementation in Nigeria, the program reduced the prevalence of recent alcohol use via three IV—negative beliefs, risk perceptions, and class climate (Vigna-Taglianti et al., 2021).

The #Tamojunto2.0 program is the third Brazilian version of the Unplugged program aimed at middle school students between 12 and 14 years of age. The first version (2013) consisted of a full translation of Unplugged, with no adaptation of activities or visual identity, with only the substitution of information on heroin to crack and cocaine. It was evaluated by Sanchez et al. (2016), presenting a marginal effect in reducing binge drinking and marijuana use. The second version (2014–2015), called #Tamojunto, was a cultural adaptation of Unplugged, that excluded its original components that reinforced non-alcohol use and replaced them with a harm reduction approach. These changes were implemented to guarantee the adaptation of Unplugged to the National Drug Policy in force in Brazil at that time (Decree 4345, 26 August 2002). Article 6 of the Decree foresaw an emphasis on harm reduction as a preventive action in the country, but the original Unplugged program was not focused on harm reduction. #Tamojunto showed mixed results: an iatrogenic effect for alcohol initiation and a reduction on past year inhalants use were found (Sanchez et al., 2018). In #Tamojunto, two IV have been evaluated: no evidence was found of effectiveness on the attitudes toward drug use (Sanchez et al., 2019a), and it was observed that the program decreased decision-making skills in the opposite direction as proposed by the theoretical model (Valente et al., 2020). Based on the negative results found in the #Tamojunto, a third version of the program was developed between 2018 and 2019, and was named #Tamojunto2.0. This adaptation, like the previous, were performed by the BMH, focusing on reinstating the original components of the Unplugged program and removing the components related to harm reduction, that were not part of the European Program. According to a recent evaluation (Sanchez et al., 2021), #Tamojunto2.0 reduced alcohol initiation in the short term. However, a statistically significant difference was not found in the prevalence of binge drinking, tobacco, inhalants, marijuana, and cocaine within the past month between intervention and control groups.

Although, it seems that the current modification of the #Tamojunto2.0 curricula may have been appropriate, the IV outcomes have not been investigated. There is literature that points out the importance of analyzing first and separately the effectiveness of drug use prevention programs on IV (or secondary outcomes), due to the low prevalence of drug use in adolescents, which can make it difficult to identify the effectiveness of programs on the prevention of drug use (Andrews et al., 2003). In this sense, it is recommended that the intermediary variables should be evaluated first in the short-term and only subsequently conduct a mediation analysis with data from a third wave (Wholey et al., 2010). Therefore, we aimed to evaluate the effectiveness of the #Tamojunto2.0 program on its secondary outcomes: drug knowledge, intention predictors, and life skill competencies.

2. Methods

2.1. Study design, randomization, and study sample

We used data from a two-armed, parallel, cluster-randomized controlled trial (cRCT) conducted to evaluate the effectiveness of #Tamojunto2.0, with eighth-grade students in 73 middle public schools in the Brazilian cities of São Paulo, Fortaleza, and Eusébio. This trial compared the results of intervention condition (integration of #Tamojunto2.0 in the school curricula) versus control condition (no integration) in 2019. The program was adapted and implemented by the BMH, and the cRCT was designed and conducted by an independent research team from the Universidade Federal de São Paulo (UNIFESP).

The randomization process was conducted in two stages. Firstly, a governmental list of public schools offering 8th grades in each municipality was retrieved from the National Institute for Educational Studies and Research ‘Anísio Teixeira’ (INEP). From 388 eligible schools in the initial list, 70 schools (i.e. based on the sample size calculation) were randomized as the main target schools of the study, while an extra 70 schools were put on a potential replacement list in case of non-acceptance to participate. Both samples were selected via an algorithm based on ‘atmospheric noise’ (a source of specific randomness based on the numbers generated by the lightning discharges static) available in www.random.org. Within both sampling groups (the target group and the replacement list), the random assignment to the arms intervention or control group was conducted using the Efron’s biased coin, allowing the maintenance of a balanced sample (1:1 allocation ratio per municipality), and was implemented in PASS version 22. Within the intervention group, all 8th-grade students participated in the #Tamojunto2.0 program, and the school assigned one teacher per class to receive training to incorporate the program in the school curricula. Because of the involvement of the government, all schools agreed to participate. We initially invited 76 (10% more, preventing some of them from withdrawing from participation) and reached a total of 73 schools (Sanchez et al., 2021).

The baseline data collection was conducted before the implementation of the program in February and March 2019, and the follow-up data were collected nine months after the baseline in November and December 2019. The study estimated a sample size of at least 3150 adolescents in the control group and 3150 in the intervention group, distributed among 35 clusters (schools), seeking to obtain a statistical power of 82% to identify a difference between groups of 2.5% for the outcome of binge drinking in the past month, with an initial prevalence of 10%, a significance level of 5%, and an intraclass correlation of 0.005. Data were collected simultaneously from the control and intervention schools. Details on the study design can be found in a previous publication (Sanchez et al., 2019b).

2.2. Ethics and trial registration

Written informed consent to participate in the study was obtained from the school directors before randomization and from students and parents after randomization. This trial and the pre-registered hypothesis were registered in the Brazilian Registry of Clinical Trials (RBR-8cnkwq), whose structure of contents is similar to other national official registers of sRCT (Freitas et al., 2016). The protocol was approved by the UNIFESP Research Ethics Committee (#2,806,301) and the Ethics Committee of the Municipal Health Secretariat (#3,099,865).

2.3. Intervention characteristics and fidelity

The #Tamojunto2.0 program consists of 12 interactive classes lasting 50 minutes and held weekly. It includes lessons that provide information on drugs, and social, interpersonal and personal skills. Each lesson had three to five activities that address these topics. The program is applied by teachers who completed 16 hours of training, using handbooks for teachers and students (EU-DAP Study Group, 2016b). To support the teachers who were applying the program in the classroom, phone calls were made every 15 days to answer questions about the program implementation, when they had doubts about how to implement an activity or what to do in a specific case. Only 67% of the enrolled intervention classes completed all lessons.

2.4. Instrument

This study collected data through an anonymous paper-and-pencil questionnaire completed by the students and administered by researchers without a teacher in the classroom. In each assessment, students provided a code generate from letters and numbers from their personal information. The datasets of the two evaluation time points were integrated by matching this confidential code using the Levenshtein algorithm (Levenshtein, 1966).

The instrument was the same as that used in the previous evaluation studies of #Tamojunto and Unplugged (EU-Dap, 2004; EU-DAP Study Group, 2016a; Faggiano et al., 2010; Giannotta et al., 2014; Sanchez et al., 2017). The Brazilian Portuguese version was adapted and supplemented (Cainelli de Oliveira Prado et al., 2016; Carlini et al., 2010; IBGE, 2016), and also validated by Galvão et al. (Galvão et al., 2021). To avoid over-reporting of drug use, we excluded questionnaires that were positive for lifetime use of a fictional drug (Holoten and Carpinol) from the analysis (baseline, n = 35; follow-up, n = 37).

2.5. Measurements

The dependent variables were secondary outcomes from the #Tamojunto2.0 program: drug knowledge, psychosocial constructs on drug use intention predictors (behavioral beliefs and attitudes), and life skills competencies for drug use resistance (decision-making and refusal skills) (see Annex 1).

As mentioned above, the instrument and scales used in this study were already validated in a Brazilian adolescent population (Galvão et al., 2021); however, to confirm and provide evidence of the construct validity of the measures used, we performed confirmatory factor analysis (CFA) for the scales of: alcohol beliefs, marijuana beliefs, attitudes, and decision-making skills. The CFA, unlike other approaches, makes it possible to contrast a model constructed in advance, in which the researcher establishes a priori, on the basis of a well-established theory, the total set of relationships between the elements that make it up and only needs to confirm that this structure can also be obtained empirically (Bentler, 2007; Ondé & Alvarado, 2020). We did not perform this analysis for the drug knowledge scale (which is not intended to measure a latent construct but rather to assess learning) nor for the questions on refusal skills (which are dichotomous variables). To evaluate the goodness of fit, we used the comparative fit index (CFI), Tucker–Lewis index (TLI), and root mean square error of approximation (RMSEA). The cutoff criteria used to determine the goodness of fit were an RMSEA estimate near or less than 0.08, RMSEA probability near or equal to 1, and CFI and TLI near or greater than 0.90 (Little, 2013). Mplus version 8.0 was used to run the CFA. As shown in Table 1, the indices indicated a close fit in all models evaluated, at least in the RMSEA estimate and CFI.

Table 1. Goodness of fit indicators for the construct validation of alchol beliefs, marijuana beliefs, attitudes, and decision-making skills variables in the #Tamojunto2.0 program, 2019 (N = 5208).

	Baseline						9-months Follow-up
	X² test of model fit		RMSEA		CFI	TLI	X² test of model fit		RMSEA		CFI	TLI
	Value	p Value	Estimate	Probability			Value	p Value	Estimate	Probability
Alcohol beliefs^a	777.336	<0.001	0.058	<0.001	0.939	0.920	602.616	<0.001	0.059	<0.001	0.949	0.933
Marijuana beliefs^a	1344.495	<0.001	0.078	<0.001	0.936	0.916	1440.776	<0.001	0.093	<0.001	0.919	0.894
Attitudes^b	404.433	<0.001	0.048	0.741	0.911	0.875	352.102	<0.001	0.051	0.366	0.946	0.924
Decision-making skills^b	418.908	<0.001	0.060	0.001	0.912	0.868	622.631	<0.001	0.081	<0.001	0.845	0.768

RMSEA: root mean square error approximation; CFI: comparative fit index; TLI: Tucker-Lewis index.

Bold letters highlight close-fit results.

^aThese models include two item correlations.

^bThese models include three item correlations.

Drug knowledge was measured through a 6-item scale (Table 3), with three answer options: incorrect, correct, and don’t know. The answers were dichotomized considering the right option as 1 and the wrong and ‘don’t know’ options as 0. Then, we summarized all answers to generate a score of ‘correct knowledge’. That is, the higher the score, the higher correct knowledge the student had.

The psychosocial constructs on drug use intention predictors came from the theoretical model of the Reasoned Action–Attitude and Planned Behaviour Theory, in which ‘intention’ is defined as the readiness to perform a behavior, but at the same time, intention is made up of the so-called ‘behavioral intention predictors’, which are: behavioral beliefs (information about behavior consequences), attitudes (evaluation of the positivity or negativity of behavior consequences), normative beliefs (perceived expectations from other important people), and subjective norms (motivations for complying with those expectations) (Ajzen & Fishbein, 2000; Fishbein & Ajzen, 1975; Hale et al., 2002; Vadrucci et al., 2016). ; We collected data on the first two variables.

Behavioral beliefs were collected using an 11-item scale (Table 3) with dichotomous answers (no/yes) for both alcohol and marijuana, asked separately, with two underlying factors each: negative and non-positive beliefs (negative information about drug use consequences), and positive and non-negative beliefs (positive information about drug use consequences). We codified affirmative answers in negative beliefs as 1, as were negative responses in positive beliefs (non-positive beliefs). We then summarized all the answers to generate a score: the higher the score, the higher the negative information about drug use consequences the student had.

Attitudes were collected using an 11-item scale (Table 3) with dichotomous answers (I agree/I disagree) for all drugs. As with behavioral beliefs, the attitudes construct had two underlying factors: negative and non-positive attitudes (when the student negatively evaluates drug use consequences), and positive and non-negative attitudes (when the student positively evaluates drug use consequences). As mentioned in literature (Garcia-Cerde et al., 2021), because the ninth statement follows a different trend compared with the other positive attitude statements, it was excluded from the analysis. To obtain the score, attitude answers were codified like behavioral belief scales: agreement answers in negative attitudes were codified as 1, as were disagreement responses in positive attitudes (non-positive attitudes). Higher scores indicated higher negative evaluation of drug use consequences.

Life skill competencies for drug use resistance were drawn from problem behavior theory, in which a ‘problem behavior’ is defined as a source of concern by social or legal norms (Jessor & Jessor, 1977). Three systems of factors can put at risk or protect against problem behavior: psychosocial, contextual, and those related to the behavior itself. The balance within and between these systems determines the probability of engaging in problem behavior. In this sense, by practicing creative thinking, decision making, problem solving, coping strategies, empathy, and communication skills, individuals can develop positive behaviors and health choices; and by practicing critical thinking, assertiveness, and refusal skills, individuals can better evaluate and react to contextual influences (Vadrucci et al., 2016). We analyzed data on the decision-making and refusal skills.

Decision-making skills were assessed using a 9-item scale (Table 3) with dichotomous answers (I disagree/I agree). This construct had two underlying factors: good and poor decision-making capacity. We codified the agreement answer in good decision-making capacity statements as 1, as were the disagreement answers in poor decision-making capacity. The higher the score, the better the student’s decision-making capacity.

Three refusal skills were analyzed as independent items. The answer options were no, yes, and maybe accept. We dichotomized the response categories, codifying the negative answer as 0, and the affirmative and maybe responses as 1.

The independent variables were group (control/intervention), sex (boys/girls), age (from 12 to 17 years), socioeconomic status, and city. Students’ socioeconomic status was assessed using the scale of the Brazilian Association of Research Companies (ABEP), which is scored from 1 to 100 points and considers the education level of the head of the household and the goods and services used, with categories ranging from A (highest) to D/E (lowest) (ABEP, 2018; see Annex 1). In the study of Galvão and colleagues (Galvão et al., 2021), this scale was also validated in a population of Brazilian adolescents. Additionally, for informational purposes only, we present in Table 2 the lifetime prevalence of drug use.

Table 2. Sociodemographic characteristics at the baseline of the participants in the randomized controlled trial of the #Tamojunto2.0 school-based drug use prevention program, 2019 (N = 5280).

	Total (N = 5208)		Intervention (N = 2840)		Control (N = 2368)		p Value^a
	n	%	n	%	n	%
Sex							0.062
Boys	2576	49.46	1436	50.56	1140	48.14
Girls	2570	49.35	1366	48.10	1204	50.84
Baseline missing	62	1.19	38	1.34	24	1.01
Age							<0.001
Average age (± SD)	13.23 ± 0.85		13.19 ± 0.81		13.28 ± 0.89
12 years	646	12.40	347	12.22	299	12.63
13 years	3199	61.42	1830	64.44	1369	57.81
14 years	800	15.36	387	13.63	413	17.44
15 years	318	6.11	154	5.42	164	6.93
16 years	96	1.86	43	1.51	53	2.24
17 years	21	0.40	11	0.39	10	0.42
Baseline missing	128	2.46	68	2.39	60	2.53
Socioeconomic status (SES)							<0.001
Average SES score (± SD)	24.75 ± 9.19		25.25 ± 9.19		24.16 ± 9.15
A: 45–100 (highest)	179	3.44	108	3.80	71	3.00
B: 29–44	1279	24.56	757	26.66	522	22.04
C: 17–28	2809	53.94	1505	52.99	1304	55.07
D/E: 1–16 (lowest)	882	16.94	429	15.11	453	19.13
Baseline missing	59	1.13	41	1.44	18	0.76
City							<0.001
Eusébio	784	15.05	364	12.82	420	17.74
Fortaleza	2051	39.38	1029	36.23	1022	43.16
São Paulo	2373	45.56	1447	50.95	926	39.10
Drug use lifetime
Alcohol	2516/5156	48.80	1367	54.33	1149	45.67	0.899
Binge drinking	1106/5187	21.32	583	52.71	523	47.29	0.183
Tobacco	614/5145	11.93	310	50.49	304	49.51	0.045
Marijuana	424/5130	8.27	203	47.88	221	52.12	0.006
Inhalants	1026/5122	20.03	549	53.51	477	46.49	0.562

N or n: sample number; %: percentages.

^aT-test or Chi-squared test.

2.6. Statistical analysis

We used the intention-to-treat (ITT) paradigm to analyze the effects of #Tamojunto2.0 on all secondary outcomes. This paradigm estimates the program effect among all participants without considering whether they were present at the follow-up assessment. To handle missing data in the ITT analysis, we used ‘multiple imputation’. This technique replaces each missing value with a set of plausible values that preserved the statistical distribution of the imputed variable and its relation with other variables in the imputation model (Graham et al., 1997). This process resulted in valid statistical inferences that could reflect the uncertainty brought by missing values (Rubin, 1996).

To examine the #Tamojunto2.0 program’s effect on each dependent variable in the intervention group, we implemented a ‘difference in differences’ (DiD) approach. DiD is a statistical technic that makes use of longitudinal data from treatment and control groups to obtain an appropriate counterfactual to estimate a causal effect. DiD is typically used to estimate the effect of a specific intervention or treatment by comparing the changes in outcomes over time between a population that is enrolled in a program (the intervention group) and a population that is not (the control group). DiD is usually implemented as an interaction term between time and treatment group dummy variables in a regression model. In this study, we carried out multi-level mixed effects regression models with random intercepts, to account for the clustering of pupils within schools. In this type of model, both the variability between the measures of the same individual and the variability between the individuals themselves are taking into account, allowing highlighting a relationship between the observed response and explanatory covariates (Beroho et al., 2020; Pinheiro & Bates, 2000). Therefore, it was performed multi-level mixed effects linear regressions to modeling the response variables of drug knowledge, alcohol beliefs, marijuana beliefs, and decision-making skills; and multi-level mixed effects logistic regressions for refusal skills. All models were estimated clustering in the school and individual levels and adjusted by sex, age, socioeconomic status, and city. For the attrition analysis, we compared students whose data from the two time points were matched with students who answered only the baseline questionnaire.

Inferential estimates were given as adjusted coefficients (Coef.) or adjusted odds ratios (ORs) with their respective 95% confidence intervals (95% CIs) and p-values. The level of significance was set at 5%. All analyses were performed using Stata SE version 16.

3. Results

Figure 1 shows the sample flowchart of the #Tamojunto2.0 cRCT. Among the 6993 students enrolled in the 73 randomized schools, 5208 answered the baseline questionnaire, and 3898 (74.8%) answered the nine-month follow-up questionnaire.

Figure 1. Flowchart of the randomized controlled trial to assess the effectiveness of #Tamojunto2.0 program, 2019 (N = 5208). ^a Students enrolled in schools drawn in 2019 - not necessarily attending school.

Table 2 presents the students’ baseline sociodemographic characteristics and lifetime drug use at baseline. The intervention and control groups were homogenous with respect to sex (p = 0.062), the average age was 13.2 years (SD ± 0.9) (p < 0.001), more than half of participants were from a middle socioeconomic status (53.9%) (p < 0.001), and 45.6% were from São Paulo (p < 0.001). Additionally, both the intervention and control groups were homogeneous with respect to lifetime alcohol use (p = 0.899), binge drinking (p = 0.183), and inhalants (p = 0.562).

Table 3 displays the comparison between the randomized groups of #Tamojunto2.0 cRCT on secondary outcomes. The columns corresponding to follow-up present the delta or change over time. Regarding the intervention group, the variables that resulted in a positive change were knowledge, behavioral beliefs, and refusal skills, meaning that the students showed increment in these variables. Bivariate comparison between the intervention and control groups at follow-up demonstrated a statistically significant difference only in alcohol beliefs.

Table 3. Comparison between groups on secondary outcomes of the randomized controlled trial of the #Tamojunto2.0 school-based drug use prevention program, 2019 (N = 5280).

	Baseline							Follow-up
	Total		Intervention		Control			Total		Intervention		Control
	n	mean ± SD or %	n	mean ± SD or %	n	mean ± SD or %	p Value^a	n	mean ± SD or %	n	mean ± SD or %	n	mean ± SD or %	p Value^a
Drug knowledge^b	4866	1.30 ± 1.11	2612	1.28 ± 1.11	2254	1.33 ± 1.12	0.103	3735	1.33 ± 1.12	2073	1.37 ± 1.13	1662	1.30 ± 1.10	0.064
Missing	342/5208	6.57	228/2840	8.03	114/2368	4.81		1473/5208	28.28	767/2840	27.01	706/2368	29.81
Behavioral beliefs
On alcohol^c	4699	7.41 ± 2.76	2546	7.46 ± 2.73	2153	7.36 ± 2.81	0.199	3592	7.72 ± 2.62	1993	7.84 ± 2.64	1599	7.57 ± 2.59	0.003
Missing	509/5208	9.77	294/2840	10.35	215/2368	9.08		1616/5208	31.03	847/2840	29.82	769/2368	32.47
On marijuana^c	4753	8.26 ± 2.49	2582	8.31 ± 2.44	2171	8.22 ± 2.56	0.224	3629	8.31 ± 2.38	2021	8.37 ± 2.38	1608	8.24 ± 2.37	0.094
Missing	455/5208	8.74	258/2840	9.08	197/2368	8.32		1579/5208	30.32	819/2840	28.84	760/2368	32.09
Attitudes^d	4485	8.11 ± 1.95	2401	8.16 ± 1.87	2084	8.04 ± 2.04	0.039	3505	7.88 ± 2.18	1952	7.91 ± 2.13	1553	7.86 ± 2.25	0.508
Missing	723/5208	13.88	439/2840	15.46	284/2368	11.99		1703/5208	32.70	888/2840	31.27	815/2368	34.42
Decision making skills^e	4284	4.62 ± 1.62	2249	4.64 ± 1.61	2035	4.60 ± 1.63	0.408	3583	4.52 ± 1.69	2002	4.53 ± 1.69	1581	4.50 ± 1.68	0.563
Missing	924/5208	17.74	591/2840	20.81	333/2368	14.06		1625/5208	31.20	838/2840	29.51	787/2368	33.23
Refusal skills
Marijuana^f
To refuse	3970/4446	89.29	2092/2326	89.94	1878/2120	88.58	0.144	3348/3785	88.45	1861/2117	87.91	1487/1668	89.15	0.235
To accept	476/4446	10.71	234/2326	10.06	242/2120	11.42		437/3785	11.55	256/2117	12.09	181/1668	10.85
Missing	762/5208	14.63	514/2840	18.10	248/2368	10.47		1423/5208	27.32	723/2840	25.46	700/2368	29.56
Tobacco^g
To refuse	4096/4415	92.77	2155/2312	93.21	1941/2103	92.30	0.242	3456/3777	91.50	1924/2112	91.10	1532/1665	92.01	0.318
To accept	319/4415	7.23	157/2312	6.79	162/2103	7.70		321/3777	8.50	188/2112	8.90	133/1665	7.99
Missing	793/5280	15.23	528/2840	18.59	265/2368	11.19		1431/5208	27.48	728/2840	25.63	703/2368	29.69
Alcohol^h
To refuse	3559/4395	80.98	1887/2304	81.90	1672/2091	79.96	0.102	2824/3766	74.99	1600/2108	75.90	1224/1658	73.82	0.144
To accept	836/4395	19.02	417/2304	18.10	419/2091	20.04		942/3766	25.01	508/2108	24.10	434/1658	26.18
Missing	813/5208	15.61	536/2840	18.87	277/2368	11.70		1442/5208	27.69	732/2840	25.77	710/2368	29.98

n: sample number; SD: standard deviation.

^aT-test or Chi-squared test.

^bThe sentences were: ‘nicotine is the substance in cigarettes that causes lung cancer’ (incorrect); ‘one needs to smoke several cigarettes per day during many years to become addicted’ (incorrect); ‘women have lower tolerance to alcohol than men’ (correct); ‘it takes about half an hour to eliminate from the body the amount of alcohol contained in a can of strong beer’ (incorrect); ‘smoking marijuana causes dependence’ (correct); and ‘high consumption of hash or marijuana decreases the production of sexual hormones’ (correct).

^cThe question was: ‘do you believe that the consumption of alcohol/marijuana can lead to the situations below?’ Negative belief sentences were: ‘get in trouble with the police’; ‘do badly in school’; ‘get into trouble with parents’; ‘getting kicked out of school’; ‘have problems with my friends’; ‘become an addict’; ‘have money problems’; and ‘having trouble finding a job’. Positive belief sentences were: ‘have more friends’; ‘feel more relaxed’; and ‘have more fun’.

^dThe question was: ‘do you agree or disagree with the following statements about drug use?’ Negative attitudes sentences were: ‘a young person should never try drugs’; ‘everyone who tries drugs eventually regrets it’; ‘the laws about illegal drugs should be made stronger’; ‘drug use is one of the biggest evils in the country’; ‘schools should teach about the real hazards of taking drugs’; and ‘to experiment with drugs is to give away control of your life’. Positive attitudes sentences were: ‘using illegal drugs can be a pleasant activity’; ‘using drugs is fun’; ‘many things are much more risky than trying drugs’; ‘drugs help people to have experience life in full’; and ‘the police should not be annoying young people who are trying drugs’.

^eThe question was: ‘These are statements about how to make decisions. Check the one that is correct for you’. Good decision-making capacity statements were: ‘when I have decided to do something, I always carry it through’; ‘I weigh up all the choices before I decide on something’; ‘when I decide something, it doesn’t matter what my friends think’; and ‘when I decide something, it doesn’t matter what my parents, or responsible, think’. Poor decision-making capacity statements were: ‘I often make up my mind without thinking of the consequences’; ‘sometimes, I make decisions with the first thing that passes in my head’; ‘sometimes make decisions and then I regret them’; ‘I always make decisions without thinking; and ‘sometimes I change my mind about something several times in a day’.

^fThe question for the three refusal skills was: ‘Imagine yourself in each of the following situations. Some of them may be very familiar to you, some others less, so that you may feel less secure in answering. It is enough you do your best. Mark the answer that is closest to your opinion’. For marijuana, the statement was: ‘You and your best friend are at a party where you meet new people, and you feel you really want to get to know them. Someone offers you to smoke marijuana together. Your friend accepts. Do you?’.

^gFor tobacco, the statement was: ‘You and the same friend are studying hard for an important test at school the day after. Both of you feel stressed and need to calm down. Your friend suggests a cigarette would help, and offers one. Do you accept?’.

^hFor alcohol, the statement was: ‘The day after, you both pass the test, and feel now it is time to celebrate. Have still some pocket-money left (R$), and the liquor store is nearby. Would you buy some alcohol to celebrate?’.

Table 4 shows the outcomes from the multilevel mixed-effect models, with multiple imputation, evaluating the short-term effects of the #Tamojunto2.0 program on drug knowledge, psychosocial constructs, and life skill competencies. The program increased 0.26 points in the drug knowledge scale, and increased 0.24 points the negative/non-positive beliefs on alcohol. It was found no statistically significant difference between groups on marijuana beliefs, attitudes, decision-making skills, and refusal skills.

Table 4. Outcomes from multilevel mixed-effect models, imputed through multiple imputation, evaluating the #Tamojunto2.0 program short-term effect on drug knowledge, psychosocial constructs, and life skill competencies according to the intention-to-treat paradigm, 2019 (N = 5208).

Continuous response variable^a	Time			Group			Interaction (time × group)
	Coef.	95% CI	p Value	Coef.	95% CI	p Value	Coef.	95% CI	p Value
Drug knowledge^b	>−0.001	(−0.06; 0.06)	0.975	−0.03	(−0.11; 0.05)	0.426	0.26	(0.17; 0.34)	<0.001
Behavioral beliefs
On alcohol^b	0.07	(−0.06; 0.20)	0.293	0.11	(−0.07; 0.29)	0.241	0.24	(0.05; 0.42)	0.012
On marijuana^b	−0.11	(−0.25; 0.20)	0.095	0.09	(−0.08; 0.27)	0.289	0.11	(−0.06; 0.28)	0.210
Attitudes^b	−0.27	(−0.39; −0.16)	<0.001	0.14	(−0.02; 0.30)	0.082	−0.02	(−0.17; 0.14)	0.817
Decision making skills^b	−0.10	(−0.19; <0.01)	0.053	0.05	(−0.06; 0.16)	0.376	0.02	(−0.10; 0.14)	0.782
	Time			Group			Interaction (time × group)
Dichotomous response variable^a	OR	95% CI	p Value	OR	95% CI	p Value	OR	95% CI	p Value
Refusal skills
To accept marijuana^c	1.13	(0.86; 1.40)	0.320	0.85	(0.58; 1.13)	0.328	1.38	(0.88; 1.88)	0.087
To accept tobacco^c	1.18	(0.86; 1.50)	0.242	0.84	(0.57; 1.12)	0.299	1.34	(0.82; 1.85)	0.138
To accept alcohol^c	1.93	(1.50; 2.35)	<0.001	0.81	(0.59; 1.03)	0.130	0.99	(0.72; 1.27)	0.948

Coef.: coefficient; OR: odds ratio; 95% CI: 95% confidence intervals.

^aAll models were estimated clustering in the school and individual levels and adjusted by sex, age, socioeconomic status, and city.

^bMultilevel mixed-effects linear regression.

^cMultilevel mixed-effects logistic regression.

Table S1 presents the attrition analysis. We compared the students who were able to be linked at baseline and follow-up with those who were lost to follow-up, both for sociodemographic covariates and response variables. Considering the differences in retention between the groups, the intervention group had slightly more losses (50.84% of the total losses) compared with the control group (49.16%). Students lost to follow-up were older; were from a lower socioeconomic status; were mostly from Fortaleza and São Paulo; and on average had more knowledge about drugs, fewer negative beliefs about alcohol and marijuana use, fewer negative attitudes about drug use, and fewer good decision-making capacity; and presented higher percentages for accepting marijuana, tobacco, and alcohol.

4. Discussion

This study evaluated the effect of #Tamojunto2.0, a Brazilian school-based program for drug use prevention in adolescents on secondary outcomes (drug knowledge, psychosocial constructs, and life skills competencies variables) according to the theoretical framework and logical model. Statistical analysis showed that #Tamojuunto2.0 produced the expected effect on drug knowledge and alcohol beliefs. That is, the intervention group had increased drug knowledge and negative/non-positive beliefs about alcohol compared with the control group nine months after intervention. No evidence was found regarding the program’s effect on marijuana beliefs, attitudes, decision-making skills, and refusal skills.

The increase in drug knowledge, as indicated in other studies, is a good indicator that the application of the program’s curriculum is being developed in accordance with the program’s logical framework, at least in terms of this variable. However, this increase in knowledge does not necessarily translate to the non-use of drugs (Newton et al., 2018; Vigna‐Taglianti et al., 2019).

Regarding the effect of the program on increasing negative/non-positive beliefs on alcohol use, our result is consistent with the finding on the effectiveness of program’s effect on alcohol onset: the intervention group had fewer chances to initiate alcohol use compared with the control group (OR = 0.782; 95%CI = 0.636–0.961) (Sanchez et al., 2021). This positive result could have the potential to prevent alcohol consumption in the target population, and its negative effects on their health, particularly in their neurocognitive development and in their future life styles as adults (Lees et al., 2020; Liang & Chikritzhs, 2015).

Regarding marijuana beliefs, we found no evidence that the program was effective in increasing negative/non-positive beliefs about marijuana use, nor did Sanchez and colleagues find evidence that the program prevents its use (Sanchez et al., 2021). Nevertheless, impacting beliefs about the consequences of using alcohol could have a potential impact on preventing polydrug use, especially the combined use of alcohol and marijuana and tobacco, since there is evidence that shows alcohol as a ‘gateway’ to the use of other substances, both legal and illegal use (Kandel et al., 2006; Kandel & Kandel, 2015; Kelly et al., 2015; Linden-Carmichael et al., 2019; Newton et al., 2018 ). For example, in a study on polydrug use conducted in 2013 in young Brazilian adults, it was observed that 37% of them reported simultaneous use of alcohol and: marijuana (79%), hallucinogens (33%), and cocaine (22%); and 26% reported concurrent use of alcohol and: marijuana (47%), amphetamines (28%), and inhalants (23%) (Oliveira et al., 2013). Regarding the combined use of alcohol and tobacco, it is well established in the scientific literature that those who smoke are more likely to drink and those who drink are more likely to smoke (Bobo & Husten, 2000). However, the relationship between increased negative alcohol use beliefs and their possible impact on the prevention of alcohol and other substance use will only be possible to assess with a subsequent mediation study and long-term data.

On the other hand, it appears that the modifications made to the #Tamojunto2.0 curriculum regarding the reinstatement of the alcohol use prevention perspective, in line with the original Unplugged logical framework, were appropriate (Sanchez et al., 2019b). According to the positive results presented in this study and those shown by Sanchez et al. (2021), the iatrogenic effect on first-time alcohol use observed in #Tamojunto was reversed. This could suggest that the harm reduction perspective on alcohol use adopted in the redesign of #Tamojunto may have produced such a negative effect (Pedroso & Hamann, 2019). This may be explained by the way in which adolescents are facilitated with the knowledge and skills to make healthy choices and reduce risky situations. In this sense, it could be hypothesized that, for the adolescent population, it seems to be more effective to talk to them about the negative consequences that alcohol has on their development in order to prevent or delay the consumption of this substance (Winters & Arria, 2011). However, this hypothesis can be evaluated through a subsequent mediation study. An interesting fact in this regard is that the effectiveness of the program on secondary outcomes among the European population differs from the Brazilian population, since it was observed that the direct effect of the program on alcohol use prevention was through decreasing positive attitudes towards drugs, increasing refusal skills and adjusting perceptions of tobacco and marijuana use among peers (Giannotta et al., 2014). In the same way, for the implementation of Unplugged in Nigeria (F. Vigna-Taglianti et al., 2021), alcohol use prevention occurred through increasing negative beliefs about alcohol, as observed in our results, but also through the improving of risk perception and class climate. These results suggest that mediating mechanisms could vary depending on contextual characteristics of the target population.

This study has several strengths. It adopted an experimental design in which the causal line of the program’s effect on IVs could be explored with a large sample size. These IVs are little explored when evaluating the effect of prevention programs, so the present study contributes to the scarce scientific literature on this topic. Finally, because we used instruments previously applied in the evaluations of #Tamojunto and Unplugged, our findings are comparable with the existing literature.

The main limitation is the use of short-term post-intervention measurements, which impeded our exploration of the effects of the program in the long term. A third wave of measurements was canceled owing to the Covid-19 pandemic. This limits a mediation analysis, considering that, ideally, three measurements are required for statistical adequacy; however, as mentioned above, according to methodological recommendations for program effectiveness evaluations, it is preferable to analyze IV immediately after the intervention and, in the long term, to evaluate the indirect (or mediating) effect of these variables on the primary outcomes (Wholey et al., 2010). On the other hand, it is important to indicate the lack of fidelity information in our study, due to it might explain the limited results on IVs. Nevertheless, since this is a study in which the impact of an intervention is observed in real life, unpredictable events affecting both the dose and fidelity of the intervention are expected to occur.

This study has implications for the dissemination of the program as public policy by the BMH. The findings suggest that #Tamojunto2.0 achieved its goals in at least two secondary outcomes: drug knowledge and negative/non-positive alcohol beliefs. These results are encouraging, since other programs based on psychosocial theories and NIDA principles, have managed to modify their secondary outcomes and, consequently, have shown effectiveness in preventing adolescent drug use (Guo et al., 2015; Huang et al., 2012; Orlando et al., 2005). However, to execute the program as public policy, the authorities must identify potential implementation problems, execute strategies to improve program implementation (perhaps including a monitoring process during program implementation), and perform more research to observe the long-term effects of the program on its primary and secondary outcomes. In fact, regarding training quality and monitoring, in a recent process implementation evaluation of #Tamojunto2.0, it was found a need to invest in the training quality because teachers tend to apply the program’s curricula better when they have received qualified technical support in both their initial training and throughout the application. Particularly, the authors identified that teachers feel insecure to address the issue of drugs with their students, indicating the need for continued training (Melo et al., 2022).

Author Contributions

The submitted manuscript has been read and approved by all authors. All authors acknowledge that they have exercised due care in ensuring the integrity of the work. RGC was responsible for drafting all sections of the article and literature review. JYV assisted the statistical analysis and its interpretation, and reviewed the full article. ZMS designed the study, wrote the grant protocol and was responsible for the final approval of the version to be published.

Acknowledgments

The authors are thankful for the technical team from the Ministry of Health, the State and Municipal Secretariats of Health and Education and all the teachers and adolescents who participated in the study. Also, the authors thank Editage (www.editage.com) for English language editing.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This project is part of the Research and Innovation grant for ‘Pesquisas e Inovações em Prevenção de Transtornos Mentais e Uso de Álcool e Outras Drogas’ (Prevention of Mental Health Disorders and Use of Alcohol and Drugs) funded by the Brazilian Ministry of Health [TED #176/2017]. RGC is thankful to the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) PhD scholarship [process number 140272/2019-4]. This study is part of the Center for Research and Innovation in Prevention of Mental Disorders and Drug Use (CEPIPREV) at the Universidade Federal de São Paulo that is funded by the Brazilian Ministry of Health. The study was registered in the Brazilian Registry of Clinical Trials (REBEC) (protocol number RBR-8cnkwq).