Effectiveness of Specific Techniques in Behavioral Teacher Training for Childhood ADHD: A Randomized Controlled Microtrial

ABSTRACT

Objective: Behavioral teacher training is the most effective classroom-based intervention for children with attention-deficit/hyperactivity disorder (ADHD). However, it is currently unknown which components of this intervention add to its effectiveness and for whom these are effective.

Method: In this microtrial, teachers of 90 children with impairing levels of ADHD symptoms (6–12 years) were randomly assigned to one of three conditions: a short (2 sessions), individualized intervention consisting of either (A) antecedent-based techniques (stimulus control), (B) consequent-based techniques (contingency management) or (C) waitlist. Primary outcome was the average of five daily assessments of four individualized problem behaviors, assessed pre and post intervention and three months later. Moderation analyses were conducted to generate hypotheses on child, teacher and classroom factors that may contribute to technique effectiveness.

Results: Multilevel analyses showed that both antecedent- and consequent-based techniques were equally and highly effective in reducing problem behaviors compared to the control condition (Cohen’s d =.9); effects remained stable up to three months later. Child’s age and class size were moderators of technique effectiveness. For younger children, consequent-based techniques were more effective than antecedent-based techniques, whereas for older children the effect was in the opposite direction. Further, beneficial effects of antecedent-based techniques increased when the number of students per class decreased, whilst effectiveness of consequent-based techniques did not depend on class size.

Conclusions: This study shows that both antecedent- and consequent-based techniques are highly effective in reducing problem behavior of children with ADHD. Interventions may be adapted to the child’s age and class size.

Introduction

Attention-deficit/hyperactivity disorder (ADHD) is characterized by symptoms of inattention, hyperactivity and/or impulsivity (DSM-5; American Psychiatric Association, 2013), often leading to difficulties at school with regard to behavioral problems (e.g., impulsive and oppositional behaviors), academic performance (e.g., lower grades) and social problems (e.g., problems in the interaction with peers) (Daley & Birchwood, 2010; Kirova et al., 2019; Wilens et al., 2002). ADHD is one of the most common childhood psychiatric disorders, with approximately 5% of children worldwide meeting full criteria for the disorder (Polanczyk et al., 2007, 2014), and 10–15% of children experiencing impairing levels of symptoms without meeting full criteria for a diagnosis (Hong et al., 2014; Kirova et al., 2019).

Behavioral teacher training is the most effective non-pharmacological classroom intervention to counteract the symptoms of ADHD and related behavior problems (Daley et al., 2014; Evans et al., 2014, 2018; Fabiano et al., 2009), and to reduce teacher burden and increase levels of teacher self-efficacy (Ross et al., 2012). In behavioral teacher training, teachers are taught to change a child’s behavior by using stimulus control techniques. These techniques aim at changing behaviors by manipulating their antecedents, or stimulus conditions, to increase the chance that a child elicits desired behavior, by establish and strengthen the relation between the stimulus condition and response (i.e., child behavior). Such antecedent-based techniques include providing structure and clear instructions, aimed at clarifying what behavior is expected of a child in a specific situation. In addition, teachers are taught contingency management techniques aimed at changing behaviors by manipulating its consequences. According to the principles of behavioral theory, the frequency of behavior will increase when this is followed by positive consequences (e.g., introducing a reward or taking away a penalty), and decrease or fade out when this is followed by negative consequences (e.g., taking away a reward or introducing a penalty). Examples of such consequent-based techniques are the use of praise, ignoring or mild punishment (DuPaul & Eckert, 1997; Owens et al., 2018). Most current interventions comprises both sets of techniques to reduce children’s problem behavior, with different focus on either predominantly antecedent- or more consequent-based techniques (DuPaul & Eckert, 1997; DuPaul & Stoner, 2003).

Currently, effect sizes for behavioral teacher training interventions are medium at best, and heterogeneity in effect sizes between studies is large (Daley et al., 2014). To increase the effectiveness of behavioral teacher training and to be able to individualize intervention plans, it is of importance to study which intervention components (i.e., sets of techniques) contribute to its effectiveness (e.g., Leijten et al., 2015), and to gain insight into child, teacher and classroom factors that moderate effectiveness. Such studies are currently lacking and are needed to optimize treatment outcome for children with ADHD (DuPaul et al., 2020; Schatz et al., 2020).

According to review studies that compared interventions predominantly consisting of antecedent- or consequent-based techniques, both sets of techniques are effective in reducing problem behavior in children with (symptoms of) ADHD (Gaastra et al., 2016; Harrison et al., 2019). The few studies that focused on effectiveness of specific consequent-based techniques showed that time out from positive reinforcement is effective in reducing disruptive behavior problems in children with ADHD symptoms (Fabiano et al., 2004; Northup et al., 1999; Pfiffner et al., 2006), although a time-out procedure also requires clear rules and therefore also contains (some) antecedent-based techniques. Further, a meta-analysis by Gaastra et al. (2016) including within-subjects design studies, showed that interventions consisting of predominantly consequent-based techniques were significantly more effective (M_SMD = 1.82) than interventions including predominantly antecedent-based techniques (M_SMD = 0.31) in reducing off-task and disruptive classroom behavior of children with ADHD. However, so far the effectiveness of these different sets of techniques in isolation has not been examined in a head-to-head comparison, which is important to rule out nonspecific treatment effects and to be sure that intervention effects are not contaminated by the other component. An experimental method to study and compare the effectiveness of these sets of techniques and investigate moderators of effect is a microtrial (Howe et al., 2010; Leijten et al., 2015).

Microtrials are randomized experiments testing the effects of relatively brief and focused environmental manipulations (i.e., intervention components) (Howe et al., 2010). These experiments can measure whether the environmental manipulations bring about meaningful, immediate, change in specific, proximal outcomes (e.g., change in problem behavior targeted with the intervention component directly after delivering the intervention), rather than distal outcomes (e.g., ADHD symptoms in general) that are measured over a longer period of time (Leijten et al., 2015). Microtrials could therefore bridge the gap between laboratory or field studies and full-scale trials (Howe et al., 2010; Leijten et al., 2015) by gaining knowledge on effective intervention components and the identification of specific subgroups that are more or less responsive. Such knowledge can be used for developing more personalized treatment approaches (e.g., does the focus need to be on more antecedent- or consequent-based techniques in school-based settings?) as well as strengthening existing interventions or the preparation of full-scale trials.

The aim of the present study was to evaluate antecedent- and consequent-based techniques in behavioral teacher training for children with ADHD symptoms. For the purpose of this microtrial, we developed two short behavioral interventions focusing on either antecedent-based (stimulus control) or consequent-based (contingency management) techniques. Based on previous findings (see for reviews: Gaastra et al., 2016; Harrison et al., 2019), we expected that both antecedent- and consequent-based techniques would be effective in reducing problem behaviors on proximal outcomes (i.e., inattention, hyperactive-impulsive and/or oppositional behavior problems targeted in the intervention) compared to a waitlist control group. Although experimental evidence based on randomized controlled trials is lacking, we hypothesized, based on Gaastra et al. (2016), that effects of consequent-based techniques would be larger than effects of antecedent-based techniques. We explored whether several variables moderated the effectiveness of the techniques to generate hypotheses regarding which set of techniques is more effective for which child, teacher and/or class, which is of pivotal importance to tailor future interventions to individual needs. Moderation analyses within microtrials are powerful to detect moderation effects given that only one aspect of an intervention (i.e., set of techniques) is manipulated and its immediate effects are measured on proximal outcomes (Howe & Ridenour, 2019). The moderators of interest were demographic and clinical variables that are commonly available in clinical or school-based practice, such as the age, and severity of problems of the child, as well as sense of efficacy, student-teacher relationship and treatment expectancy of the teacher, and the number of students in class. The teacher and class related moderators have all been found related to the effectiveness of behavioral interventions (Durlak & DuPre, 2008; Finn et al., 2003; Girolametto et al., 2003; Vancraeyveldt et al., 2015; Williford et al., 2015).

Method

Design

This study consisted of two intervention conditions (i.e., antecedent- and consequent-based, see below) and a waitlist control condition, with 30 children and their teachers participating in every condition. One of the authors, who had no contact with the teachers (ML), created a random list of numbers 1–90 to allocate teachers to one of three conditions. Randomization occurred at school level to prevent contamination from teachers receiving an intervention, to teachers in the control condition or to teachers receiving the other intervention (e.g., information drift from a teacher in the antecedent condition to a teacher in the consequent condition, and vice versa). Outcome measures were assessed in all three conditions at three time points: at baseline prior to randomization (T0), during the week immediately after the two intervention sessions, or the two-weeks waiting period (T1), and three weeks after the intervention or waiting period (T2). Given that microtrials are aimed at experimentally assessing immediate effects (Leijten et al., 2015) (T1 and T2), assessments of longer term effects were explorative (follow-up three months after baseline, T3). The total study duration was three months (T0-T3) and allowed no holidays between randomization and T2. In case the summer holiday started prior to T3, T3 took place three weeks prior to the end of the school year (but at least four weeks after T2). Since there are no guidelines for reporting on microtrials, we used the CONSORT guidelines for reporting on randomized controlled trials (Moher et al., 2001).

Participants

Participants were teachers seeking help with the behavior of one of their students showing ADHD symptoms, and were recruited through school principals, educational consulting associations, and an outpatient mental health clinic. Teachers of 90 children from 52 schools aged between 6 and 12 years old participated in this study. Teachers could participate with a maximum of two students. A total of 30 children (from 25 classrooms of 17 schools) were allocated to the antecedent condition, 30 children (from 26 classrooms of 18 schools) were allocated to the consequent condition, and 30 children (from 26 classrooms of 17 schools) were allocated to the control condition (waitlist).

Children attended regular primary education throughout The Netherlands, in both rural and urban areas, and displayed impairing levels of teacher rated ADHD symptoms. Children were included if they: (a) obtained a score > 90^th percentile on the teacher rated Inattention and/or Hyperactivity/Impulsivity scale of the Disruptive Behavior Disorders Rating Scale (DBDRS) (Oosterlaan et al., 2008; Pelham et al., 1992), (b) showed at least three symptoms (Veenman et al., 2016) on the Inattention and/or Hyperactivity/Impulsivity scale of the semi-structured Teacher Telephone Interview (TTI) (Tannock et al., 2002), based on the DSM-IV-TR (American Psychiatric Association, 2000), and (c) obtained a score > 5 (indicating impairment, range 0– 10) on at least one domain of functioning on the teacher rated Impairment Rating Scale (IRS) (Fabiano et al., 2006). For five children inclusion took place before summer holiday. For these children impairment was based on the fact that the teacher was seeking help to cope with the child’s behavior, which was substantiated by TTI scores. Children were excluded if they: (a) had an estimated full scale IQ < 70, estimated using a short form of the Dutch version of the Wechsler Intelligence Scale for Children-third edition (WISC-III-NL) (Wechsler et al., 2005) including the subtests Block Design and Vocabulary (Sattler, 2008), (b) were taking psychotropic medication during the last month, (c) had a diagnosis of autism spectrum disorder or conduct disorder according to the DSM-IV-TR (American Psychiatric Association, 2000) or DSM-5 (American Psychiatric Association, 2013) as reported by parents on a demographic questionnaire, or (d) if the teacher had received a behavioral teacher training aimed at ADHD symptoms or other behavioral problems in the past year. To check for pervasiveness of symptoms in the home situation, parents filled out the Hyperactivity scale of the Strengths and Difficulties Questionnaire (SDQ) at baseline (Goodman, 1997). The flowchart presented in Figure 1 displays the inclusion process of participants.

Figure 1. Consolidated Standards of Reporting Trials (CONSORT) flow diagram of participants during enrollment, allocation, follow-up, and analysis

Analyses were intention-to-treat with n = 30 per condition. DBDRS = Disruptive Behavior Disorder Rating Scale; TTI = Teacher Telephone Interview.

Interventions

Two short interventions were investigated, the antecedent-based intervention using only antecedent-based techniques and the consequent-based intervention using only consequent-based techniques. The interventions consisted of two individual sessions with the teacher by a trained psychologist, the first lasting two hours and taking place at the school, and the second scheduled one week later, lasting 45 minutes, and taking place by video conference. After each session, teachers were instructed to implement the techniques in the classroom for four weeks, and teachers could consult the therapist if required. These protocollized interventions were based on evidence-based behavioral parent training programs (Barkley, 1987; McMahon & Forehand, 2003; van den Hoofdakker et al., 2007) aimed at remediating ADHD and ADHD related behaviors, and consisted of psycho-education on ADHD and training of stimulus control and contingency management techniques.

Common Elements in Both Interventions

Interventions were targeted at four preselected problem behaviors from a list of 32 ADHD-related behaviors (see below), with both sessions targeting one selected problem behavior in one specifically chosen situation (see below) (e.g., difficulties staying focused during individual seatwork). The other two problem behaviors were not directly targeted in the intervention. The first session started with psycho-education about ADHD behaviors. Thereafter, one problem behavior was selected by the teacher in collaboration with the therapist, based on the severity and frequency of occurrence (preferably daily) of the problem behavior. Depending on the intervention condition (see below), the therapist and teacher made a behavioral analysis by identifying (a) antecedents that elicit the problem behavior, or (b) consequences that positively or negatively reinforce the problem behavior (i.e., functional behavior assessment, FBA; Dunlap & Kern, 2018). In the next step they defined desired target behavior. The therapist and teacher made a behavioral intervention plan with antecedent- or consequent-based techniques, depending on the assigned intervention condition. This procedure allowed us to have an individually tailored intervention plan for both problem behaviors of every child. The session ended with practicing the technique(s) through role-play and/or visualization. The teacher implemented the plan in the classroom for one week, after which the second session took place. This session started with evaluating the preceding week and adapting the intervention plan, if necessary. Thereafter, the therapist and teacher selected a second problem behavior occurring in a specific situation, and went through the same steps as in the first session (i.e., from behavioral analysis to practicing). The teacher received handouts of the specific techniques trained.

Antecedent-based Intervention

In the antecedent-based intervention (referred to as antecedent condition), teachers were taught how stimuli evoke behaviors and how executive functioning deficits in children with ADHD may lead to difficulties adapting behavior to stimuli. The therapist and teacher identified which antecedents elicited the selected problem behavior. Thereafter, teachers were taught how antecedent-based techniques may be used prior to the onset of behavior and how to alter stimuli in order to elicit changes in child behaviors (Owens et al., 2018). Given the experimental set-up of the microtrial that was aimed to study specific intervention components, teachers were only taught antecedent-based techniques in the antecedent condition. The following techniques were briefly explained in this intervention: setting clear rules, providing clear instructions, discussing challenging situations with the child in advance, and providing structure in time and space. One or more techniques were chosen to be part of the intervention plan, based on the behavioral analysis. When teachers brought up that they could use techniques from the other condition (e.g., reward desired behavior), the therapists were instructed to respond that that is a known technique, but that the current training focused on implementing antecedent-based techniques first.

As an example, a desired target behavior may be: “This child can work individually for five minutes on the assigned math task, without asking the teacher for help”. In the antecedent condition, the intervention plan may consist of (1) the teacher giving appropriate individual instructions to the child after the class wide instruction; (2) the child having a step-by-step plan with illustrations (i.e., pictograms) on how to proceed the task and what to do when a question arises; and (3) a timer on the child’s desk to show the remaining time.

Consequent-based Intervention

In the consequent-based intervention (referred to as consequent condition), teachers were taught how consequences affect behavior and that children with ADHD may suffer from an altered reward sensitivity that may influence how their behavior is shaped by the environment. The therapist and teacher identified which consequences positively or negatively reinforced or discouraged desired target behavior. Thereafter, teachers were taught how consequent-based techniques may be used following (un)desired behavior to affect the occurrence of specific behavior. The following techniques were briefly explained in this intervention: praise, reward, planned ignoring and negative consequences. When the full desired behavior was not displayed yet by the child, shaping was explained (rewarding of short sequences of the desired behavior) with the aim to elicit the full desired behavior. Consequent-based techniques such as token economy and time-out were not taught in this intervention given that these techniques also require antecedent-based techniques (e.g., clear rules, discussing in advance). As in the antecedent-based intervention, techniques for the intervention plan were chosen based on problem behavior and behavioral analysis, no antecedent-based techniques were taught, and when teachers suggested to add an antecedent-based technique to the intervention plan they were told to focus on consequent-based techniques first.

As an example, a desired target behavior may be: “This child can work individually for five minutes on the assigned math task, without asking the teacher for help”. The intervention plan in the consequent condition may consist of (1) the teacher frequently rewarding the child’s (attempts to display the) desired behavior (e.g., praise or thumbs up when the child is working or quiet); (2) the teacher praising other children who are working on their task; and (3) the teacher ignoring all of the child’s attention-seeking behavior (e.g., raising hand, calling the teacher’s name).

Therapists

Two psychologists with postgraduate training in behavioral therapy and ADHD and trained in using the intervention protocol (AS and RH), carried out the intervention. To assess intervention fidelity, we measured therapists’ adherence to the protocol based on contamination and the percentage addressed session items. Contamination was assessed using the procedures of Abikoff (Abikoff et al., 2013), and defined as therapists’ actions that resulted in the incorporation of features from the non-assigned intervention (e.g., consequent) into the assigned intervention (e.g., antecedent). This could occur either by the therapist recommending the use of non-assigned techniques or the therapist actively supporting and elaborating on the teacher’s suggestions of the use of techniques specific to the non-assigned intervention. The frequency of contamination occurrences in a session served as the outcome. We also assessed the percentage of addressed session items in each session (18 items in session one, 11 items in session two). Furthermore, all intervention sessions were audiotaped. For every therapist, the first session of both conditions were checked on intervention fidelity by two of the authors who are behavior therapists and licensed supervisors in the postgraduate behavior therapy program with ample experience in behavioral parent and teacher training programs (SvdO and BvdH). Fidelity scores were discussed with the therapist during individual supervision sessions. Further, ten percent of the sessions were listened back and scored on intervention fidelity during the study by independent evaluators. After each session, the therapists completed a fidelity checklist in which they were asked which items were covered. During the entire study, therapists and researchers held meetings every two weeks to monitor intervention fidelity and to provide supervision.

Outcome Measures

Daily Measurements of Problem Behaviors

Our primary outcome measure was the assessment of four individual problem behaviors which were derived by teachers from a list of target behaviors (van den Hoofdakker et al., 2007), listing 32 possible problem behaviors related to ADHD, including inattentive, hyperactive and impulsive symptoms, and oppositional defiant behaviors. Reliability of this list in the current sample was excellent (α = .90). The daily assessment of target behaviors is derived from the method used by van den Hoofdakker et al. (2007), and can be considered as ecological momentary assessment procedure (EMA; Bolger et al., 2003; Shiffman et al., 2008). EMA involves repeated assessment of the participant’s behavior in real time and in its natural environment (Bentley et al., 2019), therefore minimizing bias and maximizing ecological validity. Such ratings of behaviors combine the advantages of direct observation and the efficiency of questionnaires (Chafouleas et al., 2009), are easy to administer, and can be used in multiple settings and for multiple behaviors (Miller & Fabiano, 2017).

Teachers rated whether the 32 problem behaviors occurred daily (yes, no), and if yes, they scored the severity on a 5-point Likert scale ranging from 1 (not severe) to 5 (exceptionally severe). Examples of problem behaviors are: “difficulties staying focused”, “often loses things necessary to work”, “talking excessively”, and “being noncompliant”. From the problem behaviors that occurred daily, teachers selected four problem behaviors they wanted to address during the intervention. Thereafter, for all four problem behaviors, teachers selected a situation in which this behavior occurred most, or was most severe. Situations were derived from the School Situations Questionnaire (SSQ, DuPaul & Barkley, 1992). Examples of situations are: “individual seatwork”, “whole class group teaching”, and “at recess”. These four problem behaviors in specific situations were assessed by daily, brief (one to two-minutes) and protocollized phone calls on five consecutive days of a school week at every time point. Trained graduate students, who were masked to the type of intervention condition, made the phone calls. In these phone calls, the researcher repeated the four problem behaviors and teachers rated, for every problem behavior, (a) if the behavior occurred that day and if yes, (b) the severity of each of the observed problem behaviors on a 5-point Likert scale. The mean score over four problem behaviors on five consecutive days served as our dependent variable. To motivate teachers to implement the techniques, at the end of the phone call, teachers in the intervention conditions were asked whether they implemented the intervention plan during that day.

Potential Moderators

We tested several child, teacher and classroom variables as potential moderators. We assessed the following child related factors: age, sex, IQ, ADHD symptom severity, ODD symptom severity (TTI, Tannock et al., 2002), school impairment as rated by teachers (i.e., number of domains impaired, range 1– 5) (IRS, Fabiano et al., 2006), and daily ratings of problem behaviors at baseline (i.e., four behaviors measured on five days). The following teacher and classroom related factors were tested: self-esteem (Rosenberg Self-Esteem Scale, RSES, Rosenberg, 1965) (range total score: 10– 40), sense of efficacy (Teachers’ Sense of Efficacy Scale – short version, TSES, Tschannen-Moran & Hoy, 2001) (range mean: 1– 9), perception of the student-teacher relationship (Student-Teacher Relationship Scale – short version, STRS, Koomen et al., 2012; Pianta, 2001) (range total score: 15– 75), treatment expectancy of the teacher (Credibility/Expectancy Questionnaire, CEQ, Devilly & Borkovec, 2000) (range total score: 0– 30), and class size (i.e., number of students). For a detailed description of the instruments included in moderation analyses see Supplementary Material S1.

Procedure

Teachers interested in participating received an information letter explaining the research aims and responsibilities of all parties involved. They enlisted one to two children showing profound and impairing ADHD symptoms in the classroom, and informed parents about the study (i.e., provided them with the information letter and informed consent). Teachers, parents, and children older than 11 years, provided consent. After receiving consent, teachers were administered the DBDRS and TTI to screen for eligibility. Baseline assessments took place assessing the child’s behavior during a period of five consecutive schooldays (T0). After the baseline assessments were completed, children were randomized to one of the three conditions. To prevent information drift between teachers, randomization occurred at school level (see Design) and after randomization, no new teachers could enter the study. Teachers of the children either received the two-week lasting antecedent- or consequent-based intervention, or were assigned to the two-weeks control condition (waitlist). Teachers of children in the control condition were allowed to receive care as usual during the study period, and were offered the possibility to use a self-directed behavioral teacher program targeting ADHD symptoms immediately after T2 (PR Program, Veenman et al., 2016). Longer term effects were therefore only explored in children of teachers in the active intervention arms. There was no financial compensation for participating in the study. The study procedure was approved by the local medical ethical committee (University Medical Center Groningen, 2016/198) and was carried out between April 2017 and April 2019.

Statistical Analysis

Data were analyzed on an intention-to-treat basis. Analysis of variance (ANOVA), and chi-squared or Fisher’s exact tests were used to compare groups on the demographic variables assessed at baseline.

Effects of Techniques

Multilevel analyses (mixed model) were conducted in Stata (version 16), to compare the intervention conditions to the control condition. Four hierarchical levels were distinguished: observations (level 1), nested within children (level 2), nested in classrooms (level 3), and nested in schools (level 4). Random intercepts at classroom and school level were only included if significantly improving model fit as determined by Likelihood Ratio Test. We inserted condition (control, antecedent, consequent) as between subjects factor, and time (T1, T2) as within subjects variable. Baseline scores (T0) of the outcome were inserted as fixed factor, in order to control for problems at baseline. We investigated effects of condition (averaged over T1 and T2) to compare the intervention conditions to the control condition, and to compare the two intervention conditions to each other. We added the condition by time interaction to the model to test whether there were differences between conditions in development of problem behaviors on specific time points. Additional within conditions analyses were conducted to test when a significant change in problem behaviors occurred. Longer term effects were assessed by examining whether problem behaviors remained stable from T2 to T3 within each intervention condition (i.e., whether the development of problem behaviors from T2 to T3 changed significantly). We also checked whether effects were different for the two behaviors targeted in the intervention sessions, compared to effects averaged over four problem behaviors per child. Effect sizes (Cohen’s d) were calculated by dividing the difference in mean scores between two conditions averaged over T1 and T2 (short term effects) by the pooled SD (Rosnow & Rosenthal, 1996), with .20, .50, and .80 as thresholds for small, medium, and large effects, respectively (Cohen, 2013).

Sensitivity analyses were conducted to examine whether intervention effects differed between type of problem behaviors (i.e., inattentive, hyperactive-impulsive or oppositional defiant behaviors), and whether intervention effects were different for children with and without a diagnosis of ADHD. Finally, if children started pharmacological treatment after baseline, sensitivity analyses were conducted without these participants to check if this might have influenced the results.

Moderators

We explored whether child, teacher and classroom related factors moderated intervention effects by adding interactions between the intervention condition and the potential moderator to the multilevel model. We examined whether a factor significantly moderated the intervention effect (averaged over T1 and T2) when comparing both intervention conditions with each other (i.e., antecedent versus consequent condition).

Intervention Fidelity

By using independent t-tests, the contamination scores and percentages of addressed session items were compared between intervention conditions.

Results

Table 1 displays background characteristics of the sample. Conditions did not differ on any of the screening characteristics (p > .132), with the exception of hyperactivity-impulsivity symptoms on the TTI and DBDRS. In the consequent condition three children had a comorbid learning disorder.

Table 1. Background characteristics of the sample

	Antecedent (n = 30)	Consequent (n = 30)	Control (n = 30)	Group comparisons
Child factors Age in years, M (SD)	8.53 (1.63)	9.08 (1.63)	8.76 (1.52)	F(2, 89) =.88, p =.420
Sex, n (%) boys	23 (77)	23 (77)	28 (93)	χ^2 = 3.81, p =.150
Total IQ, M (SD)	99.77 (11.04)	99.33 (14.28)	104.07 (10.05)	F(2, 89) = 1.45, p =.241
SES^a	5.22 (1.24)	5.24 (1.12)	5.00 (1.03)	F(2, 88) =.41, p =.664
Caucasian, n (%)	28 (93)	27 (90)	30 (100)	Fisher’s exact =.294, p =.363
ADHD diagnosis, n (%)	8 (27)	8 (27)	7 (23)	χ^2 =.12, p =.943
Other psychiatric diagnosis, n (%)	0 (0)	3 (10)	0 (0)	Fisher’s exact =.422, p =.104
TTI symptom severity, M (SD) Inattention Hyperactivity-Impulsivity ODD CD	4.30 (1.58) 2.97 (1.85) 1.10 (1.45) .00 (.00)	5.00 (1.86) 4.83 (2.38) 1.76(1.46) .00 (.00)	4.13 (1.91) 4.60 (2.22) 1.23 (1.61) .13 (.51)	F(2, 89) = 1.99, p =.143 F(2, 89) = 6.65, p =.002 (AC < CC, WC) F(2, 89) =.10, p =.903 F(2, 89) = 2.07, p =.132
DBDRS, M (SD) norm scores Inattention Hyperactivity-Impulsivity	15.60 (1.32) 14.56 (1.78)	15.60 (1.00) 15.10 (1.42)	15.31 (1.31) 15.66 (1.42)	F(2, 88) =.55, p =.580 F(2, 88) = 3.64, p =.030 (AC < WC)
IRS impairment, M (SD) Number of domains Average score	3.07 (.98) 6.22 (1.65)	2.97 (1.27) 6.14 (1.97)	3.24 (.88) 6.29 (1.28)	F(2, 84) =.45, p =.638 F(2, 84) =.52, p =.948
Daily ratings of problem behavior at T0, M (SD)	2.15 (.79)	2.51 (.93)	2.80 (.75)	F(2, 89) = 4.63, p =.012 (AC < WC)
Teacher factors Self-esteem, M (SD) Sense of efficacy, M (SD) Student-teacher relationship, M (SD) Treatment expectancy, M (SD)	34.47 (3.87) 7.54 (.56) 48.50 (10.30) 15.57 (3.21)	34.80 (4.02) 7.40 (.73) 51.67 (9.90)b 16.61 (3.70)	32.77 (3.78) 7.33 (.54) 47.93 (7.96) 17.04 (3.98)	F(2, 89) = 2.36, p =.101 F(2, 89) =.89, p =.416 F(2, 89) = 1.36, p =.261 F(2, 88) = 1.29, p =.282
Classroom factors
Class size	24.40 (4.52)	24.17 (3.96)	23.78 (4.04)	F(2, 89) = 1.45, p =.241

AC = antecedent condition; ADHD = attention-deficit/hyperactivity disorder; CC = consequent condition; CD = conduct disorder; DBDRS = Disruptive Behavior Disorder Rating Scale; LD = learning disorder; ODD = oppositional defiant disorder; SES = socioeconomic status; TTI = Teacher Telephone Interview; WC = waitlist control condition.

^aSES was measured by parental educational level (average of both parents) through the Dutch classification system (1 = no education completed, 2 = early childhood education, 3 = primary education, 4 = lower secondary education, 5 = upper secondary education, 6 = undergraduate school, 7 = graduate school, 8 = post-graduate education) (CBS, 2016).

^b Ratings of one teacher are missing.

In total, 23 children (26%) had a diagnosis of ADHD as indicated on a demographic questionnaire by parents. None of the parents indicated that children had a diagnosis of ODD. Based on parental report on the Hyperactivity scale of the SDQ, 90% (n = 78) of the parents scored the symptoms of their children in the clinical range (Maurice-Stam et al., 2018).

Teachers had a mean age of 38.3 years (SD = 12.0), 91% was female and teachers on average had 14.4 years (SD = 11.3) of teaching experience. There were no differences between conditions (p = .418, p = .722, p = .053, respectively).

Intervention Fidelity

Therapist Reported Fidelity

Therapists’ reports of fidelity showed that on average respectively 98.9% and 99.4% (SD = 2.26, N = 29; SD = 1.37, N = 30) of the session items were carried out in the sessions of the antecedent and consequent condition. Fidelity did not differ between the two intervention conditions (t(45.77) = −1.13, p = .266).

Recorded Fidelity

Based on the recorded sessions, average protocol adherence was high in both conditions (antecedent = 98.0%, consequent = 97.8%). Contamination occurred once in a session of the consequent-based intervention and was not scored in any of the antecedent-based sessions. Contamination did not differ between the two interventions (t(3.00) = −1.00, p = .391).

Effects of Techniques

Results on the daily measurements of the four problem behaviors are depicted in Figure 2 and Table 2 (means and standard deviations at all time points are reported in Supplementary Material S2, Table A). For all outcomes, the random intercepts at levels “school” and “classroom” did not significantly improve the models, so therefore all models were reduced to two levels (observations clustered in children).

Table 2. Effects of condition on different time points

	T0 – T1/T2		T0 vs T1^a		T0 vs T2^a		T2 vs T3^a
	B (SE)	d (95% CI)	B (SE)	d (95% CI)	B (SE)	d (95% CI)	B (SE)	d (95% CI)
Antecedent vs control	−.89 (.17)*	.89 (.25– 1.53)	−.96 (.20)*	.99 (.35– 1.63)	−.80 (.20)*	.93 (.29– 1.57)	-	-
Consequent vs control	−.92 (.16)*	.93 (.29– 1.57)	−.90 (.20)*	.77 (.14– 1.40)	−.95 (.19)*	.92 (.28– 1.56)	-	-
Antecedent vs consequent	.04 (.16)	.04 (−.58 –.66)	−.06 (19)	.06 (−.56 –.68)	.15 (.19)	.14 (−.48 –.76)	−.03 (.22)	.30 (−.32 –.92)

*p <.001

^aEffect sizes were calculated on the SD of the specific time point.

Figure 2. Observed values for the development over time in the three conditions. Scores are means across four problem behaviors measured on five consecutive days

Error bars represent 95% confidence intervals.

Figure 2 shows the development of problem behaviors over time for the three conditions. After controlling for baseline problems, teachers in both the antecedent and consequent condition reported significant reductions in problem behavior of their student(s) compared to teachers in the control condition, with large effect sizes (see Table 2). There were no significant differences in the reduction of the child’s problem behavior between the two intervention conditions up to T2. Further, effects for both intervention conditions were similar when only the two problem behaviors that were targeted in the intervention were included in the analyses and compared to the control condition: for antecedent: B = −.92, SE = .18, p < .001, d = .86; for consequent: B = −1.04, SE = .18, p < .001, d = .98.

Additional within condition analyses in the antecedent condition showed that problem behaviors decreased significantly from T0 to T1 (B = −.44, SE = .14, p = .002), and remained stable from T1 to T2 (B = −.02, SE = .14, p = .866). For the consequent condition, results showed the same pattern: problem behaviors significantly decreased from T0 to T1 (B = −.56, SE = .14, p < .001), and remained stable from T1 to T2 (B = −.22, SE = .14, p = .133). In the control condition, problem behaviors did not significantly differ between the time points (for T0 to T1: B = .21, SE = .14, p = .146; for T1 to T2: B = .03, SE = .14, p = .842).

Further, within condition analyses assessing longer term effects revealed that problem behavior of children in the intervention conditions remained stable from T2 to T3 (for antecedent: B = −.09, SE = .15, p = .562; for consequent: B = .07, SE = .16, p = .649) (see Figure 2 and Table 2), indicating that the intervention effects persisted up to three months after the start of the intervention. Completers and non-completers of T3 did not differ on any of the baseline characteristics (p > .084). At T3 (N = 53), the majority of the teachers reported to still use the techniques (n = 22 [81%] in antecedent condition; n = 21 [81%] in consequent condition).

Sensitivity Analyses

Sensitivity analyses showed that effect sizes were large for inattentive behaviors (for the antecedent condition [n = 26] versus control condition [n = 19]: B = −.80, SE = .24, p = .001, d = .74, and for the consequent condition [n = 27] versus control condition: B = −1.02, SE = .23, p < .001, d = .95), and oppositional defiant behaviors (for antecedent [n = 19] versus control [n = 19]: B = −.96, SE = .25, p < .001, d = .79; for consequent [n = 12] versus control: B = −.99, SE = .28, p < .001, d = .82). For problem behaviors related to hyperactivity/impulsivity, effect sizes were medium (for antecedent condition [n = 21] versus control condition [n = 28]: B = −.65, SE = .25, p = .009, d = .56; for consequent condition [n = 24] versus control condition: B = .58, SE = .23, p = .011, d = .50).

Sensitivity analyses with the children without a diagnosis of ADHD (n = 67) revealed that results for problem behaviors remained unchanged (from T0 to T2 for antecedent versus control: B = −.82, SE = .17, p < .001, d = .87; for consequent versus control: B = −.93, SE = .16, p < .001, d = .72). Again, there were no differences between the antecedent and consequent condition (B = −.14, SE = .16, p = .385, d = .15).

One child in the consequent condition started pharmacological treatment (methylphenidate) after T0, and five more children started with methylphenidate after T2 (antecedent n = 3, consequent n = 2). To check whether starting with pharmacological treatment affected the results, multilevel analyses were repeated without the children who started medication. Results remained unchanged from T0 to T2 (for antecedent versus control: B = -.87, SE = .16, p < .001, d = .87; for consequent versus control: B = −.85, SE = .15, p < .001, d = .86; for antecedent versus consequent: B = −.01, SE = .15, p = .935, d = .01). Analyses within the intervention conditions also revealed that excluding children who started medication did not affect intervention effects from T2 to T3 (for antecedent: B = −.04, SE = .16, p = .824; for consequent: B = .08, SE = .17, p = .644).

Moderators of Technique Effectiveness

The correlation matrix of the assessed moderators and outcome variable (daily ratings of problem behaviors at T0) is depicted in Supplementary Material S3 (Table B). Planned moderating analyses with sex could not be performed because there were not enough girls in our sample, but sensitivity analyses without girls revealed approximately similar results (from T0 to T2 for antecedent versus control: B = −.77, SE = .18, p < .001, d = .78; for consequent versus control: B = −.80, SE = .17, p < .001, d = .80).

For age, there was an intervention condition (antecedent versus consequent) by age interaction (see Table 3). Further analyses based on median split analyses (median = 8.5 years) showed that for younger children (< 8.5 years) the consequent-based intervention was significantly more effective than the antecedent-based intervention (B = .50, SE = .23, p = .030, d = .50), although the intervention was effective for both age groups. For older children (> 8.5 years), the effect was in the opposite direction: the antecedent-based intervention was significantly more effective than the consequent-based intervention (B = −.49, SE = .20, p = .013, d = .50), with again, the intervention being effective for both groups.

Table 3. Results of moderator analyses (factor by intervention interaction)

	Antecedent vs. consequent
	B (SE)	p
Child factors
Age	−.25 (.01)	.007
IQ	−.00 (.01)	.817
ADHD symptoms (TTI)	.02 (.06)	.674
ODD symptoms (TTI)	.02 (.11)	.848
School impairment (number of domains, IRS)	.11 (.15)	.472
Daily ratings of problem behaviors at T0	.25 (.18)	.164
Teacher factors
Self-esteem (RSES)	.04 (.04)	.293
Sense of efficacy (TSES)	−.30 (.25)	.244
Student-teacher relationship (STRS)	−.01 (.02)	.693
Treatment expectancy (CEQ)	.02 (.05)	.601
Classroom factors
Class size	.08 (.04)	.037

Regression coefficients of the condition by moderator interaction are depicted. ADHD = attention-deficit/hyperactivity disorder; CEQ = Credibility/Expectancy Questionnaire; IRS = Impairment Rating Scale; ODD = oppositional defiant disorder; RSES = Rosenberg Self-Esteem Scale; STRS = Student-Teacher Relationship Scale; TSES = Teachers’ Sense of Efficacy Scale; TTI = Teacher Telephone Interview.

For the classroom factor (i.e., class size: number of students, range 14 to 32, median = 24), the antecedent versus consequent comparison revealed a significant moderating effect for class size, showing that within the antecedent condition, the intervention effect was larger when the number of students was smaller (B = .07, SE = .02, p < .001), whereas in the consequent condition the intervention effect was unrelated to the number of students (B = −.01, SE = .03, p = .700), see Table 3.

No other significant moderator effects were found, see Table 3.

Discussion

The present study examined the effectiveness of antecedent- and consequent-based techniques in behavioral teacher training for children with impairing levels of ADHD symptoms, using a microtrial. Our results indicate that the antecedent- and consequent-based techniques are both highly effective in reducing problem behaviors related to ADHD on the short and longer term (i.e., up to three months later). Effect sizes were large for problem behaviors related to inattention and oppositional defiant symptomatology, and medium-sized for hyperactive-impulsive behavior, with no differences in effect size between both interventions.

Our findings diverge from the meta-analysis by Gaastra et al. (2016) who showed that interventions consisting of predominantly consequent-based techniques were more effective than predominantly antecedent-based interventions. In that meta-analysis, most of the antecedent-based interventions mainly consisted of nonspecific general classroom interventions (e.g., class-wide peer tutoring, music at the background or extended time), in contrast to the consequent-based interventions that were all directly tailored to the behavior of a specific child (e.g., teacher-administered reinforcement, prudent negative consequences). Further, consequent-based interventions often included antecedent-based techniques (e.g., teacher-administered reinforcement included a visible table on the child’s desk to count rewards). Consequent-based interventions included in the meta-analysis were thus potentially more powerful, as reflected in reported larger effects.

The effectiveness of both sets of techniques on reducing problem behaviors was similar to those of full interventions (van den Hoofdakker et al., 2007), and effects were irrespective of diagnostic status of the child. We individually tailored both the antecedent- and consequent-based interventions to the needs of the teacher and the child, and the large effect sizes obtained with both sets of techniques may point to the importance of individual tailoring. Behavioral analysis (i.e., FBA) was used to discover patterns in (problem) behavior, for example, in the consequent condition the techniques were adapted to the function of a child’s target behavior, which may have increased effectiveness (Chronis et al., 2004; Dunlap & Kern, 2018; Ervin et al., 2000; Harrison et al., 2019; Miller & Lee, 2013; Pfiffner et al., 2006). Further, the short and focused nature of our intervention may have added to large effect sizes. Focusing on one behavior and teaching a few techniques may be more feasible for teachers than programs in which multiple techniques are being taught over a longer time frame, especially for overloaded teachers. Providing teachers with a detailed intervention plan that can be implemented directly into the classroom may be more effective compared to general recommendations on parenting strategies for ADHD that are not tailored to individual needs. Further, our dependent variables were those behaviors targeted in the intervention, and this choice may possibly have increased the effects obtained and emphasizes the use of ecologically valid measures. Lastly, training effects remained stable up to three months later, whilst problem behaviors usually are found to increase after treatment is withdrawn (Han & Weiss, 2005), and for microtrials one would generally not expect distal effects (Howe et al., 2010; Leijten et al., 2015). Since the follow-up period of the current study was within a school year, and therefore shorter compared to the follow-up period included in a meta-analysis of parent training studies (Lundahl et al., 2006; Pelham & Fabiano, 2008), this might explain why our effects lasted until follow-up. Further, the relatively simple and individualized intervention plan may be easy to remember and implement, also in the long run.

The second aim of this study was to generate hypotheses about “what works for whom”, by exploring potential moderators of intervention component effects. The results showed that the antecedent- and consequent-based techniques are effective, regardless of IQ, symptom severity of ADHD and ODD, school impairment, and baseline ratings of problem behaviors. Effectiveness was related to the age of the child. For younger children, consequent-based techniques were significantly more effective than antecedent-based techniques, whereas for older children effects of antecedent-based techniques were larger than effects of consequent-based techniques. This is in line with the development of motivational styles. Over time, children develop from more extrinsic to more intrinsic motivational styles, wherefore extrinsic reinforcements may be less powerful in older children (Deci et al., 1991; Evans et al., 2019). Further, teachers of higher grades, in general, provide less structure and have higher expectations of children regarding initiating and maintaining attention for prolonged periods of time (Harrison et al., 2019), whilst planning problems become more prominent when children grow older (Langberg et al., 2013), allowing these children to benefit more from external structure by teachers.

Besides child factors, we explored whether characteristics of the teacher and classroom impacted on technique effectiveness. Teacher’s self-esteem, sense of efficacy, perception of the student-teacher relationship, and treatment expectancy of the teacher did not moderate effects. Although dealing with students with ADHD has been found stressful to teachers (Greene et al., 2002), and student-teacher relationships are often worse for these children (Ewe, 2019), apparently this did not affect the effectiveness of the current interventions. Class size moderated technique effectiveness, with the beneficial effects of the antecedent-based techniques increased when the number of students per class decreased, whereas the effectiveness of the consequent-based techniques did not depend on class size. This finding is in line with the literature on the benefits of low class size for student engagement (see for review Finn et al., 2003), indicating that teachers in smaller classes may have more possibilities to one-on-one interactions with students and provide better student support, which in turn may be related to increased engagement in learning and less behavioral problems (Biddle & Berliner, 2002).

Current findings should be interpreted in light of some limitations. First, behavioral teacher training programs usually consist of a combination of antecedent- and consequent-based techniques and one might argue that they were separated somewhat artificially for the purpose of this microtrial. Teachers were trained specifically in one set of techniques and the sole focus was on training teachers how to implement this set most optimally. Given the short and personalized nature of the intervention, teachers were highly focused on implementing the intervention plan and rarely reported on the (self-initiated) use of techniques of the other set (i.e., condition) during evaluations at the second session. However, we neither tracked the extent to which teachers used techniques of the other condition nor did we track the use of strategies for other students than the target student. Thus, we cannot fully rule out the possibility that training antecedent-based techniques (e.g., providing clear rules) may have elicited (an increase of) the use of consequent-based techniques (e.g., providing more praise to a child as a result of evoking desired behavior) (Sutherland et al., 2002), and vice versa. Nevertheless, the high levels of therapists’ intervention fidelity and our moderation analyses pointing toward specific effects of antecedent-based and consequent-based techniques show that it appeared possible to separate training of specific techniques, and indicate that eliciting the use of other techniques may be a beneficial side effect of the intervention. Second, despite large effect sizes, intervention effects described here were based on teacher informed measures who may have been biased in reporting positive effects (Daley et al., 2014; Sonuga-Barke et al., 2013). However, individual daily rated problem behaviors using EMA procedures, such as used here, may overcome possible recall or memory bias that can be seen in ratings using questionnaires (Chafouleas et al., 2009). Further, researchers who conducted daily phone calls were not masked to whether teachers were in active training condition or control condition, because they asked whether the intervention was implemented that day. However, since phone calls were brief and protocollized, and researchers were not aware in which active condition the teachers were in, researcher bias is unlikely. Future research may include single blinded EMA and/or use blinded observations of the problem behavior to test whether our findings are confirmed by blinded outcomes. Third, although we showed the effectiveness of antecedent- and consequent-based techniques in this study, it is unclear to what extent both sets of techniques add to each other. Single case experimental designs may be used to assess the additional effects of both sets of techniques (Kazdin, 2019). Lastly, moderation analyses were done as a first attempt to study what works for whom. We explored multiple (potential) moderators in a relatively small sample, which generated important hypotheses for future treatment development and research regarding potential effects of age of the child and class size on the effectiveness of the techniques studied. However, due to limited power, weaker moderators may have remained undetected. Further research is needed to replicate these findings and draw more strong conclusions about subgroups that are more or less likely to respond to the different sets of techniques.

Clinical Implications and Future Studies

Although the aim of this study was to examine intervention components rather than developing a new intervention, results showed that the current interventions appeared to be as effective as other teacher training programs in reducing problem behaviors. The current interventions may be optimized by combining both highly effective sets of techniques in one intervention. Given the need for short, brief, and evidence based behavioral teacher interventions (DuPaul et al., 2019; Gaastra et al., 2020), that are preferably individually tailored (Egan et al., 2019), a short and individually tailored approach such as used in the current study seems to meet these demands. Our finding that most of the teachers continued to use the techniques up to three months later, strengthens our idea that the current intervention meets teacher needs. After testing its effectiveness, this optimized intervention with both sets of techniques may be implemented as stand-alone first-line treatment to provide teachers with psycho-education and individually tailored advices on behavioral management for ADHD (NICE Guidelines, Dutch Guidelines; Akwa, 2019; National Collaborating Centre for Mental Health, 2018).

Future studies may take generalization of effects on broader outcomes such as ADHD rating scales or impairment measures into account, albeit reductions in individually chosen problem behavior as reported by the teacher may be most relevant for the teachers themselves (Cuijpers, 2019). Further, it is of interest to examine whether neuropsychological functioning of the child affects the technique effectiveness. We will further investigate this as part of our larger research project.

Conclusion

To conclude, both antecedent- and consequent-based techniques are highly effective in reducing problem behaviors of a diverse and representative sample of primary school-aged Dutch children with impairing levels of ADHD (i.e., subthreshold or clinical ADHD), on the short and longer term. There are some indications that age and class size may affect technique effectiveness, which is important information for treatment development and tailoring. The short and individualized nature of the intervention may have added to its effectiveness. Given the high teacher workload and increase in number of students with ADHD in regular primary education, a short and individualized intervention, such as developed in the current study and including both sets of techniques, may be implemented as stand-alone treatment.

Acknowledgments

We thank all children, parents and teachers for participating in this study, and students for their support in data collection.

Data Availability

The data that support the findings of this study are available from the corresponding author (AS), upon reasonable request.

Disclosure Statement

No potential conflict of interest was reported by the authors.

Supplementary Material

Supplemental material for this article can be accessed online at https://doi.org/10.1080/15374416.2020.1846542.

Additional information

Funding

This work was supported by The Netherlands Organization for Health Research and Development (ZonMw), grant number 729300013.