Evidence Base Update for Psychosocial Treatments for Disruptive Behaviors in Children

Abstract

This article reviews the state of the science on psychosocial treatments for disruptive behaviors in children, as an update to Eyberg, Nelson, and Boggs (2008). We followed procedures for literature searching, study inclusion, and treatment classification as laid out in Southam-Gerow and Prinstein (2014), focusing on treatments for children 12 years of age and younger. Two treatments (group parent behavior therapy, and individual parent behavior therapy with child participation) had sufficient empirical support to be classified as well-established treatments. Thirteen other treatments were classified as probably efficacious. Substantial variability in effectiveness of different programs within the same treatment family has been previously documented; thus, a particular level of evidence might not hold true for every individual program in a treatment family. Systematic investigations of implementation, dissemination, and uptake are needed to ensure that children and families have access to effective treatments. Investigations into how to blend the strengths of the effective approaches into even more effective treatment might also lead to greater impact.

Disruptive behavior disorders (DBDs) are a set of disorders characterized by a range of symptoms that put the individual frequently at odds with peers, family members, and authority figures. The two most common diagnoses for DBDs are oppositional defiant disorder (ODD) and conduct disorder (CD). ODD manifests as a pattern of developmentally inappropriate, negative, aggressive, and defiant behavior that occurs for 6 months or longer (American Psychiatric Association, 2013). CD has a more targeted set of behaviors that consistently ignore the basic rights of others and violate social norms and rules (American Psychiatric Association, 2013). In 2007–08, the National Survey of Children’s Health asked parents whether a doctor or other health care provider had told them that the child had behavioral or conduct problems such as ODD or CD. In 2007–08, 4.6% of children 3–17 years of age (approximately 2.8 million children) had a parent-reported history of a behavioral or conduct problem such as ODD or CD, with an estimated 3.5% (or approximately 2.2 million children) reported as currently having a behavioral or conduct problem (Perou et al., 2013). These data do not include children who have high levels of disruptive behaviors but have not received a DBD diagnosis.

Although some acting out, aggression, defiance, and rule-breaking behaviors are common among typically developing children, especially at younger ages, extreme and persistent disruptive behaviors put children at high risk of impairment and dysfunction in childhood and negative outcomes later in life. ODD tends to occur at younger ages than CD, with an estimated 30% of children diagnosed with ODD later being diagnosed with CD (Connor, 2002; Loeber, Burke, Lahey, Winters, & Zera, 2000). Among those diagnosed with CD, about 40% go on to have antisocial personality or other personality disorders (Zoccolillo, Pickles, Quinton, & Rutter, 1992). In a reanalysis of data from six longitudinal studies from the United States, Canada, and New Zealand, Broidy and colleagues (2003) reported that for boys, aggressive and nonaggressive conduct problems in childhood significantly increased the risk of violent and nonviolent delinquency in adolescence. The same pattern was not confirmed for girls, though the authors noted that girls have much lower rates of delinquent behavior than boys. Based on data from the Inner London longitudinal study, children who had CD at age 10 cost 10 times more in public services through age 28 than children without CD at age 10 (Scott, Knapp, Henderson, & Maughan, 2001).

Given the substantial impact of disruptive behaviors at the level of individual, family, and society, it is important that children receive treatments that have measurable and long-term functional impacts. The American Academy of Child and Adolescent Psychiatry (AACAP) has published psychiatric practice parameters for treatment of ODD and CD, although the parameters for CD have not been revised in nearly two decades (Steiner, 1997). For CD, child and adolescent psychiatrists are advised to treat comorbid disorders; use family interventions such as parent guidance, training, and family therapy; and provide individual or group therapy, with a preference for a combined behavioral and explorative approach. Social skills training is recommended as a supplement, and other interventions with peers, the school, and other community services should be considered as indicated (Steiner, 1997). The practice parameter for ODD, which was supported by AACAP’s highest recommendation of “Minimal Standards,” guides the clinician to consider parent intervention based on one of seven empirically tested behavioral parent therapies, with medication potentially helpful as an adjunct to other treatment (Steiner & Remsing, 2007). Psychosocial treatments are thus recommended for both diagnosed conditions.

Although there is considerable research investigating the effects of psychosocial treatments for DBDs, available systematic summaries of evidence across studies do not include current evidence. Brestan and Eyberg (1998) and Eyberg, Nelson, and Boggs (2008) published evidence reviews of psychosocial treatments for children with DBDs. Both previous reviews reported that programs in the Parent Management Training Oregon model (based on Patterson and Gullion’s 1968 manual Living with Children) reached the level of a well-established treatment. The Incredible Years Parent Training program (e.g., Webster-Stratton, Reid, & Hammond, 2004) was erroneously listed as well-established in 1998 due to a coding error. However, the 2008 review corrected this and rated it as probably efficacious, along with 14 other treatment programs.

This article updates the two previous reviews of evidence-based psychosocial treatments for children with DBDs (Brestan & Eyberg, 1998; Eyberg et al., 2008), focusing on children up to 12 years of age. In addition to including studies published since the 2008 review, this update follows a new directive for Society for Clinical Child and Adolescent Psychology Evidence Base Updates (Southam-Gerow & Prinstein, 2014) to review the evidence on treatment families (e.g., group-delivered parent behavior therapy, individual child behavior therapy) rather than brand-name programs. The formerly used brand-name program approach to an evidence-based review had the advantage of assisting individuals, referring health care providers, or therapists in choosing among the many available treatment programs. The newer, more generic treatment family approach identifies the common factors across treatment programs based on their approach, which can inform larger policy-based decisions about the types of approaches likely to be effective on a broader population scale. Information about the effectiveness of programs in a treatment family may also help achieve the goals of comprehensive geographic coverage and saturation of services, which individual programs have not been able to achieve except in limited state or local areas. As well, when named programs are not available, treatment family descriptions can help parents and referring providers select among available services in their area, choosing the ones consistent with the treatment family or families for which there is strongest evidence. Finally, individual therapists who are not trained in a particular program model can use these treatment family descriptions to identify which of the general treatment approaches in which they have been trained are most likely to be effective for their clients. Thus, evidentiary reviews of treatment families have much to offer the field.

We followed procedures and criteria established in publications by Chambless and colleagues (Chambless et al., 1998; Chambless et al., 1996) for the Society for Clinical Child and Adolescent Psychology and updated by Southam-Gerow and Prinstein (2014). The review protocol involved two steps: (a) determining the body of sufficiently well-conducted studies to be included in the review and (b) aggregating findings from the well-conducted studies to evaluate the level of evidence supporting each treatment family.

METHODS

Literature Search

Construction of the pool of studies to be considered for inclusion began with all studies in the 1998 and 2008 reviews. Following the procedures reported in 2008, we then conducted Medline and PsycINFO searches of peer-reviewed journals from 2007 to October 2016 using search strings with the combination of “oppositional defiant disorder,” “conduct disorder,” “aggression,” “disruptive behavior disorder,” “child behavior disorder,” or “behavior problem” with “treatment” or “therapy,” limiting to journal articles published in English. We additionally conducted the search using the alternate spelling of “behaviour” to ensure that studies of non-U.S. origin were captured. We next also searched the tables of contents from the sets of journals listed in the 2008 review: Behavior Modification, Behaviour Research and Therapy, Behavior Therapy, Child and Family Behavior Therapy, Child Development, Cognitive Therapy and Research, Journal of Clinical Psychology, Journal of Abnormal Child Psychology, Development and Psychopathology, Journal of Applied Behavior Analysis, Journal of Child Psychology and Psychiatry, Journal of Clinical Child and Adolescent Psychology, Journal of Consulting and Clinical Psychology, Journal of Counseling Psychology, Journal of the American Academy of Child and Adolescent Psychiatry, and Prevention Science.

Identification of Eligible Studies

This review focuses on psychosocial treatments for children 12 years of age and younger. The Evidence Base Update for ages 12 and older is presented in a separate review (McCart & Sheidow, 2016). Studies with an upper child age of 13 years or older were included in the present review only if the majority of children were younger than age 13 (e.g., the age range was 3–13, with a mean age of 8 years). All research designs were eligible for inclusion at this stage, as nonrandomized designs or a sizable number of single-group studies could result in classification per the Chambless and colleagues’ (Chambless et al., 1998; Chambless et al., 1996) criteria. Duplicate reports from studies of the same sample were removed from consideration, such that only one report on disruptive behavior outcomes for a particular sample of children was included. Per the Evidence Base Update guidelines (Southam-Gerow et al., 2014), eligible studies were those that investigated the effects of a manualized psychosocial treatment on outcomes of children with identified disruptive behavior problems (based on clear behavioral cutoffs or diagnosis), using reliable and valid measures, with an adequate sample size and appropriate analyses to detect effects.

• To qualify for the manualized treatment requirement, treatment manuals or protocols could take different forms (e.g., manuals for therapists to follow; curriculum materials designed to be self-administered online) but needed to be essentially the same for all participants. Thus, programs that involved designing a treatment plan for a particular child or parent based on a process of family goal-setting and selection of intervention approaches were excluded.

• Each study needed to have applied participant inclusion criteria based on the behaviors specified for change (i.e., oppositional behaviors and/or conduct problems), such as a diagnosis of ODD or CD, exceeding clinical cutoffs for established behavior problem measures, or referral for treatment of conduct problems. As in Eyberg et al. (2008), studies focusing exclusively or mainly on children with attention deficit hyperactivity disorder (ADHD) were excluded (see Evans, Owens, & Bunford, 2014, for the Evidence Base Update on psychosocial treatments for ADHD specifically). Studies that allowed for children with comorbid ADHD were included. Studies that offered treatment generally to parents who had expressed concerns over their child’s behavior, without formal screening or assessment of those children, were excluded. Also excluded were studies with samples specifically of disruptive behaviors comorbid with developmental disabilities.

• The criterion for “reliable and valid” outcome assessments was not previously operationalized by Eyberg and colleagues (Brestan & Eyberg, 1998; Eyberg et al., 2008). In reviewing the current set of studies, all measures of disruptive behaviors had been previously used in a published study, and none raised concerns from the current authors about validity. Most measures were used by multiple eligible studies by different investigators. Thus, no studies were excluded based on this criterion.

• Similarly, no studies were excluded based on sample size alone. Although Eyberg et al. (2008) stated “an arbitrary cutoff” of 12 participants per group, one study included in that review (Christensen, Johnson, Phillips, & Glasgow, 1980) did not reach that criterion. Hence we included all studies regardless of sample size. Analyses were considered appropriate if the study reported conventional statistics used to assess group differences at posttest (e.g., analysis of covariance using pretest scores as covariates, repeated-measures analysis of variance). Analyses that aggregated two or more treatment groups and compared to a no-treatment group, and therefore the results did not necessarily reflect the effect of either single treatment group individually, were not appropriate for inclusion in this review. Analyses comparing those who completed treatment to those who dropped out, or only examined the effects of dosage or participation on outcomes, were also excluded.

Treatment Family Categorization

From the pool of 64 eligible studies, six broad categories of treatment approaches emerged, each of which included variants on whether there were additional participants in treatment and on how treatment was delivered.

The first category is parent behavior therapy. A treatment approach was classified as parent behavior therapy if the intent was to teach the parent(s) to be more effective behavioral reinforcers, in line with a behavioral treatment orientation. This type of therapy typically has a dual focus on (a) strengthening the parent–child relationship so that the child is more motivated to behave in the way that the parent wants, and (b) providing the parents with more effective child behavior management strategies (Hanf, 1969). Treatments of this type are often described as based on social learning principles, that is, that positive reinforcement (attending to desirable behaviors) and withholding positive reinforcement (planned ignoring or time-out for undesirable behaviors) increase the child’s socially acceptable behaviors and decrease aggressive and oppositional behaviors. Relationship-enhancing strategies may include providing the child with positive attention, engaging in joint activities, and communication skills that convey to the child that the parent understands and wants to provide for the child’s needs. Specific child behavior management strategies that are taught might include setting and clearly communicating developmentally appropriate limits and rules, selecting and enforcing effective consequences for difficult behaviors, and preventing misbehavior. Parent behavior therapy can be delivered in groups, to parents individually with or without child participation, and via self-directed methods (e.g., bibliotherapy or computer assisted).

The second category is child behavior therapy. A treatment approach was classified as child behavior therapy if a therapist interacted directly with the child or children to teach appropriate social skills. The theoretic orientation of treatment typically involves cognitive behavioral techniques of helping the child identify and understand their emotions and behavioral triggers, evaluate ambiguous or threatening social situations, and select appropriate behavioral responses. Similar to parent behavior therapy, child behavior therapy is based on social learning principles and positive reinforcement, but the therapist interacts directly with the child. Therapists often use modeling, role-plays, and behavior charts (with or without a token reward system) to teach and reinforce child behaviors. Specific skills taught to the child might include emotion regulation (e.g., relaxation, anger management), perspective taking, conflict resolution, and how to make friends or enter peer group activities. Child behavior therapy can be delivered individually or to groups of children, with or without parent participation.

The third category is teacher training. Teacher training for DBDs also follows behavioral principles but focuses on classroom/group behavior management strategies and (similar to parent behavior therapy), making the teacher a more effective reinforcer of children’s behavior. Classroom behavior management strategies might have included visual cues to children about their behavior (e.g., token reward systems) and preventive strategies (e.g., providing children with forewarnings before activity transitions). This type of training might or might not include helping the teacher to teach social problem-solving skills directly to children.

The fourth category is parent-focused therapy. This classification was used for treatments that focused primarily on parents’ emotions, attitudes, or boundaries. Some were described by their developers as “client-centered” or “emotion-focused,” which target the parents’ emotion awareness and regulation, and attitudes and perceptions about their child, rather than behaviors. The stated goals of these therapies are to address the underlying emotional or psychological issues within the parent, which are then assumed to translate into more positive parenting and fewer child behavior problems. Specific skills to be taught could include emotion regulation (e.g., relaxation, anger management), perspective taking, empathy, and knowledge of and attitudes about children’s behavior. Other parent-focused programs targeted dysfunctional family processes and structures with the goal of reestablishing boundaries that were either too rigid or too enmeshed. Parent-focused therapy can be delivered individually or in groups, with or without child participation.

The fifth category is child-centered play therapy. This classification was used when the therapist meets with the child or children, with the goal of providing a close, supportive relationship for the child. According to developers of this type of program, the goal is for the therapist to provide nondirective positive regard, warmth, and empathy to the child; help the child express feelings; and provide the child a “safe” space through which to explore and work through negative emotions. The assumption is that by consistently providing this nurturing relationship, the child’s behavior will naturally improve. This type of nonbehavioral child therapy typically does not involve the parent(s) but can be delivered individually or to groups of children.

The sixth category is family problem-solving training. This treatment engages the parent(s), child, and siblings in a problem-solving process to build the family’s ability to collaboratively resolve issues resulting from oppositional behavior. The process involves teaching the family how to identify unsolved problems contributing to oppositional behavior, how to prioritize which problems most need addressing, and how to collaboratively resolve the problem.

In addition to these six broad categories, four adjunct modules that have been tested in combination with them were identified. None of these four would be considered a stand-alone psychosocial treatment for DBDs as defined for the purposes of this review. They are included here only as modules that were used or tested as add-ons to the core modes of treatment just presented. By allowing combinations of core treatment modes and adjunct modules to define treatment families, treatments that differed by an adjunct module (e.g., “individual parent behavior therapy with child participation” and “individual parent behavior therapy with child participation + addressing parent mental health needs”; Sanders, Markie-Dadds, Tully, & Bor, 2000) could be reviewed as separate treatment families.

The first module is school consultation. This component involved communication between the therapist and the child’s school to discuss specific behavioral or educational concerns and identify and enact potential solutions.

The second module is addressing parental mental health needs. Several parent behavior therapy programs have sought to enhance effectiveness by the addition of modules to address specific psychological or behavioral issues identified in the parents, such as depression, anger, stress, and partner discord or violence.

The third module is case management. This module incorporated concrete, instrumental assistance or referrals to help families manage challenges with basic necessities (e.g., adequate food, housing, and medical care) and mental health needs.

The fourth module is medication review. This module was described as assessing the appropriateness of medication for the child, weighing the benefits and risks, and prescribing medication if warranted for behavioral issues. Many of the treatments reviewed here included children who were already taking medication for ADHD or other behavioral issues, and a medication review was likely previously conducted for those children. However, a treatment was classified as containing this module only if every child in the treatment group was reviewed for medication as a part of the treatment program.

Using the six core treatment modes, delivery variants on those modes (e.g., individual or group delivery; parent behavior therapy with or without child participation), and the four adjunct modules, 26 distinct treatment families were identified for this review. For 15 of the treatment families, there was only a single treatment modality (e.g., individual child behavior therapy with parent participation). The remaining 11 treatment families were multimodal (e.g., group parent behavior therapy plus group child behavior therapy).

Application of the Evidence-Based Ratings

Two additional study design characteristics had to be classified for each study prior to application of the rating criteria. The first characteristic was the type of comparison group, according to categories outlined in the Evidence Base Update criteria: no-treatment group, waitlist group, placebo group, other treatment, or a well-established treatment. Multiarm studies (e.g., a three-arm study with a no-treatment group, a treatment group, and an alternate treatment group) were eligible for inclusion as specific two-group comparisons (e.g., as a treatment vs. no-treatment study and as a treatment vs. alternate treatment study), and thus a single publication could provide information about more than one type of research design. Within-study comparisons that amounted to dismantling studies (e.g., Treatment A vs. one component of Treatment A), additive studies (e.g., existing disruptive behavior treatment alone vs. existing disruptive behavior treatment plus Treatment A), or comparison of the same treatment delivered in different contexts (e.g., clinic vs. community setting) were excluded, as the Evidence Base Update criteria do not accommodate those study designs. The second study design characteristic was whether random assignment was used. Studies that randomly assigned preexisting groups (e.g., classrooms) were counted as having used random assignment only if statistical analyses accounted for clustering, such as through multilevel modeling. Studies that randomly assigned individuals and treated in a group format were not required to account for clustering in statistical analyses.

Finally, for each eligible comparison within a study (e.g., Treatment A vs. no treatment), reported outcomes on disruptive behavior measures were tallied for each set of group comparisons to determine how many outcomes within each study favored the treatment group, how many favored the comparison group, and how many failed to reveal a significant group difference. Only immediate posttreatment results (i.e., based on data collected within 1 month of the end of treatment) were included. As laid out in Eyberg et al. (2008), a treatment was considered superior if half or more of the relevant disruptive behavior measures evidenced significant effects in favor of the treatment group. A treatment was considered inferior if half or more of the relevant disruptive behavior measures evidenced significant effects in favor of the comparison group. For any other pattern of results, the treatment and comparison group were considered equivalent.

Within the 26 treatment families, the Evidence Base Update criteria (Table 1) were then applied to determine the level of evidence to support each treatment family. Treatments that were superior to psychological placebo or another active treatment, or equivalent to an already well-established treatment, in at least two independent settings by two independent teams achieved the highest rating of “well-established treatment.” The second level, “probably efficacious,” was applied when a treatment had been shown superior to a waitlist (i.e., no treatment) control group in at least two studies, or when it had been shown to be superior to another treatment or equivalent to a well-established treatment in one study or in multiple studies by nonindependent teams of researchers. Level 3 “possibly efficacious” treatments were those with at least one study documenting superiority over a waitlist/no-treatment control group, or two or more nonrandomized but otherwise methodologically strong studies. Treatments that had not yet been published with methodologically rigorous designs attained the label “experimental treatments” Southam-Gerow and Prinstein (2014) added a fifth level reserved for treatments for which the evidence had been negative—“treatments of questionable efficacy”—which was not applicable to any treatment families in this review.

TABLE 1 Evidence Base Update Criteria

Methods Criteria

M.1. Group design: Study involved a randomized controlled design

M.2. Independent variable defined: Treatment manuals or logical equivalent were used for the treatment

M.3. Population clarified: Conducted with a population, treated for specified problems, for whom inclusion criteria have been clearly delineated

M.4. Outcomes assessed: Reliable and valid outcome assessment measures gauging the problems targeted (at a minimum) were used

M.5. Analysis adequacy: Appropriate data analyses were used & sample size was sufficient to detect expected effects

Level 1: Well-Established Treatments

1.1 Efficacy demonstrated for the treatment in at least two (2) independent research settings and by two (2) independent investigatory teams demonstrating efficacy by showing the treatment to be either:

1.1.a. Statistically significantly superior to pill or psychological placebo or to another active treatment

OR

1.1.b. Equivalent (or not significantly different) to an already well-established treatment in experiments

AND

1.2. All five (5) of the Methods Criteria

Level 2: Probably Efficacious Treatments

2.1. There must be at least two good experiments showing the treatment is superior (statistically significantly so) to a waitlist control group

OR

2.2. One or more good experiments meeting the well-established treatment level with the one exception of having been conducted in at least two independent research settings and by independent investigatory teams

AND

2.3. All five (5) of the Methods Criteria

Level 3: Possibly Efficacious Treatments

3.1. At least one good randomized controlled trial showing the treatment to be superior to a waitlist or no-treatment control group

AND

3.2. All five (5) of the Methods Criteria

OR

3.3. Two or more clinical studies showing the treatment to be efficacious, with two or more meeting the last four (of five) Methods Criteria, but none being randomized controlled trials

Level 4: Experimental Treatments

4.1. Not yet tested in a randomized controlled trial

OR

4.2. Tested in 1 or more clinical studies but not sufficient to meet Level 3 criteria.

Level 5: Treatments of Questionable Efficacy

5.1. Tested in good group-design experiments and found to be inferior to other treatment group and/or waitlist control group, that is, only evidence available from experimental studies suggests the treatment produces no beneficial effect.

RESULTS

Table 2 shows the set of 64 studies composing the eligible evidence base for this review, along with descriptive information about the evaluation design, sample, and treatment. Some studies contributed potentially classifiable information about more than one treatment family (e.g., group parent behavior therapy and self-directed parent behavior therapy). Some studies or group comparisons, although eligible for the review, did not contribute critical information in classifying the level of evidence. For example, when there were sufficient numbers of studies comparing a particular treatment family to placebo or alternate treatment to achieve well-established status, studies comparing that treatment family to no treatment became irrelevant. All eligible studies are included in Table 2, even if they did not contribute to final classifications, to provide full information for future reviews. Table 3 includes only the studies that contributed to classifying the level of evidence for a treatment family.

TABLE 2 Sample Description for Included Studies

Study Authors, Year	Study Arms	Ethnicity/Race	Country	N (Outcome)^a	Included Outcomes (Informants)
Randomized Studies
Augimeri, Farrington, Koegl, & Day, 2007	1. Group parent behavior therapy + group child behavior therapy	not rep.	Canada	30	CBCL(P)
	2. Attention control
Axberg & Broberg, 2012	1. Group parent behavior therapy	94% both parents Swedish	Sweden	62	ECBI, SESBI
	2. Treatment as usual			(54)	(P, T)
Baker-Henningham, Scott, Jones, & Walker, 2012	1. Teacher training2. No treatment	not rep.	Jamaica	225(210)	DPICS, MOOSES, SESBI, SDQ, Conner, PKBS, ECBI(P, T, O)
Behan, Fitzpatrick, Sharry, Carr, & Waldron, 2001	1. Group parent behavior therapy	not rep.	Ireland	40	PGS, SDQ, CBCL, PSI
	2. No treatment				(P)
Berkovits, O’Brien, Carter, & Eyberg, 2010	1. Individual parent behavior therapy with child participation2. Self-directed parent behavior therapy	67% White 23% African American 7% Hispanic 3% Biracial	U.S.	30(21)	ECBI(P)
Bernal et al., 1980	1. Individual parent behavior therapy with child participation	not rep.	U.S.	36	DBO, TC, Becker(P, O)
	2. Individual parent-focused therapy with child participation
	3. No treatment
Bjorknes & Manger, 2013	1. Group parent behavior therapy2. No treatment	59% Pakistani immigrants41% Somalian immigrants	Norway	96(83)	CP Composite, TRF, ECBI(P, T)
Braet et al., 2009	1. Group parent behavior therapy	not rep.	Belgium	64	CBCL, TRF
	2. No treatment			(40)	(P, T)
Bratton et al., 2013	1. Individual child-centered play therapy2. Attention control	42% African American 39% Hispanic 18% Caucasian	U.S.	54	C–TRF(T)
Chacko et al., 2015	1. Group parent behavior therapy with child participation + Family problem-solving training2. Treatment as usual	51% Hispanic 31% African American 8% Caucasian 3% Native American 7% Other	U.S.	320(306)	IOWA CRS(P)
Christensen et al., 1980	1. Individual parent behavior therapy with child participation	not rep.	U.S.	36(28)	PBOR, Becker, CO(P, O)
	2. Group parent behavior therapy
	3. Self-directed parent behavior therapy
Connell, Sanders, & Markie-Dadds, 1997	1. Self-directed parent behavior therapy	not rep.	Australia	24(23)	ECBI, PDR(P)
	2. No treatment
Cunningham, Bremner, & Boyle, 1995	1. Group parent behavior therapy	18% immigrant	Canada	150	HSQ, CBCL, CO
	2. Individual parent behavior therapy			(114)	(P, O)
	3. No treatment
David, David, & Dobrean, 2014	1. Group parent behavior therapy + Group parent-focused therapy	not rep.	Romania	130(106)	CBCL, TRF(P, T)
	2. No treatment
Enebrink et al., 2012	1. Self-directed parent behavior therapy	97% born in Sweden	Sweden	104(86)	ECBI, SDQ(P)
	2. No treatment
Feinfield & Baker, 2004	1. Individual parent behavior therapy with child participation + Group parent behavior therapy + Group child behavior therapy + Individual child behavior therapy	45% White 17% African American 8% Hispanic 3% Asian 27% mixed	U.S.	47	CBCL, ECBI, HSQ, TBGCR, TRF, SSQ, WMS(P, T)
	2. No treatment
Frank, Keown, & Sanders, 2015	1. Group parent behavior therapy2. No treatment	81% New Zealand European descent 9.5% Maori/Pacific Islander 9.5% Asian	New Zealand	42(40)^b	ECBI(P)
Gardner, Burton, & Klimes, 2006	1. Group parent behavior therapy2. No treatment	not rep.	UK	76(71)	ECBI, CO(P, O)
Havighurst et al., 2013	1. Group parent-focused therapy2. Treatment as usual	77.4% native English speakers22.6% other European or Asian	Australia	54(42)	ECBI, SESBI(P, T)
Homem, Gaspar, Seabra-Santos, Canavarro, & Azevedo, 2014	1. Group parent behavior therapy2. No treatment	not rep.	Portugal	36(33)	PACS, PKBS(P)
Hutchings et al., 2007	1. Group parent behavior therapy	not rep.	UK	153	ECBI, DPICS, SDQ
	2. No treatment			(116)	(P, O)
Kazdin et al., 1987	1. Individual child behavior therapy	76.8% White23.2% Black	U.S.	56(47)	CBCL, SBCL(P, T)
	2. Individual child-centered play therapy
	3. Attention control
Kazdin et al., 1989	1. Individual child behavior therapy	54.5% White45.5% Black	U.S.	112(97)	CBCL, PDR, IAB, SBCL, CATS
	2. Individual child behavior therapy with parent participation				(P, T, C)
	3. Individual child-centered play therapy
Kazdin, Siegel, & Bass, 1992	1. Individual parent behavior therapy with child participation	69.1% White30.9% Black	U.S.	97(76)	CBCL, TRF, IAB, CATS, SDR
	2. Individual child behavior therapy				(P, T, C)
	3. Individual child behavior therapy with parent participation
	4. Individual parent behavior therapy
Kierfeld, Ise, Hanisch, Gortz-Dorten, & Dopfner, 2013	1. Self-directed parent behavior therapy2. No treatment	not rep.	Germany	48(46)	CBCL, FBB-SSV(P)
Kim, Doh, Hong, & Choi, 2011	1. Group child behavior therapy2. Group parent behavior therapy + Group child behavior therapy3. No treatment	100% Korean	South Korea	20	PSBS(T, PR)
Kjobli & Ogden, 2012	1. Individual parent behavior therapy2. Treatment as usual	93.5% Norwegian 1.9% other western European 4.6% other ethnicity	Norway	216	ECBI, HCSBS, SSBS(P, T)
Kjobli et al., 2013	1. Group parent behavior therapy2. Treatment as usual	93% Norwegian 0.7% other western European 7.3% other ethnicity	Norway	137(126)	ECBI, HCSBS, SSBS(P, T)
Kling et al., 2010	1. Group parent behavior therapy	78% Swedish	Sweden	159	ECBI, PDR
	2. Self-directed parent behavior therapy	22% Immigrant		(145)	(P)
	3. No treatment
Kolko et al., 2010	1. Individual child behavior therapy + Individual parent behavior therapy + Medication review + School consultation + Case management + Peer group engagement + Individual parent behavior therapy with child participation2. Treatment as usual	80% Caucasian	U.S.	163(141)	PSC-17, KSADS(P, T)
Larsson et al., 2009	1. Group parent behavior therapy	99% native Norwegians	Norway	127	ECBI, CBCL
	2. Group parent behavior therapy + Group child behavior therapy				(P)
	3. Treatment as usual
Leung et al., 2003	1. Group parent behavior therapy2. Treatment as usual	most were native born Hong Kong residents	Hong Kong	91(69)	PDR, ECBI, SDQ(P)
Lochman, Coie, Underwood, & Terry, 1993	1. Individual child behavior therapy + Group child behavior therapy 2. No treatment	African-American	U.S.	52	TBC, peer nomination (P, PR)
McCabe & Yeh, 2009	1. Individual parent behavior therapy with child participation	Mexican American	U.S.	58(54)	ECBI, CBCL, DPICS, ECI
	2. No treatment				(P, O)
Meany-Walen, Bratton, & Kottman, 2014	1. Individual child-centered play therapy2. Attention control	48% Latino 33% European American 19% African American	U.S.	67(58)	TRF, DOF(T)
Mejia, Calam, & Sanders, 2015	1. Group parent behavior therapy2. No treatment	not rep.	Panama	108(94)	ECBI(P)
Niec et al., 2016	1. Group parent behavior therapy with child participation	86% White 2% Native American	U.S.	94(81)	BASC, ECBI(P)
	2. Individual parent behavior therapy with child participation	10% multi-racial 2% not reported
Nixon, Sweeney, Erickson, & Touyz, 2003	1. Individual parent behavior therapy with child participation	95% Caucasian 5% other: Koori, Chinese, Indian	Australia	54	ECBI, CBCL, HSQ(P)
	2. No treatment
Ogden & Hagen, 2008	1. Individual parent behavior therapy with child participation	94% Norwegian	Norway	112(97)	CBCL, PDR, TRF(P, T)
	2. Treatment as usual	6% other western European
Ojiambo & Bratton, 2014	1. Group child-centered play therapy	All ethnic groups in Uganda represented	Uganda	60	CBCL, TRF(P, T)
	2. Attention control
Patterson, Chamberlain, & Reid, 1982	1. Individual parent behavior therapy with child participation	not rep.	U.S.	19	PDR,TAB(P, O)
	2. Treatment as usual
Peed, Roberts, & Forehand, 1977	1. Individual parent behavior therapy with child participation	not rep.	U.S.	12	BRS, Becker, CO(P, O)
	2. No treatment
Pepler et al., 2010	1. Group parent behavior therapy + Group child behavior therapy2. No treatment	41.9% Caucasian 22.6% African Canadian 3.2% Pacific Islander 3.2% Latino 3.2% Native Canadian 25.8% Other	Canada	80(52)	CBCL, TRF(P, T)
Perrin, Sheldrick, McMenamy, Henson, & Carter, 2014	1. Group parent behavior therapy2. No treatment	Race: 74% White 12% African American 1% Asian 12% other; Ethnicity: 18% Hispanic	U.S.	273(194)	ECBI, DPICS, (P, O)
Porzig-Drummond, Stevenson, & Stevenson, 2015	1. Self-directed parent behavior therapy2. No treatment	not rep.	Australia	84(62)	ECBI(P)
Sanders et al., 2000	1. Individual parent behavior therapy with child participation	predominantly Caucasian	Australia	305(254)	ECBI, PDR, FOS-R-III(P, O)
	2. Individual parent behavior therapy with child participation + Addressing parent mental health needs
	3. Self-directed parent behavior therapy
	4. No treatment
Schuhmann, Foote, Eyberg, Boggs, & Algina, 1998	1. Individual parent behavior therapy with child participation	77% Caucasian, not of Hispanic origin	U.S.	64(42)	ECBI, DPICS(P, O)
	2. No treatment	14% African American; not of Hispanic origin 9% Hispanic, Asian, or mixed
Sumi et al., 2013	1. Individual parent behavior therapy with child participation + Individual child behavior therapy with parent participation + Teacher training	45% White 27% Hispanic 24% African American 13% English language learners	U.S.	286(280)	SSRS, SSBD(P, T)
	2. No treatment
Vitaro, Brendgen, Pagani, Tremblay, & McDuff, 1999	1. Individual parent behavior therapy with child participation	100% French Caucasian	Canada	73	SBQ(P)
	2. No treatment
Walker et al., 1998	1. Individual parent behavior therapy with child participation + Individual child behavior therapy with parent participation + Teacher training	7% minority status	U.S.	42	TRF(T)
	2. No treatment
Webster-Stratton & Hammond, 1997	1. Group parent behavior therapy2. Group child behavior therapy3. Group parent behavior therapy + Group child behavior therapy4. No treatment	86% Caucasian	U.S.	97	ECBI, CBCL, PBQ, DPICS, PPS-I-Care(P, T, O)
Webster-Stratton, 1990	1. Self-directed parent behavior therapy	not rep.	U.S.	47(43)	ECBI, CBCL, PDR, DPICS(P, O)
	2. No treatment
Webster-Stratton, 1992	1. Self-directed parent behavior therapy	not rep.	U.S.	100	ECBI, CBCL, PBQ, PDR, DPICS
	2. No treatment				(P, T, O)
Webster-Stratton, Kolpacoff, & Hollingsworth, 1988	1. Self-directed parent behavior therapy	not rep.	U.S.	194^c(178)	ECBI, CBCL, PBQ, PDR(P, T)
	2. No treatment
Webster-Stratton et al., 2004	1. Group parent behavior therapy2. Group child behavior therapy3. Group child behavior therapy + Teacher training4. Group parent behavior therapy + Teacher training5. Group parent behavior therapy + Group child behavior therapy + Teacher training6. No treatment	79% Euro-American	U.S.	159	ECBI, TASB, SHP, PCSC, MOOSES, DPICS, DPIS(P, T, O)
Wells & Egan, 1988	1. Individual parent behavior therapy with child participation	not rep.	U.S.	23(19)	CO(O)
	2. Individual parent-focused therapy with child participation
Nonrandomized Studies
Abrahamse et al., 2012	1. Individual parent behavior therapy with child participation2. No treatment	62% Caucasian 11% Surinamese 8% Moroccan 3% Turkish 16% other	Netherlands	95(84)	ECBI(P)
Abrahamse et al., 2016	1. Individual parent behavior therapy with child participation	70% Caucasian	Netherlands	45(40)	ECBI, CBCL, DPICS(P)
	2. Treatment as usual
Hanisch et al., 2010	1. Group parent behavior therapy + teacher training	not rep.	Germany	155(121)	CBCL, PCL, HSQ, OBDT
	2. No treatment				(P, T, O)
Leung, Tsang, Heung, & Yiu, 2009	1. Individual parent behavior therapy with child participation	not rep.	Hong Kong	130(110)	ECBI, PSI(P, O)
	2. No treatment
Meany-Walen, Bullis, Kottman, & Dillman Taylor, 2015	1. Group child-centered play therapy2. No treatment (single case design)	100% Caucasian	U.S.	2	DOF(T)
Ollendick et al., 2016	1. Family problem-solving training	83% Caucasian17% non-Caucasian	U.S.	134(100)	DBDRS(P)
	2. Individual parent behavior therapy with child participation
Shapiro, Youngstrom, Youngstrom, & Marcinick, 2012	1. Individual parent behavior therapy2. Treatment as usual	59% African American 25% Caucasian 10% Biracial 5% Hispanic	U.S.	348(194)	CBCL, OS(P)
Stoltz et al., 2013	1. Individual child behavior therapy	68% Dutch32% Immigrant	Netherlands	271(264)	SHP, Observation(T, O)
	2. No treatment

Note: Abbreviations for Outcomes: Becker = Bi-Polar Adjective Checklist; BRS = Behavior Rating Scale; CATS = Children’s Action Tendency Scale; CBCL = Child Behavior Checklist; ChIA = Children’s Inventory of Anger; CO = coded observation; Conner = Conner’s Global Index; CP composite = Conduct problems composite = ECBI and PDR; C–TRF = Teacher report form for children 1.5–5; DBDRS = Disruptive Behavior Disorders Rating Scale; DBO = Deviant Behavior Observation; DOF = Direct Observation Form; DPICS = Dyadic Parent–Child Interaction Coding System; DPIS = Dyadic Peer Interaction Scale; ECBI = Eyberg Child Behavior Inventory; ECI = Early Childhood Inventory; ERQ = Emotion Recognition Questionnaire; FBB-SSV = Fremdbeurteilungsbogen für Störungen des Sozialverhaltens [observation scale for social behavior problems; ODD subscale]; FOS-R-III = Revised Family Observation Schedule; HCSBS = Home and Community Social Behavior Scales; HIS = Home Interview Scale; HSQ = Home Situations Questionnaire; IAB = Interview for Antisocial Behavior; IOWA CRS = IOWA Conners Rating Scale; KSADS = Kiddie-Schedule for Affective Disorders and Schizophrenia; MOOSES = Multi-Option Observation System for Experimental Studies; OBDT = Observed Behavior During the Test; OS = Ohio Scales; PACS = Parent Account of Child Symptoms; PAES = Pediatric Anger Expression Inventory; PBOR = Parent behavior observation record = PBQ = Preschool Behavior Questionnaire (teacher); PCL = Problem Checklist; PCSC = Perceived Competence Scale for Young Children; PDR = Parent Daily Report (aka PDRC = Parent Daily Report Checklist); PGS = Parent Goal Scales; PKBS = Preschool and Kindergarten Behavior Scales; PPS-I-Care = Peer problem solving interaction communication affect rating coding system; PSBS = Preschool Social Behavior Scale; PSC-17 = Pediatric Symptom Checklist; PSI = Parent Stress Index; SBCL = School Behavior Checklist; SBQ = Social Behavior Questionnaire; SDQ = Strengths and Difficulties Questionnaire; SDR = Self-Report Delinquency Checklist; SESBI = Sutter-Eyberg Student Behavior Inventory; SHP = Social Health Profile; SPSM = Social Problem Solving Measure; SSBD = Systematic Screening for Behavior Disorders; SSBS = School Social Behavior Scales; SSQ = School situation questionnaire; SSRS = Social Skills Rating Scale; STAXI = State-Trait Anger Expression Inventory; TAB = Total Aversive Behavior on Family Interaction Coding System; TASB = Teacher Assessment of School Behavior; TBC = Teacher Behavior Checklist; TBGCR = Three Behavior Global Change Rating; TC = Tailored Checklist; TOCA-R = Teacher Observation of Child Adaptation-Revised; TRF = Teacher report form (CBCL); WMS = Walker–McConnell = Scale of Social Competence and School Adjustment. Abbreviations for Informants: P = Parent; T = Teacher; O = Observer or Clinician; C = Child; PR = Peer.

^aOutcome sample size is reported if sample size at post treatment assessment differs from reported sample size.

^bNumber of children; study reported both parents separately.

^c114 families participated.

TABLE 3 Treatment Family and Qualifying Studies, by Level of Evidence

Treatment Family	Qualifying Studies	Gender: % Male	Age in Years: Range (M)	Program	Effect Size^a	Follow-Up	No. of Effects Maintained
Level 1: Well-Established
Group parent behavior therapy	Superior to alternate treatment
	Axberg & Broberg, 2012	84	4–8	Incredible Years Basic	−0.99	1 year	2 of 4
	Kjobli et al., 2013	63	3–12 (9)	PMTO (Norway)	N/A	None	—
	Kling et al., 2010	60	3–10 (6)	COMET	−0.97	6 months	2 of 3 (further improvement) (1 of 3 delayed improvement)
	Larsson et al., 2009	80	4–8 (7)	Incredible Years Basic	−1.41	1 year	0
	Leung et al., 2003	64	3–7 (4)	Triple P (Level 4)	−0.77	None	—
	Equivalent to alternate treatment
	Christensen et al., 1980	78	4–12	PMTO	N/A	None	—
	Cunningham et al., 1995	51	5–6 (Kindergarten)	unnamed	0.02	6 months	1 of 3 (delayed improvement)
	Webster-Stratton et al., 2004	90	4–8 (6)	Incredible Years Basic	−0.64	1 year	3 of 3
Individual parent behavior therapy with child participation	Superior to alternate treatment
	Bernal et al., 1980	86	5–12 (8)	Behavioral parent training	N/A	6 months	3 of 4 (further improvement)
	Ogden & Hagen, 2008	80	4–12 (8)	PMTO (Norway)	−0.54	None	—
	Patterson et al., 1982	68	3–12 (7)	OSLC	−1.11	None	—
	Wells & Egan, 1988	not rep.^b	3–8	Social learning parent training (Hanf model)	−3.68	None
	Equivalent to alternate treatment
	Berkovits et al., 2010	70	3–6 (4)	Abbreviated PCIT	−1.43	6 month	0
	Christensen et al., 1980	78	4–12	PMTO	N/A	None	—
	Kazdin et al., 1992	78	7–13 (11)	Parent Management Training	−0.46	1 year	1 (of 7) (1 delayed, 3 decayed and 2 no effect)
	Sanders et al., 2000	68	3–4 (3)	Triple P Precursor^c	−0.55	1 year	2 (2 no effect, 1 delayed)
Level 2: Probably Efficacious
Group parent behavior therapy + group child behavior therapy	Superior to alternate treatment
	Augimeri et al., 2007	73	< 12 (9)	SNAP Under 12 Outreach Program	N/A	18 months	2 of 2
	Equivalent to alternate treatment
	Larsson et al., 2009	80	4–8 (7)	Incredible Years Basic + Dinosaur School (Norway)	−1.11	1 year	5 of 6 no effect
Group parent behavior therapy with child participation + Family problem-solving training	Superior to alternate treatment
	Chacko et al., 2015	68	7–11 (9)	Multiple Family Group	−0.46	None	—
Group parent behavior therapy with child participation	Equivalent to well-established treatment
	Niec et al., 2016	72	3–6 (4)	Group Parent-Child Interaction Therapy	N/A	None	—
Individual parent behavior therapy	Superior to alternate treatment
	Kjobli & Ogden, 2012	58	3–12 (7)	Brief PMTO (Norway)	−0.53	None	—
	Equivalent to alternate treatment
	Cunningham et al., 1995	51	5–6 (Kindergarten)	unnamed	N/A	None	—
Individual parent behavior therapy with child participation + individual child behavior therapy with parent participation + teacher training	Superior to no treatment
	Sumi et al., 2013	77	1st–3rd grade (8)	First Step to Success	−0.53	1 year	2 (1 delayed, 2 no effect)
	Walker et al., 1998	74	Kindergarten	First Step to Success	−1.08	1–2 years	2 of 2
Self-directed parent behavior therapy	Superior to no treatment
	Connell et al., 1997	43	2–6 (4)	Every Parent	−1.74	4 months	5 (3 delayed)
	Enebrink et al., 2012	58	3–12 (7)	COMET	−1.65	6 months	3 of 3
	Kierfeld et al., 2013	50	3–6	THOP	−1.12	None	—
	Kling et al., 2010	60	3–10 (6)	COMET	−0.52	6 months	3 of 3 further improvement
	Porzig-Drummond et al., 2015	50	2–10 (5)	1–2-3 Magic	N/A	6 months	2 of 2
	Webster-Stratton, 1992	72	3–8 (5)	Incredible Years Precursor^d	−0.88	1 year;	7 of 10; (3 of 10 delayed improvement)
	Equivalent to no treatment
	Sanders et al., 2000	68	3–4 (3)	Self-Directed Behavioral Family Intervention	−0.55	1 year	2 (1 delayed, 2 no effect)
	Webster-Stratton et al., 1988	69	3–8	Incredible Years Precursor	−0.86	None	—
	Webster-Stratton, 1990	91	3–8 (5)	Incredible Years Precursor	−0.76	None	—
Group child behavior therapy	Superior to no treatment
	Kim et al., 2011	75	4–5	unnamed	−0.94	None	—
	Webster-Stratton et al., 2004	90	4–8 (6)	Incredible Years Dinosaur School	−0.32	1 year	2 of 3
Group child behavior therapy	Equivalent to well-established treatment
+ teacher training	Webster-Stratton et al., 2004	90	4–8 (6)	Incredible Years Dinosaur School plus Teacher Training	−0.75	1 year	2 of 3
Individual child behavior therapy	Superior to alternate treatment
	Kazdin et al., 1987	80	7–13 (11)	Problem Solving Skills Training	−1.80	1 year	2 of 2
	Kazdin et al., 1989	78	7–12 (11)	Problem Solving Skills Training	−0.77	1 year	3 of 3 (delayed effect)
Individual child behavior therapy with parent participation	Superior to alternate treatment
	Kazdin et al., 1989	78	7–12 (11)	Problem Solving Skills Training with in vivo practice	−1.34	1 year	3 of 3
	Kazdin et al., 1992	78	7–13 (11)	Problem Solving Skills Training with in vivo practice	−0.84	1 year	6 of 7
Group parent-focused therapy	Superior to alternate treatment
	Havighurst et al., 2013	78	4–6 (6)	Tuning Into Kids	−1.22	6 months	2 of 2 (different measure than reported at posttest)
Group child-centered play therapy	Superior to alternate treatment
	Ojiambo & Bratton, 2014	50	10–12	Group Activity Play Therapy	−0.91	None	—
Individual child-centered play therapy	Superior to alternate treatment
	Meany-Walen et al., 2014	79	5–9 (6)	Adlerian Play Therapy	−0.99	None	—
Level 3: Possibly Efficacious
Group parent behavior therapy + individual parent behavior therapy with child participation + group child behavior therapy + individual child behavior therapy	Superior to no treatment
	Feinfield & Baker, 2004	86	4–8 (7)	Project TEAM	−0.55	5 months	3 of 3 decayed
Group parent behavior therapy + group parent-focused therapy	Superior to no treatment
	David et al., 2014	48	4–12 (6)	Rational Positive Parenting Program	−0.35	1 month	1 of 2
Group parent behavior therapy + teacher training + group child behavior therapy	Superior to no treatment
	Webster-Stratton et al., 2004	90	4–8 (6)	Incredible Years Basic plus Teacher Training plus Dinosaur School	−0.66	1 year	2 of 3
Group parent behavior therapy + teacher training	Superior to no treatment
	Webster-Stratton et al., 2004	90	4–8 (6)	Incredible Years Basic plus Teacher Training	−0.73	1 year	3 of 3
Individual parent behavior therapy with child participation + addressing parental mental health needs	Superior to no treatment
	Sanders et al., 2000	68	3–4 (3)	Enhanced Behavioral Family Intervention	−1.00	1 year	5 of 5
Group child behavior therapy + individual child behavior therapy	Superior to no treatment
	Lochman et al., 1993	52	third grade	unnamed	N/A	None	—
Teacher training	Superior to no treatment
	Baker-Henningham et al., 2012	61	3–6	Incredible Years Teacher Training	−0.34	None	—
Level 4: Experimental
Family problem-solving training	Not yet tested in a randomized trial
	Ollendick et al., 2016	62	7–14 (9)	Collaborative & Proactive Solutions	N/A	6 months	2 of 2

Note. COMET = Communication Method program; OSLO = Oregon Social Learning Center program; PCIT = Parent-Child Interaction Therapy; PMTO = Parent Management Training Oregon; SNAP = Stop Now and Plan; THOP = Therapy program for children with hyperkinetic and oppositional problem behavior.

^aEffect sizes shown represent a simple baseline to posttest change (as a function of baseline standard deviation) among the treatment group based on available data. These effect sizes should not be compared with effect sizes calculated using other methods more typically used in meta-analysis that would incorporate the correlation between baseline and posttest scores into the effect size.

^bBoys and girls included, but proportion not reported.

^cName of program: Standard Behavioral Family Intervention.

^d Name of program: Individually administered videotape modeling parent training program (IVM).

Leveling of Treatments

Of the 26 treatment families, 23 had sufficient evidence to meet the leveling criteria. Two treatment families achieved the highest distinction of well-established treatments: group parent behavior therapy, and individual parent behavior therapy with child participation. Both of these treatment families had multiple published randomized trials by independent research teams documenting superiority of the treatment over a psychological placebo or another treatment. Group parent behavior therapy was examined in eight such studies comparing treatment to an alternate treatment, five of which produced favorable results; individual parent behavior therapy with child participation was examined in eight studies comparing treatment to an alternate treatment, four of which produced favorable results.

Thirteen treatment families were classified as Level 2, probably efficacious. Six met criteria based on single studies showing superiority to an alternate treatment, services as usual, or attention control group: group parent behavior therapy + group child behavior therapy, group parent behavior therapy with child participation + family problem-solving training, individual parent behavior therapy, group parent-focused therapy, group child-centered play therapy, and individual child-centered play therapy. Two treatment families were classified as probably efficacious based on multiple studies by the same investigators showing superiority to an alternate treatment: individual child behavior therapy alone and with parent participation. Three treatment families (individual parent behavior therapy with child participation + individual child behavior therapy with parent participation + teacher training, self-directed parent behavior therapy, and group child behavior therapy) were classified as probably efficacious on the basis of multiple studies showing superiority over no treatment. Two treatment families (group parent behavior therapy with child participation and group child behavior therapy + teacher training) were each equivalent to a well-established treatment in a single study.

Seven treatment families met criteria for possibly efficacious (Level 3), all based on single studies showing superiority to a no-treatment group. Five of those included a well-established treatment mode in combination with other components. The final two possibly efficacious treatments were teacher training and a combination of individual child behavior therapy plus group child behavior therapy. One treatment family should still be considered experimental (Level 4). Although family problem-solving training had a single study suggesting superiority over a waitlist group, initial randomization was violated so results should be interpreted with caution.

Unclassifiable Treatment Families

Three treatment families had eligible studies but did not meet criteria for any of the levels of evidence. Individual parent-focused therapy with child participation had one eligible study showing no benefit compared to a no treatment group (Bernal, Klinnert, & Schultz, 1980) and one study showing inferiority compared to an individual behavioral parent therapy with child participation (Wells & Egan, 1988). Individual child-centered play therapy had three qualifying studies—one showed superiority over a reading mentoring program (Bratton et al., 2013), one showed equivalence to an attention control group (Kazdin, Esveldt-Dawson, French, & Unis, 1987), and one showed inferiority compared to individual child behavior therapy with or without parent participation (Kazdin, Bass, Siegel, & Thomas, 1989). Finally, an intensive treatment consisting of individual parent behavior therapy with child participation, individual child behavior therapy, school consultation, case management, and medication review showed no difference compared to treatment as usual (Kolko, Campo, Kelleher, & Cheng, 2010). The evidence for these treatment families is thus insufficient for review by the Evidence Base Update criteria.

Demographic Moderators

As with all evidence reviews, questions must be asked about whom the treatments have been tested with and if there are differences for whom treatments work best. Table 3 provides the age and gender distribution for the samples in the studies that determined the level of evidence for different treatment families (i.e., the “qualifying studies”). It is important to note that in most instances, a single qualifying study contributed to determining the level of evidence. Specifically, of the 13 probably efficacious treatment families, six treatment families have each been tested only in single well-conducted studies. Another six treatment families were represented by two eligible studies. Only a single probably efficacious treatment family (self-directed parent behavior therapy) had been tested with more than two eligible studies. All seven of the possibly efficacious treatment families and the experimental treatment family were classified based on the results of single studies. Thus, only limited statements can be made about the populations for whom those treatment families work.

With respect to gender, all 39 studies that contributed information to the leveling of the treatment families included both boys and girls in their samples. Only eight eligible studies reported on analysis of gender as a potential moderator of treatment effects, seven of which reported no significant differences in outcomes by gender (Kjobli, Hukkelberg, & Ogden, 2013; Kjobli & Ogden, 2012; Kling, Forster, Sundell, & Melin, 2010; Leung, Sanders, Leung, Mak, & Lau, 2003; Ogden & Hagen, 2008; Ollendick et al., 2016; Walker et al., 1998). One study of self-directed parent behavior therapy reported a single gender difference on only one of several indicators analyzed, favoring girls (Enebrink, Hogstrom, Forster, & Ghaderi, 2012). Thus, although limited, the available evidence for the treatment families in this review appears to be equally strong for boys and girls.

An even smaller number of studies (five) investigated age as a potential moderator. One study investigating group and individual parent behavioral therapies (Ogden & Hagen, 2008) across a wide age range (4–12 years) reported better outcomes for younger children than older children. One nonrandomized study investigating family problem-solving training (Ollendick et al., 2016) with 7- to 14-year-olds suggested that younger ages in that range benefited more. Four studies with large age ranges (ages 3 or 4 to 12 years) reported no significant differences in effects by age for Internet-based, individual, group or self-directed parent behavior therapy (Enebrink et al., 2012; Kjobli et al., 2013; Kjobli & Odgen, 2012; Kling et al., 2010, respectively). These patterns do not lend themselves to even tentative statements about age as a moderator for specific programs.

Looking at age differences across treatment families, however, a few potential patterns emerged. For example, six treatment families (individual child behavior therapy with or without parent participation, individual plus group child behavior therapy, group child-centered play therapy, group parent behavior therapy with child participation + family problem-solving training, and family problem-solving training) only have qualifying studies that included the older ages covered by this review (i.e., 7–13 years of age). Eleven other treatment families only included younger ages covered by this review (i.e., up to age 9) in qualifying studies. The final six classified treatment families (group parent behavior therapy, group parent behavior therapy + group child behavior therapy, group parent behavior therapy + group parent-focused therapy, individual parent behavior therapy with and without child participation, and self-directed parent behavior therapy) have been tested and found superior to no treatment or an alternate treatment across the entire age range covered by this review (i.e., ages 3–12). Comparisons between these findings and McCart and Sheidow’s (2016) Evidence Base Update of treatments for youth ages 13 and older may provide further insight.

Information on the race or ethnicity of children or parents was missing from 22 of 64 eligible studies for this review, and effects of those characteristics were not analyzed in most of the studies that had information. Twenty-nine studies reported on racial and/or ethnic background of participants but were relatively homogeneous samples (i.e., made up of more than 80% of participants of the same race or ethnicity). Although each of those 29 individual studies included a relatively homogeneous set of participants with respect to racial or ethnic groups, the studies as a group represent a wide range of backgrounds. From the United States there were studies with primarily White, African American, and Mexican American samples. From Canada there were primarily English-speaking and entirely French Canadian samples. The four Australian/New Zealand studies were primarily White. Scandinavia was represented by multiple studies from Norway and Sweden, with one Norwegian sample involving more than half Pakistani immigrants. The three studies from the Netherlands included approximately one third immigrants; studies from other European countries (United Kingdom, Ireland, Belgium, Germany, Romania, Portugal) did not discuss ethnicity. Studies were also conducted in Hong Kong, South Korea, Jamaica, Panama, and Uganda. The whole of the literature on these treatments is thus relatively heterogeneous, even though individual studies in the group were not. Fifteen of the 64 eligible studies reported the race and/or ethnicity of their participants and likely had enough variability to at least analyze group differences but either did not analyze or did not report on those analyses. Only a single study (Ollendick et al., 2016) analyzed effects by race/ethnicity. Effects for Caucasian children were similar to those for non-Caucasian children.

Effect Sizes and Follow-Up

Although not considered during the application of the Evidence Base Update criteria, Table 3 also displays information about effect size, duration of follow-up, and maintenance of effects at follow-up. Effect sizes shown represent baseline to posttest change (as a function of baseline standard deviation) for the treatment group to show the relative amount of improvement for treated participants across studies. Traditionally, within-group effect sizes are attenuated for the degree of correlation between the baseline and posttest scores, prior to aggregation for a meta-analysis. However, that correlation was almost never provided in articles; thus we report only the simple baseline to posttest change index (without correction for correlation) here. These effect sizes should not be compared with effect sizes calculated using other methods more typically used in meta-analysis. Not surprisingly, visual inspection suggests larger effects among studies with “superior” outcomes than studies with “equivalent” outcomes. Also notable is the overall range of effect sizes, even within treatment families. For example, effect size for group parent behavior therapy ranged from a low of 0.02 (indicating essentially no change from baseline to post, possibly a slight worsening of the treatment group) to a high of −1.41 (almost 1.5 standard deviation improvement). An explicitly quantitative examination of these studies, such as a formal meta-analysis, might reveal patterns of moderation that are not apparent from visual inspection.

Evidence Base Update criteria similarly do not incorporate information on maintenance of effects after treatment ends, which should also be an important consideration in deliberations about selecting treatment families. Only slightly more than half (56%) of the qualifying studies in this review reported on at least one relevant outcome at a follow-up assessment point, ranging from 1 month to 2 years posttreatment. Many nonreporting studies used a waitlist control group, which precludes follow-up assessment of those who did not receive the treatment. Other studies may have published follow-up results in a subsequent publication, so could not be reflected here. Most studies reporting a follow-up assessment (17 of 22) documented maintained effects on at least two disruptive behavior measures, with many indicating either delayed improvement (i.e., the group difference was not significant at posttest but was significant at follow-up) or further improvement following the significant posttest difference.

DISCUSSION

This review applies an established preponderance of evidence approach to reviewing psychosocial treatments for DBDs in children 12 years of age and younger. Two treatments—both of which are parent-focused treatments that incorporate behavioral elements—met criteria for the highest level of evidence, well-established. The two previous Society for Clinical Child and Adolescent Psychology reviews in this area (Brestan & Eyberg, 1998; Eyberg et al., 2008) also concluded that an individual parent behavior therapy with child participation (Parent Management Training Oregon Model) was well-established. This review extends the well-established classification to include group parent behavior therapy. This review also complements practice parameters from child psychiatry. AACAP’s (Steiner, 1997) recommendations for the treatment of CD include “family interventions” (i.e., parent guidance, training and family therapy) but do not prioritize family approaches over other domains such as treatment of comorbid disorders, child psychotherapy, and child psychosocial skill-building. AACAP’s more recent (Steiner & Remsing, 2007) ODD practice parameters highlight parent management training as recommended for “almost all cases.” Their definition is consistent with both of the well-established treatment families as classified in this review and treatment families that have not attained as high a level of evidence (i.e., individual parent behavior therapy, self-directed parent behavior therapy, and parent-directed behavior therapies in conjunction with other psychosocial modalities).

In this review, treatments including a behavioral orientation appeared in general to have a stronger evidence base than treatments lacking behavioral elements (such as relationship-based or parent-focused therapy or child-centered play therapy). Two previous meta-analyses have produced somewhat inconsistent conclusions when comparing behavioral and nonbehavioral approaches to child problem behaviors. Lundahl, Risser, and Lovejoy (2006) found no significant difference between behavioral and nonbehavioral parent training programs. Comer, Chow, Chan, Cooper-Vince, and Wilson (2013) reported significantly greater effects for behavioral versus nonbehavioral psychosocial treatments for disruptive behavior problems in children younger than 8. However, meta-analytic results are based on average effect sizes aggregated across separate studies and thus not direct comparisons of behavioral and nonbehavioral approaches within the same sample. Four studies eligible for the current review directly tested behavioral against nonbehavioral approaches, all of which led to similar conclusions about the superiority of treatments with behavioral elements. Bernal and colleagues (1980) tested individual client-centered parent therapy against individual parent behavior therapy with child participation for families with children ages 5 to 12, with the behavioral approach resulting in significantly better outcomes. Kazdin and colleagues (Kazdin et al., 1989; Kazdin et al., 1987) compared individual child relationship therapy to individual child behavior therapy with and without parent participation in families of children ages 7–13, with the behavioral approaches resulting in significantly better child disruptive behavior outcomes. Wells and Egan (1988) reported significantly better outcomes among their sample of families with 3- to 8-year-old children who participated in behavioral as compared to nonbehavioral individual parent therapy with child participation. Behaviorally oriented treatments thus have a consistently stronger evidence base both in systematic reviews and head-to-head comparisons.

Distinctions between this type of review that examines strength of evidence and reviews that examine the relative strength of effects are not always made evident to consumers of evidence-based reviews, and are thus important to highlight here. Evidentiary reviews, the approach used in the present study, classify the level of available evidence and thus identify treatments for which there is the greatest degree of confidence that implementation will have a significant effect. In contrast, strength of effects analyses (e.g., meta-analyses) identify which interventions have the largest effect sizes and are thus likely to have the largest impact on participants if implemented. Thus, although both group parent behavior therapy and individual parent behavior therapy with child participation are likely to significantly decrease children’s disruptive behavior (based on their classifications as well-established treatments), the two treatment types might not be equivalent with respect to the strength of their effects (i.e., how much of a decrease in problematic behavior is observed after participation). Although the current type of review cannot answer that question, other clues exist to inform future hypotheses.

Two key differences between the two well-established treatment families offer insight into how their effectiveness might compare. The first is that the treatments differ with respect to who participates in treatment—one treatment includes parents only, the other includes the child in the parent-directed treatment. Child involvement in parent-focused behavior therapy allows the therapist to directly observe the behaviors and the relationships rather than relying only on parent report and can provide an opportunity for the parents to practice skills with their child during treatment, which was a robust predictor of larger effects in a meta-analysis of parent training programs (Kaminski, Valle, Filene, & Boyle, 2008). Thus, although both of the well-established treatment families were parent focused, indicating a high level of evidence for intervening with parents for the treatment of DBDs in children ages 12 years and younger, the likely importance of child involvement should not be overlooked. In addition to potentially stronger effects, inclusion of the child in parent-focused programs may also have pragmatic implications. When reimbursement for therapy is contingent upon the diagnosed individual’s participation in treatment, reimbursement for parent-focused therapy may be easier to justify if the child were also involved.

The second difference between the two well-established treatments that is important with respect to potential differences in strength of effects is the number of participants being treated simultaneously, that is, whether treatment is being delivered individually or to groups. Group delivery offers the opportunity for parent-to-parent peer support and destigmatization of therapy by having parents socialize with others facing similar parenting challenges and allows for a single therapist to treat a larger number of parents. However, individually delivered treatment can provide deeper engagement with parents and the potential for individualized learning or pacing of a program. Not yet known is whether the potential greater efficiency of treating a larger number of clients might be at the cost of intensity of treatment, leading to weaker effects. This question has been examined using meta-analysis, with inconsistent results. A meta-analysis of parent training programs in general (i.e., not restricted by child age or to a particular type of outcome; Lundahl et al., 2006) reported larger effect sizes for individually delivered programs than for group-delivered programs. A more recent meta-analysis of psychosocial treatments for disruptive behavior problems in children ages 8 and younger (i.e., not restricted to parents as the focal participants; Comer et al., 2013) reported similar effect sizes for group- versus individually delivered programs. Although both meta-analyses investigated delivery mode as a moderator of effect sizes, neither could isolate the effect of delivery mode from other factors such as treatment orientation and whether the child was involved in treatment. Further, as recent evidence has emerged that parents may differ in their preference of one type of treatment approach over another based on their needs and perception of their child’s diagnosis (Wymbs et al., 2015), investigations of the impact of parent preferences on program outcomes may be valuable.

Rather than continued independent streams of effectiveness trials of the two well-established treatment families, greater health impact could be achieved by investigation into how to blend the strengths of effective approaches into an even more effective treatment. A combined approach including both group- and individually delivered parent behavior therapy with child participation was represented in only a single study eligible for the current review (Feinfield & Baker, 2004, in which treatment was superior to a no-treatment control group and the effects persisted to a 5-month follow-up assessment). As that multimodal treatment also included both group- and individually delivered child-focused behavior therapy, the simple effects of a blended approach to parent behavior therapy cannot be estimated. However, Niec, Barnett, Prewett, and Shanley Chatham (2016) directly compared group parent behavior therapy with child participation versus individual parent behavior therapy with child participation and reported that the group version was not statistically inferior to the individually administered version, leading to its classification as probably efficacious. As well, at least one preventive program has shown significant child behavior outcomes with a combination of group delivery with individual parent–child time in the program (Kaminski et al., 2013), suggesting promise for a blended approach to treatment. Thus, instead of pitting efficiency of delivery against intensity of effects, future efforts could focus on maximizing both.

As with all studies, this type of evidence review has its limitations. The established criteria restrict study eligibility in certain ways (e.g., to published studies, to manualized treatments, to specific sample inclusion criteria) that limit the generalizability of the results to the population of studies represented by the eligible sample of articles. Each individual study restricted inclusion as well (e.g., by excluding children with substantial developmental delays), thus limiting the generalizations that can be made about the strength of evidence for these treatments for all children with DBDs. The nature of the review involves a “vote counting” technique (i.e., based on the number of studies with significant differences) that has been criticized for relying on a somewhat arbitrary statistical significance cutoff and capitalizing on chance by aggregating based on p values alone (Bushman, 1994). Although not a recommended approach to aggregating empirical findings, this type of preponderance of evidence approach works well to show how much rigorous evidence has accumulated in support of particular approaches. A review of evidence approach also has the benefit of being more inclusive of the literature than meta-analyses, which typically exclude studies or findings that report only whether there was a significant effect.

The change from reviewing specific treatment packages in Brestan and Eyberg (1998) and Eyberg et al. (2008) to reviewing treatment families also has important limitations and implications. As just noted, classification of treatment families has the advantages of informing population-level, professional, and family decisions where packaged brand-name programs are unavailable. However, substantial variability in content, delivery, and effects within treatment families has been previously documented, suggesting caution in drawing conclusions about individual programs based only on what is known about the more general classes of treatment families. For example, Kaminski and colleagues (Kaminski et al., 2008) reported effect sizes from behavioral parent training programs ranging from –0.61 (i.e., a difference of more than .5 standard deviation favoring the comparison group) to 3.69 (i.e., a difference of more than 3.5 standard deviations favoring the treatment group). Among the studies of the two treatments attaining a well-established level of evidence in this review, there were a handful of studies for which the treatment group was not superior to the comparison group and a wide range of effect sizes, indicating that even with a well-established treatment family, there is no guarantee of success. This underscores the importance of not relying on broadly construed treatment families (e.g., group parent behavior therapy) or single program characteristics (e.g., child participation in parent-focused treatment) to draw conclusions about likely impact if a particular program is implemented. Conclusions from a review of treatment families (such as this one) can be interpreted only as indicative of a general class of treatments, and might not be true for every program that could fall within a treatment family. One limitation in understanding not only whether an approach can work but for whom it works best is evident in the very limited analyses of family characteristics such as race or ethnicity, as most study samples were generally homogeneous. Although treatment programs can be effectively used with different cultural groups (e.g., Gardner, Montgomery, & Knerr, 2015), there is also evidence that cultural background affects the effectiveness of interventions such as parent training (Lau, 2006) and thus should be carefully evaluated. Whenever possible, selection of a particular treatment or treatment family should thus be guided by more information than can be provided here.

With two well-established, 13 probably efficacious, seven possibly efficacious, one experimental, and three unclassifiable treatment families, this Evidence Base Update documents the state of a sizable body of evidence regarding efficacy, effectiveness, or relative effectiveness of psychosocial treatments for DBDs for children ages 12 and younger. As the individual program models continue to compete for superiority, more such studies will surely accumulate. Potentially more valuable, however, and with a much smaller base of existing knowledge, are systematic investigations of implementation, dissemination, and uptake. For example, few studies exist on how to expand implementation of these mostly university- or clinic-tested treatments to existing and wider reaching infrastructures. Similarly, the extent to which delivery could expand beyond the narrower set of providers holding advanced academic credentials—perhaps either through adapted curricula and training or for children with less severe symptoms—has not been investigated. Moving from understanding which approach is superior overall to understanding which approach is best under which conditions or for which families would also facilitate more efficient use of treatment resources. In other words, this field has many answers to questions about “what works” for children with DBDs but is still in need of actionable strategies to get what works to all of the children and families who need it.

ACKNOWLEDGMENTS

The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.