Incremental Validity of Test Session and Classroom Observations in a Multimethod Assessment of Attention Deficit/Hyperactivity Disorder

Abstract

This study tested the incremental validity of behavioral observations, over and above parent and teacher reports, for assessing symptoms of Attention Deficit/Hyperactivity Disorder (ADHD) in children ages 6 to 12, using the Test Observation Form (TOF) and Direct Observation Form (DOF) from the Achenbach System of Empirically Based Assessment. The TOF Attention Problems and DOF Intrusive scales contributed significant unique variance, over and above parent and teacher ratings, to predicting parent and teacher ratings of hyperactivity and impulsivity and predicting categorical diagnoses of ADHD-Combined type versus Non-ADHD and ADHD-Combined type versus ADHD-Predominantly Inattentive type. The TOF Oppositional and Attention Deficit/Hyperactivity Problems scales contributed unique variance to predicting parent ratings of hyperactivity and impulsivity and the DOF Oppositional and Attention Deficit/Hyperactivity Problems scales contributed unique variance to predicting teacher ratings of hyperactivity and impulsivity.

Preparation of this article was supported in part by Grant R01 HD40220 from the National Institute of Child Health and Human Development to Stephanie McConaughy at the University of Vermont and by the University of Vermont Research Center for Children, Youth, and Families. Statements do not represent the position or policy of these agencies and no official endorsement by them should be inferred.

Notes

Note. DSM–IV–TR = Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition–Text Revision; ADHD = Attention Deficit/Hyperactivity Disorder; ADHD-C = ADHD-Combined type; ADHD-IN = ADHD-Inattentive type; SES = socioeconomic status.

^a N = 310.

^b n = 98.

^c n = 23.

^d n = 79.

^e SES scored on an adapted version of Hollingshead's (1975) scale where 1 = lowest and 9 = highest (n = 284).

^f Children with comorbid diagnoses were counted more than once for the different diagnostic categories.

¹Across the three models using the CBCL, TRF, and TOF scales as predictors (nine statistical tests at Step 2), two effects (significant Beta weights) at p < .05 might be due to chance, using a p < .05 protection level. Across the four models using the CBCL, TRF, and DOF scales as predictors (12 statistical tests at Step 2), 2 to 3 effects (significant Beta weights) at p < .05 might be due to chance, using a p < .05 protection level (Sakoda et al., 1954). Possible chance effects are marked in tables for sets of hierarchical multiple regressions for each dependent variable and each set of multinomial logistic regressions. Effects at p < .01 and p < .001 were not considered possible chance effects.

Note. N = 310. Means and standard deviations (in parentheses) are on the diagonal. CBCL = Child Behavior Checklist; TRF = Teacher's Report Form; TOF = Test Observation Form; DOF = Direct Observation Form; WISC–IV = Wechsler Intelligence Scale for Children–Fourth edition; ADHDRS–IV = Attention Deficit/Hyperactivity Disorder Rating Scale–IV.

*p < .05. **p < .01. ***p < .001.

Note. N = 310. Means and standard deviations (in parentheses) are on the diagonal. CBCL = Child Behavior Checklist; TRF = Teacher's Report Form; TOF = Test Observation Form; DOF = Direct Observation Form; DSM = Diagnostic and Statistical Manual of Mental Disorders; WISC–IV = Wechsler Intelligence Scale for Children–Fourth edition; ADHDRS–V = Attention Deficit/Hyperactivity Disorder Rating Scale–IV. Correlations between ADHDRS–IV scales are shown in Table 1.

*p < .05. **p < .01. ***p < .001.

Note. N = 310.Semipartial r ² = unique variance accounted for by each predictor; CBCL = Child Behavior Checklist; TRF = Teacher's Report Form; TOF = Test Observation Form; WISC–IV = Wechsler Intelligence Scale for Children–Fourth edition.

^a From Attention Deficit/Hyperactivity Disorder Rating Scale–IV-Home version.

^b From Attention Deficit/Hyperactivity Disorder Rating Scale–IV-School version.

^c Possible chance effect when corrected for number of analyses (Sakoda et al., 1954).

^d With removal of two outliers, β = .07, p = .065.

^e With removal of one outlier, β = .08, p = .042.

*p < .05. **p < .01. ***p < .001.

Note. N = 310. Semipartial r ² = unique variance accounted for by each predictor; CBCL = Child Behavior Checklist; TRF = Teacher's Report Form; DOF = Direct Observation Form.

^a From Attention Deficit/Hyperactivity Disorder Rating Scale–IV-Home version.

^b From Attention Deficit/Hyperactivity Disorder Rating Scale–IV-School version.

^c Possible chance effect when corrected for number of analyses (Sakoda et al., 1954).

^d With removal of two outliers, β = .09, p = .039.

^e With removal of one outlier, β = .05, p = .104.

*p < .05. **p < .01. ***p < .001.

Note. N = 200. CBCL = Child Behavior Checklist; TRF = Teacher's Report Form; TOF = Test Observation Form; ADHD-C = Attention Deficit/Hyperactivity Disorder-Combined type; ADHD-IN = Attention Deficit/Hyperactivity Disorder-Inattentive type; WISC–IV = Wechsler Intelligence Scale for Children–Fourth edition.

^a Odds ratios (95% confidence interval) showing independent contribution of each predictor with Non-ADHD as reference category.

^b Odds ratios (95% confidence interval) showing independent contribution of each predictor with ADHD-IN as reference category.

^c Possible chance effect when corrected for number of analyses (Sakoda et al., 1954).

Note. N = 200. CBCL = Child Behavior Checklist; TRF = Teacher's Report Form; DOF = Direct Observation Form; ADHD-C = Attention Deficit/Hyperactivity Disorder-Combined type; ADHD-IN = Attention Deficit/Hyperactivity Disorder-Inattentive type.

^a Odds ratios (95% confidence interval) showing independent contribution of each predictor with Non-ADHD as reference category.

^b Odds ratios (95% confidence interval) showing independent contribution of each predictor with ADHD-IN as reference category.

^c Possible chance effect when corrected for number of analyses (Sakoda et al., 1954).