Examining the Influence of Interval Length on the Dependability of Observational Estimates

Author Note. The authors wish to extend their thanks to Mary Woerner for her data collection efforts.

Abstract

Systematic direct observation is a tool commonly employed by school psychologists to investigate student behavior. As these data are used for educational decision-making, ensuring the psychometric adequacy of the obtained data is an important consideration. Given that procedural aspects of systematic direct observation have been shown to influence the psychometric properties of obtained data, this study was designed to explore how interval length influences the dependability of academic engagement data when using a momentary time sampling procedure. Twenty seventh-grade students were each observed for two 15-min sessions during math instruction. A series of generalizability studies were conducted to examine how manipulations to interval length influenced reliability-like coefficients. In general, shorter interval lengths (i.e., 10 s, 15 s) were shown to produce higher levels of dependability. For example, an acceptable level of dependability (i.e., ϕ = .70) required twice as many 30-min observations when utilizing 20- or 30-s sampling as were required when utilizing 10- or 15-s sampling. Furthermore, whereas an acceptable level of dependability (i.e., ϕ = .70) could not be obtained using any interval length when conducting a single observation, this criterion was met using either 10- or 15-s sampling across two 30-min observations.

Additional information

Notes on contributors

Amy M. Briesch

Amy M. Briesch is an associate professor in the Department of Applied Psychology at Northeastern University and codirector of the Center for Research in School-Based Prevention. Her research interests include the role of student involvement in intervention design and implementation, as well as the development of feasible and psychometrically sound measures for the assessment of student behavior in multitiered systems of support.

Tyler David Ferguson

Tyler David Ferguson is a school psychologist in the Lynn Public Schools, as well as an adjunct professor at Northeastern University and Salem State University. His clinical interests include cognitive and behavioral assessment, autism spectrum disorders, and emotional intelligence.

Brian Daniels

Brian Daniels is an assistant professor in the school psychology program at the University of Massachusetts, Boston. His research focuses on improving the feasibility and treatment utility of assessment tools for monitoring student progress in response to social–emotional and behavioral intervention.

Robert J. Volpe

Robert J. Volpe is a professor in the Department of Applied Psychology at Northeastern University and codirector of the Center for Research in School-Based Prevention. His research focuses on designing academic and behavioral interventions for students with disruptive behavior disorders, as well as feasible systems for assessing student behavior in problem-solving models.

Adam B. Feinberg

Adam B. Feinberg is an assistant research professor at the University of Connecticut and the director of the Northeast PBIS Network. His research and clinical interests include the development and implementation of multitiered systems of supports in schools and districts, with a focus on developing and supporting coaching knowledge, skills, and networks.