An Exploratory Comparison of Three Treatment Fidelity Assessment Methods: Time Sampling, Event Recording, and Post-observation Checklist

ABSTRACT

When assessed, treatment fidelity is most often evaluated by checklists of intervention steps after an observation session, though estimates can vary depending on how intervention steps are operationalized and rated. A more straightforward approach may involve the adoption of direct observation methods such as time sampling or event recording, which are regularly used to evaluate student behavior. This study explores the utility of using time sampling and event recording in comparison to the traditional post-observation checklist method. Five paraeducator consultees’ treatment fidelity of behavior support plans for students with autism was assessed by post-observation ratings. In addition, 15-sec whole-interval recording of treatment fidelity across intervention steps and event recording of prompts (an intervention step for all plans) were collected throughout the observation session. To compare treatment fidelity data, we present visual analysis, associated quantitative metrics, and correlations. Implications for future treatment fidelity assessment research and consultation practice are described.

Acknowledgement

Preparation of this article was supported by a grant from the University of Massachusetts Boston Healey Research Grant Program. Opinions expressed herein do not necessarily reflect the position of University of Massachusetts Boston, and such endorsements should not be inferred.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Melissa A. Collier-Meek

Melissa A. Collier-Meek, PhD, is an assistant professor in the school psychology program in the Department of Counseling and School Psychology at University of Massachusetts Boston. Her research interests include the implementation of evidence-based interventions, particularly treatment integrity assessment and support.

Lisa M. H. Sanetti

Lisa H. Sanetti, PhD, BCBA-D is an associate professor in the Neag School of Education and a research scientist with the Center for Behavioral Education and Research at the University of Connecticut. Her primary areas of research interest include treatment fidelity assessment and promotion and educator wellness.

Kaitlin Gould

Kaitlin Gould, PhD, BCBA is a postdoctoral fellow at the Kennedy Krieger Institute/Johns Hopkins School of Medicine. In this role, she is responsible for providing assessment and intervention for children with challenging behaviors. She completed her PhD in school psychology at University of Massachusetts Boston. Her research interests include combined interventions and the treatment of severe behavior problems.

Brittany Pereira

Brittany A. Pereira, M.Ed, is a doctoral student in the school psychology program at the University of Massachusetts Boston. Her research involves collaborative consultation, inclusion, and social-emotional and behavioral supports for students with disabilities, as well as mindfulness-based practices.