Methodological Issues in Analog Acceptability Research: Are Teachers' Acceptability Ratings of Assessment Methods Influenced by Experimental Design?

Abstract

The present study explored the relationship between experimental design method and teacher-rated acceptability of two analog approaches for assessing academic skills problems. A total of 619 general elementary education teachers were assigned to either (a) a between-subjects group design condition (n=418) in which they evaluated the acceptability of one of two psychoeducational assessment case summaries or (b) a within-subjects group design condition (n = 201) in which they rated the acceptability of both case summaries. Comparisons between the acceptability ratings of the curriculum-based assessment and published norm-referenced test protocols across design conditions suggested that the magnitude of differences increased when the within-subjects group design condition was employed. In addition, curriculum-based assessment was consistently rated as a more acceptable method of assessment than published, norm-referenced tests. Results are discussed in relation to the influence of design method and the acceptability of psychoeducational assessment methods.

Additional information

Notes on contributors

Tanya L. Eckert

Tanya L. Eckert, PhD, is Assistant Professor of Psychology at Syracuse University, Syracuse, New York. She received her PhD in School Psychology from Lehigh University. Her research interests include examining procedures for assessing academic and behavior problems, developing classroom-based interventions, and measuring the acceptability of assessment and intervention procedures.

Edward S. Shapiro

Edward S. Shapiro, PhD, is currently Professor, School Psychology Program at Lehigh University, Bethlehem, Pennsylvania. He earned his PhD in School Psychology from the University of Pittsburgh. His research interests include curriculum-based assessment, interventions for inclusion of students with E/BD in regular classrooms, behavioral assessment, and the integration of health-care and psychological services within school setting.