Rasch Validation of a Measure of Reform-Oriented Science Teaching Practices

Abstract

Growing evidence from recent curriculum documents and previous research suggests that reform-oriented science teaching practices promote students’ conceptual understanding, levels of achievement, and motivation to learn, especially when students are actively engaged in constructing their ideas through scientific inquiries. However, it is difficult to identify to what extent science teachers engage students in reform-oriented teaching practices (RTPs) in their science classrooms. In order to exactly diagnose the current status of science teachers’ implementation of the RTPs, a valid and reliable instrument tool is needed. The principles of validity and reliability are fundamental cornerstones in developing a robust measurement tool. As such, this study was motivated by the desire to point out the limitations of the existing statistical and psychometric analyses and to further examine the validation of the RTP survey instrument. This paper thus aims at calibrating the items of the RTPs for science teachers using the Rasch model. The survey instrument scale was adapted from the 2012 National Survey of Science and Mathematics Education (NSSME) data. A total of 3701 science teachers from 1403 schools from across the USA participated in the NSSME survey. After calibrating the RTP items and persons on the same scale, the RTP instrument well represented the population of US science teachers. Model-data fit determined by Infit and Outfit statistics was within an appropriate range (0.5–1.5), supporting the unidimensional structure of the RTPs. The ordered category thresholds and the probability of the thresholds showed that the five-point rating scale functioned well. The results of this study support the use of the RTP measure from the 2012 NSSME in assessing usage of RTPs.

Keywords:

• Rasch validation
• Reform-oriented teaching practices (RTPs)
• Instructional strategies
• Science teaching

Acknowledgments

These data were collected by Horizon Research, Inc. under National Science Foundational Award Number DRL-1008228. Any opinions, findings, and conclusions or recommendations expressed herein are those of the authors and do not necessarily reflect the views of the National Science Foundation or Horizon Research, Inc.

Notes

1 Theta represents underlying ability of a particular person with a scale typically depicted as ranging from ‒3 to 3, with 0.0 representing average ability.

2 Logit = represents the probability of correctly responding to an item.

3 Parameter invariance: The word parameter indicates population quantities, and the word invariance indicates that parameter values are identical in different population groups or across different measurement conditions (Rupp & Zumbo, 2006).