Teachers' Assessments of Students' Achievements: The Ecological Validity of Studies Using Case Vignettes

ABSTRACT

In two studies, we investigated the extent to which research using case vignettes of fictitious students is able to yield research results that are ecologically valid representations of teachers' assessments of students' educational achievement in their real classrooms. The type of assessment was teachers' tracking decisions from primary to secondary school. In Study 1, which was conducted in the German educational system, teachers decided which secondary school track their own students would attend and were subsequently confronted with the same decision for fictitious students described in vignettes. Study 2 was a replication in the Luxembourgish educational system. Ordinal and logistic regression models were used to compare decision making in the two conditions of actual students versus case vignettes. No major differences were found between the two conditions. Results are discussed with respect to the ecological validity of research designs that use case vignettes.

KEYWORDS:

• Case vignettes
• ecological validity
• tracking decisions

Acknowledgments

The authors would like to thank Cornelia Gräsel, Bergische Universität Wuppertal, for her collaboration in conducting Study 1 under a joint research project funded by the DFG. Furthermore, the authors would like to thank the inspectors of “Collège des Inspecteurs de Ministère de l'Éducation Nationale” of Luxembourg for their support in contacting the primary school teachers who participated in Study 2.

Funding

The research reported in this paper was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) and the Research Council of Luxembourg (Fonds National de la Recherche, FNR) Grants KR 2162/4-1, INTER/DFG/09/01, and C10/LM/784116.

Notes

1. In most parts of Germany, the transition from primary to secondary school takes place at the end of the fourth grade in primary school when the students are about 10 years old.

2. In Luxembourg, the transition from primary to secondary school takes place at the end of the sixth grade in primary school when the students are about 12 years old.