Do Reactions to Tests Produce Changes in the Construct Measured?

Abstract

Applicants for state trooper positions completed two tests and three measures of test reactions as part of a study that examined their reactions to different types of measures and the impact these reactions had on the construct measured by the test. African American examinees felt a video-based measure that involved their interpretation of job-related incidents was more face valid than a reading comprehension measure based on job-related material though their perceptions of the two tests' fairness and predictive validity were similar. Caucasian examinees reacted less favorably to the video-based measure. Results also indicate that test reactions and race are related minimally to both video and reading comprehension testlets beyond their relationship to the latent factor underlying the measure. The latter finding is taken as evidence that test reactions did not affect the nature of the underlying factor measured by a test in this situation.