Test validity from a temporal perspective: Incorporating time in validation research

Abstract

As psychological theories and methodologies are increasingly being permeated with time, the need for reviewing and updating test validation practices is becoming apparent. After a brief comparison of the differential (between-subjects) and temporal (within-subjects) approaches to psychological research, this article examines the role that time has played in traditional validation research, and identifies a number of conceptual and methodological shortcomings that complicate meta-analysis, hamper theory building, and limit practical applications. Starting from the assumption that human traits and behaviours unfold over time, and that stability is a special form of change, it argues that time should be an explicit part of any measurement, either test or criterion, and that validation should include an evaluation of dynamic aspects of the test–criterion relationship. It furthermore argues that when stability is assumed, its temporal boundaries should be marked and supporting evidence be presented, using instruments displaying measurement invariance over time. The article continues to present a framework based on a Variables × Subjects × Time data model, and distinguishes four conditions for validation, namely (1) stable predictor, stable criterion, (2) stable predictor, dynamic criterion, (3) dynamic predictor, stable criterion, and (4) dynamic predictor, dynamic criterion, which allow the definition of five new types of validity. It discusses implications for validation practice, meta-analysis, and theory development as well as for practical applications.

Keywords:

• Time
• Testing
• Validity

Acknowledgments

This article was written during a stay at the Center for Advanced Studies at the University of Leipzig, Germany. I gratefully acknowledge the hospitality of the centre.

Notes

1 Of course, attenuation correction may be useful to correct for other sources of error, such as lack of homogeneity in measurement instruments.