Comparison of Two Approaches to Interpretive Use Arguments

ABSTRACT

The Standards for Educational and Psychological Testing recommend an argument-based approach to validation that involves a clear statement of the intended interpretation and use of test scores, the identification of the underlying assumptions and inferences in that statement—termed the interpretation/use argument, and gathering of evidence to support or refute the assumptions and inferences. We present two approaches to articulating the interpretation/use argument. One approach uses the five sources of validity evidence in the Standards for Educational and Psychological Testing as a framework and the other approach uses Kane’s chain of assumptions/inferences as a framework. Through this process we identified aspects of these approaches that need to be further clarified for instrument developers to consistently implement either approach, identified important differences in the perspective each approach takes on validation, and highlight important questions for the measurement and mathematics education research fields to consider.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

¹ The term assumptions/inferences is used consistently throughout the article to refer to several terms in the literature, such as assumptions, inferences, claims, and propositions. While there are likely important differences between these terms, they are not used in a consistent manner in the argument-based validation literature. For clarity we use assumptions/inferences throughout but address this inconsistency further in the discussion.

² It is important to note that this summation of the “category” approach is based on what has often been published in practice and it may not represent what was intended by theory. Therefore, it is important to determine whether argument-based approaches implemented in practice are a closer match to intended theory. In other words, is it the theory that needs to change or is it actually the implementation of validation approaches in practice where issues arise?

³ The validity by design approach dictates the development and validation process must be done in conjunction with one another. The example assessments were not developed via this framework; therefore, this approach would not apply well. In addition, there are multiple examples of validity by design being used in practice so there is less need for examples illustrating their IUAs.