Comparing Test Scores Using Information From Criterion-Related Validity Studies

Abstract

There is frequently a need to compare a client's test scores from different instruments. If the scores come from instruments that use the same scale, it is tempting to compare the scores directly. Unfortunately, this method can lead clinicians to believe that there is a large difference between scores when the difference is minimal. As an alternative, we outline a method for score comparison that uses information from criterion-related validity studies. Using three examples, we show why this method is more psychometrically sound, produces more accurate comparison scores, and requires little extra work for clinicians than the direct comparison approach. To make the score comparison process easy for clinicians to use, we include an appendix that demonstrates how to implement this method in Microsoft Excel and the free R program.

Key words:

• psychological assessment
• score comparison
• score exchangeability
• test scores

Notes

¹This is an oversimplification of the transformation process, as techniques such as smoothing, continuous norming, and score normalization are also frequently employed during the raw-score conversion process.

²In some texts, the slope estimated using Z scores is called a standardized regression coefficient.

³R is a free program that can be downloaded from http://www.r-project.org