Maximal Criterion Validity and Scale Criterion Validity: A Latent Variable Modeling Approach for Examining Their Difference

Abstract

This article studies the difference between the criterion validity coefficient of the widely used overall scale score for a unidimensional multicomponent measuring instrument and the maximal criterion validity coefficient that is achievable with a linear combination of its components. A necessary and sufficient condition of their identity is presented in the case of measurement errors being uncorrelated among themselves and with a used criterion. An upper bound of the difference in these validity coefficients is provided, indicating that it cannot exceed the discrepancy between the maximal reliability and composite reliability indexes. A readily applicable latent variable modeling procedure is discussed that can be used for point and interval estimation of the difference between the maximal and scale criterion validity coefficients. The outlined method is illustrated with a numerical example.

Keywords:

• criterion
• criterion validity
• maximal criterion validity
• maximal reliability
• measurement error
• reliability
• validity

ACKNOWLEDGMENTS

This research was in part conducted while Tenko Raykov was visiting the Leibniz Institute for the Social Sciences, Mannheim, Germany. We are grateful to G. Hancock, G. A. Marcoulides, and S. Penev for valuable discussions on maximal reliability and validity, and to B. Rammstedt for helpful support.

Notes

¹ In case p = 2, additional constraints are needed to identify the congeneric model (Equation 1), such as error variance equality or loading equality, which need to be correct in a studied population if to be imposed to use the procedure discussed here.