Improved Estimation of Maximal Reliability for Unidimensional Multicomponent Measuring Instruments in Repeated Measure Studies

Abstract

In repeated measure studies with unidimensional scales, measurement invariance, and specificity stability over time, the specificity variance in each instrument component can be identified. This article describes for that setting an improved point and interval estimation procedure for the maximal reliability coefficient associated with a given set of homogeneous measures. The method is developed within the framework of latent variable modeling and can also be readily used in longitudinal studies for improved point and interval estimation of individual measure reliability and scale reliability at each assessment occasion. The procedure is based on empirically testable conditions and is illustrated with an example.

Keywords:

• factor analysis
• identification
• maximal reliability
• reliability
• scale reliability
• specificity
• unique factor
• variance

ACKNOWLEDGMENTS

This research was in part conducted while Tenko Raykov was visiting the Leibniz Institute for the Social Sciences, Mannheim, Germany. Thanks are due to J. Tisak, Y. Amemiya, M. W. Browne, G. Hancock, and P. M. Bentler for valuable discussions on indicator specificity and maximal reliability, as well as to C. Wolf and B. Rammstedt for helpful support.

Notes

¹ A necessary condition for identification of the models with MI and SS over time used in this article is pq > 4; this condition is also assumed fulfilled in the remainder (see also Raykov & Amemiya, 2008). When the last condition is violated (i.e., if pq ≤ 4), model identification and specificity variance identification cannot be ensured. In particular, when p = q = 2, the model in Equation 6 with MI and SS, the general form of which is of central relevance in this article, is not identified, and the specificity variance in any of the two repeated measures (scale components) is not identified then.

² In this article, the term specificity is used to refer to that part of an observed score that is not associated with commonality or pure measurement error (e.g., Harman, 1976). This usage is common in most of the psychometric and FA literature, unlike some statistical literature referring to the unique factors in ε as specificity factors (or specificities).

³ Note that the right sides of Equations 4, 10, and 13 are increasing functions in δ_jk (or δ_j in Equation 4). Hence, the improved individual measure, scale reliability, or maximal reliability coefficients cannot be smaller than their corresponding (routinely used) version not accounting for indicator specificity.