Implications of Item Keying and Item Valence for the Investigation of Construct Dimensionality

Abstract

Factor analysis and nomological network analysis are commonly used as complementary procedures in the investigation of the dimensionality of constructs (e.g., self-esteem, job satisfaction). Although it has been demonstrated that factor analyses are often biased toward a two-dimensional solution for measures including regular- and reverse-keyed items, less attention has been paid to the implications for nomological network analyses. We propose, and demonstrate empirically in two studies, that item keying is confounded with item valence (i.e., favorability of item content), and that item valence can bias the results of both factor analysis and nomological network analysis toward a two-dimensional interpretation. We also demonstrate that the valence effect is related to, but distinguishable from, social desirability response bias. We caution that the practice of excluding reverse-keyed items to achieve unidimensionality can lead to distortion in correlations among constructs, and we offer alternative remedies to the valence problem.

Notes

Two items relevant to sex-relevant behaviors and love-related cognition were removed from the data collection due to concerns from the ethics board.

The dimensionality of BZSR in Sample 1 could not be tested with CFA because it contains only one positively valence item.

For each construct, positively valenced items and negatively valenced items are parceled separately.

Some researchers may question whether our item valence effect is simply caused by item extremity (Spector, van Katwyk, Brannick, & Chen, 1997). Extreme items may load on two separate factors (McPherson & Mohr, 2005). However, this alternative reason does not explain two of our important findings, namely (a) why items of the same valence across different measures will load on the same method factor in our MTMM analysis, and (b) why constructs of the same valence (e.g., introversion and neuroticism) will correlate stronger with each other (as compared to constructs of opposing valence). In addition, McPherson and Mohr (2005) found that even for the regular-keyed and reverse-keyed items that are moderate in wordings, they still load on two separate factors in factor analysis. Item extremety thus cannot fully explain our findings.