Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

ABSTRACT

Although the structure of the Rosenberg Self-Esteem Scale (RSES) has been exhaustively evaluated, questions regarding dimensionality and direction of wording effects continue to be debated. To shed new light on these issues, we ask (a) for what percentage of individuals is a unidimensional model adequate, (b) what additional percentage of individuals can be modeled with multidimensional specifications, and (c) what percentage of individuals respond so inconsistently that they cannot be well modeled? To estimate these percentages, we applied iteratively reweighted least squares (IRLS) to examine the structure of the RSES in a large, publicly available data set. A distance measure, d_s, reflecting a distance between a response pattern and an estimated model, was used for case weighting. We found that a bifactor model provided the best overall model fit, with one general factor and two wording-related group factors. However, on the basis of d_r values, a distance measure based on individual residuals, we concluded that approximately 86% of cases were adequately modeled through a unidimensional structure, and only an additional 3% required a bifactor model. Roughly 11% of cases were judged as “unmodelable” due to their significant residuals in all models considered. Finally, analysis of d_s revealed that some, but not all, of the superior fit of the bifactor model is owed to that model's ability to better accommodate implausible and possibly invalid response patterns, and not necessarily because it better accounts for the effects of direction of wording.

KEYWORDS:

• Iterated reweighted least squares
• bifactor model
• model fit
• Rosenberg Self Esteem Scale

Article information

Conflict of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This work was supported by Grant 1317428 (Peter Spirtes, PI) from the National Science Foundation's Division of Mathematical Sciences (DMS) program in Computational and Data-Enabled Science and Engineering in Mathematical and Statistical Sciences (CDS&E-MSS), and additional support was obtained through Grant No. 1U2-CCA186878-01 (David Cella, PI) from the National Institutes of Health NIH Roadmap for Medical Research.

Role of the Funders/Sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The authors thank Peter Bentler for comments on prior versions of this manuscript. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institutions, the National Science Foundation, or the National Institutes of Health is not intended and should not be inferred.

Notes

1 Thissen (2016), in discussing IRT tests of “unidimensionality” (vs. not), astutely observed that even asking the question of whether data are strictly unidimensional is, bluntly, not a good or even meaningful question. The history of psychometrics tells us that data typically are more or less consistent with a unidimensional model. Thus the better question is to what degree the data are unidimensional.

2 The term robust, as used here, refers to factor-loading estimates for which the influence of cases with response patterns that are inconsistent with the estimated model have been diminished during estimation. The term robust, as used here, has nothing to do with standard errors or fit indices that are ostensibly adjusted for nonnormality.

3 Diagonally weighted least squares is thought to provide better estimation for ordinal items, although many argue that, with five or more response options, it makes little difference. We do not use these ordinal methods here because the points we are trying to demonstrate do not depend on it. There are also technical reasons, which are beyond the present scope.

4 Note that the use of a chi-square-based cutoff does not imply an assumption that d²_S is chi-square distributed asymptotically. In fact, the sampling distribution of d_s would be tedious to derive given that the elements that go into its computation are normals, squared normals, and products of normals.

5 The computation of d_S and d_r do not require IRLS estimation. These indices can always be computed under any model with any estimator.

6 It is interesting to note that Bartlett factor score estimates from the unidimensional model are correlated r =.92 with general factor estimates from the bifactor. Thus it is unclear what real “adjustment” or “control for multidimensional” is being made by specifying a bifactor model. Within the bifactor model, factor score estimates for the group factors are correlated r =.32, and positively worded and negatively worded group factors are correlated -.23 and -.29 with the general factor scores, respectively.