A Moderated Nonlinear Factor Model for the Development of Commensurate Measures in Integrative Data Analysis

Abstract

Integrative data analysis (IDA) is a methodological framework that allows for the fitting of models to data that have been pooled across 2 or more independent sources. IDA offers many potential advantages including increased statistical power, greater subject heterogeneity, higher observed frequencies of low base-rate behaviors, and longer developmental periods of study. However, a core challenge is the estimation of valid and reliable psychometric scores that are based on potentially different items with different response options drawn from different studies. In Bauer and Hussong (2009) we proposed a method for obtaining scores within an IDA called moderated nonlinear factor analysis (MNLFA). Here we move significantly beyond this work in the development of a general framework for estimating MNLFA models and obtaining scale scores across a variety of settings. We propose a 5-step procedure and demonstrate this approach using data drawn from n = 1,972 individuals ranging in age from 11 to 34 years pooled across 3 independent studies to examine the factor structure of 17 binary items assessing depressive symptomatology. We offer substantive conclusions about the factor structure of depression, use this structure to compute individual-specific scale scores, and make recommendations for the use of these methods in practice.

View correction statement:

Corrigendum

Notes

By harmonized we mean the altering of an item response to make it comparable across studies (e.g., collapsing a five-option response to a three-option response); see Hussong et al. (2013, p. 69) for further details.

We conducted sensitivity analysis to compare the dichotomized responses with the original response categories. Significant estimation problems resulted due to the highly sparse response patterns in the higher categories; when models did converge, substantive findings were quite similar to those identified using the dichotomization.

A caveat to this principle is when a construct manifests itself differently at different points in development or in different subgroups. For example, items showing different developmental patterns may be indicative of heterotypic continuity. If theory suggests that this is the case, these seemingly aberrant items can be retained and the differences in age trends can be accommodated in the MNLFA model discussed later.

We did not require all items to load on depression in all within-study analyses to acknowledge differences in factoring due to changes in item sets across studies.

We divided chronological age by 10 to reduce the scale of the powered terms used in the polynomials.

Because of the limited effects of covariates on the depression factor variance and the significantly increased computational burden of retaining these effects in the second series of models, we omitted predictors of the factor variance in this model building stage; extensive sensitivity analysis showed this did not affect the tests of intercepts and slopes in any substantive way.

For items that were not present in all studies, only appropriate study contrasts were included as predictors of the intercept and factor loading for that item.

The elimination of these nonsignificant covariate interactions represents a highly conservative strategy in that their inclusion is not of theoretical interest and they were tested to protect against potential model misspecification.