Selecting the “Best” Factor Structure and Moving Measurement Validation Forward: An Illustration

ABSTRACT

Despite the broad literature base on factor analysis best practices, research seeking to evaluate a measure's psychometric properties frequently fails to consider or follow these recommendations. This leads to incorrect factor structures, numerous and often overly complex competing factor models and, perhaps most harmful, biased model results. Our goal is to demonstrate a practical and actionable process for factor analysis through (a) an overview of six statistical and psychometric issues and approaches to be aware of, investigate, and report when engaging in factor structure validation, along with a flowchart for recommended procedures to understand latent factor structures; (b) demonstrating these issues to provide a summary of the updated Posttraumatic Stress Disorder Checklist (PCL–5) factor models and a rationale for validation; and (c) conducting a comprehensive statistical and psychometric validation of the PCL–5 factor structure to demonstrate all the issues we described earlier. Considering previous research, the PCL–5 was evaluated using a sample of 1,403 U.S. Air Force remotely piloted aircraft operators with high levels of battlefield exposure. Previously proposed PCL–5 factor structures were not supported by the data, but instead a bifactor model is arguably more statistically appropriate.

Acknowledgments

The authors would like to give special thanks to Tanya Goodman for assistance with measure administration, data collection, and institutional review board approval. The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the U.S. Air Force, the Department of Defense, or the U.S. Government.

Notes

¹ The PCL was also updated to the 20-item PCL–5 (Weathers et al., 2013) to match the 20 posttraumatic stress disorder (PTSD) symptoms of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders ([DSM–5]; American Psychiatric Association, 2013; see Bovin et al., 2016).

² The American Psychological Association provides reporting standards for quantitative research in psychology, which includes structural equation modeling standards in Table 7 (Appelbaum et al., 2018).

³ The eigenvalue-greater-than-1 rule or Kaiser criterion (K1) somehow still manages to persist in practice, but it should not be used as it has been shown to be inaccurate. Van der Eijk and Rose (2015) provided a nice open access review of the problems with the K1 criterion.

⁴ Consistent with the procedures outlined in Figure 2, WLSMV EFA was conducted later because of less than ideal model fit and psychometric properties for WLSMV CFA. Because the EFA suggested either a one- or two-factor model, more parsimonious WLSMV CFA models were also estimated here for comparability purposes.

⁵ All interfactor correlations corresponding to the models in Table 2 are available from the corresponding author.

⁶ The Sample 2 results are nearly identical and reached the same conclusions and, therefore, are not presented here. These results are available from the corresponding author.