A Mode-Jumping Algorithm for Bayesian Factor Analysis

Abstract

Exploratory factor analysis is a dimension-reduction technique commonly used in psychology, finance, genomics, neuroscience, and economics. Advances in computational power have opened the door for fully Bayesian treatments of factor analysis. One open problem is enforcing rotational identifability of the latent factor loadings, as the loadings are not identified from the likelihood without further restrictions. Nonidentifability of the loadings can cause posterior multimodality, which can produce misleading posterior summaries. The positive-diagonal, lower-triangular (PLT) constraint is the most commonly used restriction to guarantee identifiability, in which the upper m × m submatrix of the loadings is constrained to be a lower-triangular matrix with positive-diagonal elements. The PLT constraint can fail to guarantee identifiability if the constrained submatrix is singular. Furthermore, though the PLT constraint addresses identifiability-related multimodality, it introduces additional mixing issues. We introduce a new Bayesian sampling algorithm that efficiently explores the multimodal posterior surface and addresses issues with PLT-constrained approaches. Supplementary materials for this article are available online.

KEYWORDS:

• Factor model
• Identifiability
• Markov chain Monte Carlo
• Mixing
• Orthogonal rotation
• Posterior multimodality

Acknowledgments

We would like to thank two anonymous reviewers and the associate editor for their helpful comments and thoughtful suggestions. We would also like to thank Christian Aßmann, Jens Boysen-Hogrefe, and Markus Pape for providing their code. This work made use of the Illinois Campus Cluster, a computing resource that is operated by the Illinois Campus Cluster Program (ICCP) in conjunction with the National Center for Supercomputing Applications (NCSA) and which is supported by funds from the University of Illinois at Urbana-Champaign.