Generalized Linear Factor Models: A New Local EM Estimation Algorithm

Abstract

In this article, a general approach to latent variable models based on an underlying generalized linear model (GLM) with factor analysis observation process is introduced. We call these models Generalized Linear Factor Models (GLFM). The observations are produced from a general model framework that involves observed and latent variables that are assumed to be distributed in the exponential family. More specifically, we concentrate on situations where the observed variables are both discretely measured (e.g., binomial, Poisson) and continuously distributed (e.g., gamma). The common latent factors are assumed to be independent with a standard multivariate normal distribution. Practical details of training such models with a new local expectation-maximization (EM) algorithm, which can be considered as a generalized EM-type algorithm, are also discussed. In conjunction with an approximated version of the Fisher score algorithm (FSA), we show how to calculate maximum likelihood estimates of the model parameters, and to yield inferences about the unobservable path of the common factors. The methodology is illustrated by an extensive Monte Carlo simulation study and the results show promising performance.

Keywords:

• EM algorithm
• Factor models
• Generalized linear models
• Scores algorithm
• Simulations

Mathematics Subject Classification:

• Primary 62H25
• Secondary 62J12

Acknowledgments

An earlier version of this article was presented at the 1st Stochastic Modeling Techniques and Data Analysis International Conference STMDA-2010 in Chania Crete, Greece. We are grateful to the participants of the Congress for their helpful discussion. In addition, we would like to thank an Associate Editor and two anonymous referees for their detailed and useful comments that greatly improved this article.

Notes

The symboles ′ and ″ in the functions b _i (x) and g(x) are used to represent, respectively, the first and second derivatives of these functions with respect to x. However, for all vectors and matrices, the symbole ′ represents the transpose operator.

In this article, the authors have presented a slightly similar proof for the case of standard latent factor models.