Learning Subspaces of Different Dimensions

Abstract

We introduce a Bayesian model for inferring mixtures of subspaces of different dimensions. The model allows flexible and efficient learning of a density supported in an ambient space which in fact can concentrate around some lower-dimensional space. The key challenge in such a mixture model is specification of prior distributions over subspaces of different dimensions. We address this challenge by embedding subspaces or Grassmann manifolds into a sphere of relatively low dimension and specifying priors on the sphere. We provide an efficient sampling algorithm for the posterior distribution of the model parameters. We illustrate that a simple extension of our mixture of subspaces model can be applied to topic modeling. The utility of our approach is demonstrated with applications to real and simulated data.

KEYWORDS:

• Conway embedding
• Mixtures of subspaces
• Subspaces of different dimensions

Acknowledgments

We thank to the editor, the associate editor, and three reviewers for their valuable comments which have led to substantial improvement of our article. SM and BST thank to Robert Calderbank, Daniel Runcie, and Jesse Windle for useful discussions.

Additional information

Funding

SM is pleased to acknowledge support from grants NIH (Systems Biology) 5P50-GM081883, AFOSR FA9550-10-1-0436, and NSF CCF-1049290. BST is pleased to acknowledge support from NSF grant DMS-1127914 to the Statistics and Applied Mathematics Institute. The work of LHL is partially supported by NSF IIS-1546413, NSF DMS-1854831, and DARPA HR00112190040. The contribution of LL was supported by IIS 1663870, DMS-1654579, DMS-2113642 and a grant R01ES017240 from the National Institute of Environmental Health Sciences (NIEHS) of the National Institute of Health (NIH).