Bayesian Factorizations of Big Sparse Tensors

Abstract

It has become routine to collect data that are structured as multiway arrays (tensors). There is an enormous literature on low rank and sparse matrix factorizations, but limited consideration of extensions to the tensor case in statistics. The most common low rank tensor factorization relies on parallel factor analysis (PARAFAC), which expresses a rank k tensor as a sum of rank one tensors. In contingency table applications in which the sample size is massively less than the number of cells in the table, the low rank assumption is not sufficient and PARAFAC has poor performance. We induce an additional layer of dimension reduction by allowing the effective rank to vary across dimensions of the table. Taking a Bayesian approach, we place priors on terms in the factorization and develop an efficient Gibbs sampler for posterior computation. Theory is provided showing posterior concentration rates in high-dimensional settings, and the methods are shown to have excellent performance in simulations and several real data applications.

• Bayesian
• Categorical data
• Contingency table
• Log-linear model
• Low rank
• PARAFAC
• Sparsity
• Tensor factorization.

Notes

For p = 2, ${\psi }^{\left(1\right)}\otimes {\psi }^{\left(2\right)}={\psi }^{\left(1\right)}{\psi }^{\left(2\right)\mathrm{T}}$. In general, ${({\psi }^{\left(1\right)}\otimes \cdots \otimes {\psi }^{\left(p\right)})}_{c_1...c_p}={\psi }_{c_1}^{\left(1\right)}...{\psi }_{c_p}^{\left(p\right)}$.

Mult({1, …, d}; λ₁, …, λ_d) denotes a discrete distribution on {1, …, d} with probabilities λ₁, …, λ_d associated to each atom.

For sequences a_n, b_n, we write a_n = o(b_n) if a_n/b_n → 0 as n → ∞ and a_n = O(b_n) if a_n ⩽ Cb_n for all large n.

Given a metric space $(\mathcal{X},d)$, let $N(ϵ;\mathcal{X},d)$ denote its ε-covering number, that is, the minimum number of d-balls of radius ε needed to cover $\mathcal{X}$.

Additional information

Notes on contributors

Jing Zhou

Jing Zhou, School of Public Health, University of North Carolina, Chapel Hill, NC 27599 (E-mail: [email protected]). Anirban Bhattacharya is Assistant Professor, Department of Statistics, Texas A&M University, 3143 TAMU, College Station, TX 77843 (E-mail: [email protected]). Amy H. Herring, Department of Biostatistics, The University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 (E-mail: [email protected]). David B. Dunson is Professor, Department of Statistical Science, Box 90251, 214 Old Chemistry Building, Duke University, Durham, NC 27708-0251 (E-mail: [email protected]). The research was partially supported by grant number R01 ES017240-01 from the National Institute of Environmental Health Sciences (NIEHS) of the National Institutes of Health (NIH). The second author would like to acknowledge support from the Office of Naval Research (ONR BAA 14-0001).

Anirban Bhattacharya

Jing Zhou, School of Public Health, University of North Carolina, Chapel Hill, NC 27599 (E-mail: [email protected]). Anirban Bhattacharya is Assistant Professor, Department of Statistics, Texas A&M University, 3143 TAMU, College Station, TX 77843 (E-mail: [email protected]). Amy H. Herring, Department of Biostatistics, The University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 (E-mail: [email protected]). David B. Dunson is Professor, Department of Statistical Science, Box 90251, 214 Old Chemistry Building, Duke University, Durham, NC 27708-0251 (E-mail: [email protected]). The research was partially supported by grant number R01 ES017240-01 from the National Institute of Environmental Health Sciences (NIEHS) of the National Institutes of Health (NIH). The second author would like to acknowledge support from the Office of Naval Research (ONR BAA 14-0001).

Amy H. Herring

Jing Zhou, School of Public Health, University of North Carolina, Chapel Hill, NC 27599 (E-mail: [email protected]). Anirban Bhattacharya is Assistant Professor, Department of Statistics, Texas A&M University, 3143 TAMU, College Station, TX 77843 (E-mail: [email protected]). Amy H. Herring, Department of Biostatistics, The University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 (E-mail: [email protected]). David B. Dunson is Professor, Department of Statistical Science, Box 90251, 214 Old Chemistry Building, Duke University, Durham, NC 27708-0251 (E-mail: [email protected]). The research was partially supported by grant number R01 ES017240-01 from the National Institute of Environmental Health Sciences (NIEHS) of the National Institutes of Health (NIH). The second author would like to acknowledge support from the Office of Naval Research (ONR BAA 14-0001).

David B. Dunson

Jing Zhou, School of Public Health, University of North Carolina, Chapel Hill, NC 27599 (E-mail: [email protected]). Anirban Bhattacharya is Assistant Professor, Department of Statistics, Texas A&M University, 3143 TAMU, College Station, TX 77843 (E-mail: [email protected]). Amy H. Herring, Department of Biostatistics, The University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 (E-mail: [email protected]). David B. Dunson is Professor, Department of Statistical Science, Box 90251, 214 Old Chemistry Building, Duke University, Durham, NC 27708-0251 (E-mail: [email protected]). The research was partially supported by grant number R01 ES017240-01 from the National Institute of Environmental Health Sciences (NIEHS) of the National Institutes of Health (NIH). The second author would like to acknowledge support from the Office of Naval Research (ONR BAA 14-0001).