A Penalized Likelihood Method for Classification With Matrix-Valued Predictors

ABSTRACT

We propose a penalized likelihood method to fit the linear discriminant analysis model when the predictor is matrix valued. We simultaneously estimate the means and the precision matrix, which we assume has a Kronecker product decomposition. Our penalties encourage pairs of response category mean matrix estimators to have equal entries and also encourage zeros in the precision matrix estimator. To compute our estimators, we use a blockwise coordinate descent algorithm. To update the optimization variables corresponding to response category mean matrices, we use an alternating minimization algorithm that takes advantage of the Kronecker structure of the precision matrix. We show that our method can outperform relevant competitors in classification, even when our modeling assumptions are violated. We analyze three real datasets to demonstrate our method’s applicability. Supplementary materials, including an R package implementing our method, are available online.

KEYWORDS:

• Alternating minimization algorithm
• Classification
• Penalized likelihood

Supplementary Materials

• Appendix: Includes simulations comparing our method to the methods proposed by Zhong and Suslick (2015) and vector-valued sparse linear discriminant methods; simulations illustrating the efficiency gained by joint estimation of the μ_*j’s, Δ_*, and Φ_* using (3(3) $$\begin{array}{ccc}\hfill (\widehat{\mu },\widehat{\Phi },\widehat{\Delta })& =& \underset{(\mu ,\Phi ,\Delta )\in \mathcal{T}}{error}\{g(\mu ,\Phi ,\Delta )+{\lambda }_1\sum _{j<m}{\parallel w_{j,m}\circ ({\mu }_j-{\mu }_m)\parallel }_1\hfill \\ & & +{\lambda }_2{\parallel \Delta \otimes \Phi \parallel }_1\},\text{subject}\text{to}{\parallel \Phi \parallel }_1=r\hfill \end{array}$$(3) ); and simulations investigating the sensitivity of (3(3) $$\begin{array}{ccc}\hfill (\widehat{\mu },\widehat{\Phi },\widehat{\Delta })& =& \underset{(\mu ,\Phi ,\Delta )\in \mathcal{T}}{error}\{g(\mu ,\Phi ,\Delta )+{\lambda }_1\sum _{j<m}{\parallel w_{j,m}\circ ({\mu }_j-{\mu }_m)\parallel }_1\hfill \\ & & +{\lambda }_2{\parallel \Delta \otimes \Phi \parallel }_1\},\text{subject}\text{to}{\parallel \Phi \parallel }_1=r\hfill \end{array}$$(3) ) to the choice of weights.

• Code: Includes R scripts to create the real datasets we analyze in Section 5, and to reproduce the simulation results.

• MatrixLDA: An R package implementing our method, along with auxiliary functions for prediction and tuning parameter selection.

Acknowledgments

The authors thank the associate editor and referees for helpful comments.

Additional information

Funding

This research was supported in part by the Doctoral Dissertation Fellowship from the University of Minnesota and the National Science Foundation grant DMS-1452068.