Splitting Methods for Convex Clustering

Abstract

Clustering is a fundamental problem in many scientific applications. Standard methods such as k-means, Gaussian mixture models, and hierarchical clustering, however, are beset by local minima, which are sometimes drastically suboptimal. Recently introduced convex relaxations of k-means and hierarchical clustering shrink cluster centroids toward one another and ensure a unique global minimizer. In this work, we present two splitting methods for solving the convex clustering problem. The first is an instance of the alternating direction method of multipliers (ADMM); the second is an instance of the alternating minimization algorithm (AMA). In contrast to previously considered algorithms, our ADMM and AMA formulations provide simple and unified frameworks for solving the convex clustering problem under the previously studied norms and open the door to potentially novel norms. We demonstrate the performance of our algorithm on both simulated and real data examples. While the differences between the two algorithms appear to be minor on the surface, complexity analysis and numerical experiments show AMA to be significantly more efficient. This article has supplementary materials available online.

Keywords:

• Alternating direction method of multipliers
• Alternating minimization algorithm
• Convex optimization
• Hierarchical clustering
• k-means
• Regularization paths

ACKNOWLEDGMENTS

The authors thank Jocelyn Chi, Daniel Duckworth, Tom Goldstein, Rob Tibshirani, and Genevera Allen for their helpful comments and suggestions. All plots were made using the open source R package ggplot2 (Wickham 2009). This research was supported by the NIH United States Public Health Service grants GM53275 and HG006139.

Additional information

Notes on contributors

Eric C. Chi

Eric C. Chi is Postdoctoral Research Associate, Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005 (E-mail: [email protected]). Kenneth Lange is Professor, Departments of Biomathematics, Human Genetics, and Statistics, University of California, Los Angeles, CA 90095 (E-mail: [email protected]).

Kenneth Lange

Eric C. Chi is Postdoctoral Research Associate, Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005 (E-mail: [email protected]). Kenneth Lange is Professor, Departments of Biomathematics, Human Genetics, and Statistics, University of California, Los Angeles, CA 90095 (E-mail: [email protected]).