Clustering from Categorical Data Sequences

Abstract

The three-parameter cluster model is a combinatorial stochastic process that generates categorical response sequences by randomly perturbing a fixed clustering parameter. This clear relationship between the observed data and the underlying clustering is particularly attractive in cluster analysis, in which supervised learning is a common goal and missing data is a familiar issue. The model is well equipped for this task, as it can handle missing data, perform out-of-sample inference, and accommodate both independent and dependent data sequences. Moreover, its clustering parameter lies in the unrestricted space of partitions, so that the number of clusters need not be specified beforehand. We establish these and other theoretical properties and also demonstrate the model on datasets from epidemiology, genetics, political science, and legal studies.

• Classification
• Random partition
• Supreme court
• Voting data

Additional information

Notes on contributors

Harry Crane

Harry Crane is Professor, Department of Statistics and Biostatistics, Rutgers University, 110 Frelinghuysen Road, Room 501, Hill Center, Piscataway, NJ 08854 (E-mail: [email protected]). Author is partially supported by NSF grant DMS-1308899 and NSA grant H98230-13-1-0299. The author is indebted to several people. Hon. Jud Crandal's many suggestions improved the exposition. Marcin Hitczenko provided references to Bork (1997) and Toobin (2008) and enriched the Supreme Court example through lively discussions. Steinback Polinski provided early insights into the applications in Sections 7.1 and 7.2. Adrian Di Antonio lent some biological expertise for Section 7.3.