Jaccard matrix for nonlinear filter statistics

ABSTRACT

We propose the Jaccard matrix (JM) and the Jaccard cell (JC), define them as the extended concepts of the Jaccard index, and theoretically and numerically analyze them. The data on the Euclidean plane can derive the JM as a sparse matrix. We show the JC inherits the feature of similarity of the Jaccard index as the exponential function of mutual information. We theoretically and numerically confirm that the local correlation coefficient of the data on the Euclidean plane relates the JC to the mutual information. Although one could potentially select an arbitrary cell size of the grid to make the JM, the knowledge we can obtain from the matrix decreases if the cell size is too big or too small to distinguish the data clusters appropriately. Therefore, the JM needs a computational procedure to determine the cell size within the appropriate scale. Maximizing the variance of the JCs supports determining the unique cell size, which value locates in the middle range of the parabolic function of the cell-size parameter. The JM could derive an index extracting nonlinear correlation of the data. The maximized standard deviation of the JCs as such an index is a decreasing function of the noise scale of the data under the constraint conditions. The ability to determine the homogeneous rectangular grid pattern of the JM might be a significant feature for finding nonlinear correlation. We would summarize this study as that of a nonlinear filter working as an efficient component of explainable AI and statistics.

KEYWORDS:

• Jaccard index
• nonlinear correlation
• sparse matrix
• mutual information
• data mining
• variance maximization

Acknowledgments

We are grateful to the editors and anonymous reviewers for giving valuable and dedicated comments to improve this research.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Moto Kamiura

Moto Kamiura received B.Sc. in 2002 from Toho University, Japan; M.Sc. in 2004, and Ph.D. in Physics in 2007 from Kobe University, Japan. He was an assistant professor at Tokyo University of Science from 2008 to 2011; a Cooperative Researcher at the Research Institute of Electrical Communication, Tohoku University from 2010 to 2016; an assistant professor at Tokyo Denki University from 2011 to 2018; Co-Founder/Vice-President of Felixia Inc. in Japan from 2017 to the present. He is presently an associate professor at the Institute for Advanced Research and Education, Doshisha University, Japan.

Ryo Sekine

Ryo Sekine received B.Info. in 2016 and M.Info. in 2018 from Tokyo Denki University, Japan. He is presently an engineer at Japan Information Processing Service Co., Ltd., Japan.