Mutual information as a measure of multivariate association: analytical properties and statistical estimation

References

• Schmid , F. , Schmidt , R. , Blumentritt , T. , Gaißer , S. and Ruppert , M. Copula-based measures of multivariate association . Copula Theory and its Applications – Proceedings of the Workshop held in Warsaw, 25–26 September 2009 . Edited by: Jaworski , P. , Durante , F. , Härdle , W. and Rychlik , T. pp. 209 – 235 . Berlin, Heidelberg : Springer .
   Google Scholar
• Kullback , S. and Leibler , R. A. 1951 . On information and sufficiency . Ann. Math. Statist. , 22 : 79 – 86 .
   Google Scholar
• Kullback , S. 1952 . An application of information theory to multivariate analysis . Ann. Math. Statist. , 23 : 88 – 102 .
   Google Scholar
• Linfoot , E. H. 1957 . An informational measure of correlation . Inf. Control , 1 : 85 – 89 .
   Google Scholar
• Kullback , S. 1959 . Information Theory and Statistics , New York : Wiley .
   Google Scholar
• Joe , H. 1987 . Majorization, randomness and dependence for multivariate distributions . Ann. Probab. , 15 : 1217 – 1225 .
   Web of Science ®Google Scholar
• Joe , H. 1989b . Relative entropy measures of multivariate dependence . J. Amer. Statist. Assoc. , 84 : 157 – 164 .
   Web of Science ®Google Scholar
• Darbellay , G. A. 1999 . An estimator of the mutual information based on a criterion for independence . Comput. Statist. Data Anal. , 32 : 1 – 17 .
   Web of Science ®Google Scholar
• Darbellay , G. A. and Vajda , I. 1999 . Estimation of the information by an adaptive partitioning of the observation space . IEEE Trans. Inform. Theory , 45 : 1315 – 1321 .
   Web of Science ®Google Scholar
• Kraskov , A. , Stögbauer , H. and Grassberger , P. 2004 . Estimating mutual information . Phys. Rev. E , 9 : 066138/1 – 066138/16 .
   Google Scholar
• Joe , H. 1989a . Estimation of entropy and other functionals of a multivariate density . Ann. Inst. Statist. Math. , 41 : 683 – 697 .
   Web of Science ®Google Scholar
• Györfi , L. and van der Meulen , E. C. 1989 . “ An entropy estimate based on a kernel density estimation ” . In Colloquia Mathematica Societatis János Bolyai 57, Limit Theorems in Probability and Statistics , Edited by: Berkes , I. , Csáki , E. and Révész , P. 229 – 240 . Amsterdam : North Holland .
   Google Scholar
• Beirlant , J. , Dudewicz , E. J. , Györfi , L. and van der Meulen , E. C. 1997 . Nonparametric entropy estimation: An overview . Int. J. Math. Stat. Sci. , 6 : 17 – 39 .
   Google Scholar
• Hall , P. and Morton , S. C. 1993 . On the estimation of entropy . Ann. Inst. Statist. Math. , 45 : 69 – 88 .
   Web of Science ®Google Scholar
• Sklar , A. 1959 . Fonctions de répartition à n dimensions et leurs marges . Publ. Inst. Statist. Univ. Paris , 8 : 229 – 231 .
   Google Scholar
• Nelsen , R. B. 2006 . An Introduction to Copulas , 2 , New York : Springer .
   Google Scholar
• Joe , H. 1997 . Multivariate Models and Dependence Concepts , London : Chapman & Hall .
   Google Scholar
• Cherubini , U. , Luciano , E. and Vecchiato , W. 2004 . Copula Methods in Finance , Chichester : Wiley .
   Google Scholar
• McNeil , A. J. and Nešlehová , J. 2009 . Multivariate Archimedean copulas, d-monotone functions and l 1-norm symmetric distributions . Ann. Statist. , 37 : 3059 – 3097 .
   Web of Science ®Google Scholar
• Pearson , K. 1904 . “ Mathematical contributions to the theory of evolution, XIII: On the theory of contingency and its relation to association and normal correlation ” . In Draper's Company Research Memoirs (Biometric Series I) , London : University College . (reprinted in Karl Peason's early statistical papers, E.S. Pearson, ed., University Press, Cambridge, 1948, 1956, 443–475)
   Google Scholar
• Guerrero-Cusumano , J.-L. 1996a . An asymptotic test of independence for multivariate t and Cauchy random variables with applications . Inform. Sci. , 92 : 33 – 45 .
   Web of Science ®Google Scholar
• Guerrero-Cusumano , J.-L. 1996b . A measure of total variability for the multivariate t distribution with applications to finance . Inform. Sci. , 92 : 47 – 63 .
   Web of Science ®Google Scholar
• Wu , F. , Valdez , E. A. and Sherris , M. 2007 . Simulating from exchangeable multivariate Archimedean copulas . Comm. Statist. Simulation Comput. , 36 : 1019 – 1034 .
   Web of Science ®Google Scholar
• Darbellay , G. A. and Vajda , I. 1998 . “ Entropy expressions for multivariate continuous distributions ” . Prague : The Institute of Information Theory and Automation . Research Report No. 1920
   Google Scholar
• Scott , D. W. 1992 . Multivariate Density Estimation – Theory, Practice and Visualization , New York : Wiley .
   Google Scholar
• Wand , M. P. and Jones , M. C. 1995 . Kernel Smoothing , London : Chapman & Hall .
   Google Scholar
• Charpentier , A. , Fermanian , J. D. and Scaillet , O. 2006 . “ The estimation of copulas: Theory and practice ” . In Copulas: From Theory to Application in Finance , Edited by: Rank , J. 35 – 60 . London : Risk Book .
   Google Scholar
• Silverman , B. W. 1986 . Density Estimation for Statistics and Data Analysis , New York : Chapman & Hall .
   Google Scholar
• Loftsgaarden , D. O. and Quesenberry , C. P. 1965 . A nonparametric estimate of a multivariate density function . Ann. Math. Statist. , 36 : 1049 – 1051 .
   Google Scholar
• Moore , D. S. and Yackel , J. W. 1977 . “ Large sample properties of nearest neighbor density function estimators ” . In Statistical Decision Theory and Related Topics , Edited by: Gupta , S. S and Moore , D. S. 269 – 279 . New York : Academic Press .
   Google Scholar
• Hall , P. 1983 . On near neighbour estimates of a multivariate density . J. Multivariate Anal. , 13 : 24 – 39 .
   Web of Science ®Google Scholar
• Brown , B. M. and Chen , S. X. 1999 . Beta-Bernstein smoothing for regression curves with compact support . Scand. J. Statist. , 26 : 47 – 59 .
   Web of Science ®Google Scholar
• Chen , S. X. 2000 . Beta kernel smoothers for regression curves . Statist. Sinica , 10 : 73 – 91 .
   Web of Science ®Google Scholar
• Chen , S. X. 1999 . Beta kernel estimators for density functions . Comput. Statist. Data Anal. , 31 : 131 – 145 .
   Web of Science ®Google Scholar
• Bouezmarni , T. and Rolin , J.-M. 2003 . Consistency of the beta kernel density function estimator . Can. J. Stat. , 31 : 89 – 98 .
   Web of Science ®Google Scholar
• Sancetta , A. and Satchell , S. 2004 . The Bernstein copula and its applications to modelling and approximations of multivariate distributions . Econometric Theory , 20 : 535 – 562 .
   Web of Science ®Google Scholar
• Bouezmarni , T. , Rombouts , J. V.K. and Taamouti , A. 2010 . Asymptotic properties of the Bernstein density copula estimator for α-mixing data . J. Multivariate Anal. , 101 : 1 – 10 .
   Web of Science ®Google Scholar
• Wolff , E. F. 1980 . n-dimensional measures of dependence . Stochastica , IV : 175 – 188 .
   Google Scholar
• Schmid , F. and Schmidt , R. 2007 . Multivariate extensions of Spearman's rho and related statistics . Statist. Probab. Lett. , 77 : 407 – 416 .
   Web of Science ®Google Scholar
• Gaißer , S. , Ruppert , M. and Schmid , F. 2010 . A multivariate version of Hoeffding's Phi-Square . J. Multivariate Anal. , 101 : 2571 – 2586 .
   Web of Science ®Google Scholar
• Ihara , S. 1993 . Information Theory for Continuous Systems , Singapore : World Scientific .
   Google Scholar
• Darbellay , G. A. 1998 . “ Predictability: An information-theoretic perspective ” . In Signal Analysis and Prediction , Edited by: Procházka , A. 249 – 262 . Boston : Birkhäuser .
   Google Scholar