Dimensionless Measures of Variability and Dependence for Multivariate Continuous Distributions

Abstract

In this article, we suggest dimensionless descriptive measures of multivariate variability and dependence which can be used in comparisons of random vectors of different dimensions. Our work generalizes the measures of scatter and linear dependence proposed by Peña and Rodríguez (2003). The measure of variability we introduce is the rth root of the (transformed) entropy and the measure of dependence is based on the mutual information between the components of an r-dimensional random vector, capturing general stochastic dependence instead of merely linear dependence. We further investigate decompositions of the measure of variability into a measure of scale and a measure of stochastic dependence. The decomposition resulting from independent components provides a representation of variability by the scale of independent components and thus generalizes the explanation of covariance by principal components in classical multivariate analysis. We illustrate our ideas for examples of non Gaussian random vectors.

Keywords:

• Entropy
• Independent components
• Mutual information

Mathematics Subject Classification:

• Primary 62H20
• Secondary 62B10

Acknowledgments

The article was initiated while the second author visited Madrid in 2003, a collaboration funded by grant SEJ2004-03303 of the Ministry of Education, Spain.