A Mass-Shifting Phenomenon of Truncated Multivariate Normal Priors

Abstract

We show that lower-dimensional marginal densities of dependent zero-mean normal distributions truncated to the positive orthant exhibit a mass-shifting phenomenon. Despite the truncated multivariate normal density having a mode at the origin, the marginal density assigns increasingly small mass near the origin as the dimension increases. The phenomenon accentuates with stronger correlation between the random variables. This surprising behavior has serious implications toward Bayesian constrained estimation and inference, where the prior, in addition to having a full support, is required to assign a substantial probability near the origin to capture flat parts of the true function of interest. A precise quantification of the mass-shifting phenomenon for both the prior and the posterior, characterizing the role of the dimension as well as the dependence, is provided under a variety of correlation structures. Without further modification, we show that truncated normal priors are not suitable for modeling flat regions and propose a novel alternative strategy based on shrinking the coordinates using a multiplicative scale parameter. The proposed shrinkage prior is shown to achieve optimal posterior contraction around true functions with potentially flat regions. Synthetic and real data studies demonstrate how the modification guards against the mass shifting phenomenon while retaining computational efficiency. Supplementary materials for this article are available online.

KEYWORDS:

• Bayesian
• Basis expansion
• Comparison inequality
• Constrained estimation
• Gaussian process
• Shrinkage

Supplementary Materials

The supplemental materials contain additional plots, proofs of main results in the manuscript, as well as remaining technical results and additional details and analyses on numerical studies. Code and data (sources) are also included that are needed to replicate the results presented in both simulations and real applications of the manuscript.

Disclosure Statement

The authors report there are no competing interests to declare.

Funding

Notes

1 The prior probability assigned to arbitrarily small Kullback–Leibler neighborhoods of any point is positive.

2 ${\mathbb{P}}_0$ denotes the true data generating measure.

Additional information

Funding

Pati and Bhattacharya acknowledge support from NSF DMS (1854731, 1916371). In addition, Bhattacharya acknowledges the NSF CAREER 1653404 award for supporting this project.