Behaviour of the Gibbs sampler when conditional distributions are potentially incompatible

Abstract

The Gibbs sampler has been used extensively in the statistics literature. It relies on iteratively sampling from a set of compatible conditional distributions and the sampler is known to converge to a unique invariant joint distribution. However, the Gibbs sampler behaves rather differently when the conditional distributions are not compatible. Such applications have seen increasing use in areas such as multiple imputation. In this paper, we demonstrate that what a Gibbs sampler converges to is a function of the order of the sampling scheme. Besides providing the mathematical background of this behaviour, we also explain how that happens through a thorough analysis of the examples.

Keywords:

• Gibbs chain
• Gibbs sampler
• potentially incompatible conditional-specified distribution

Acknowledgements

The work was supported by NIH grants 1R21AG042761-01 and 1U01HL101066-01.