Eliciting expert judgements about a set of proportions

Abstract

Eliciting expert knowledge about several uncertain quantities is a complex task when those quantities exhibit associations. A well-known example of such a problem is eliciting knowledge about a set of uncertain proportions which must sum to 1. The usual approach is to assume that the expert's knowledge can be adequately represented by a Dirichlet distribution, since this is by far the simplest multivariate distribution that is appropriate for such a set of proportions. It is also the most convenient, particularly when the expert's prior knowledge is to be combined with a multinomial sample since then the Dirichlet is the conjugate prior family. Several methods have been described in the literature for eliciting beliefs in the form of a Dirichlet distribution, which typically involve eliciting from the expert enough judgements to identify uniquely the Dirichlet hyperparameters. We describe here a new method which employs the device of over-fitting, i.e. eliciting more than the minimal number of judgements, in order to (a) produce a more carefully considered Dirichlet distribution and (b) ensure that the Dirichlet distribution is indeed a reasonable fit to the expert's knowledge. The method has been implemented in a software extension of the Sheffield elicitation framework (SHELF) to facilitate the multivariate elicitation process.

Keywords:

• elicitation
• Dirichlet distribution
• SHELF
• over-fitting
• multinomial distribution

Acknowledgements

The first named author is grateful for support from the Programa de Mejoramiento del Profesorado (PROMEP) and the Instituto Mexicano del Seguro social (IMSS) to carry out this work.