Prior near ignorance for inferences in the k-parameter exponential family

Abstract

This paper proposes a model of prior ignorance about a multivariate variable based on a set of distributions $\mathcal{M}$. In particular, we discuss four minimal properties that a model of prior ignorance should satisfy: invariance, near ignorance, learning and convergence. Near ignorance and invariance ensure that our prior model behaves as a vacuous model with respect to some statistical inferences (e.g. mean, credible intervals, etc.) and some transformation of the parameter space. Learning and convergence ensure that our prior model can learn from data and, in particular, that the influence of $\mathcal{M}$ on the posterior inferences vanishes with increasing numbers of observations. We show that these four properties can all be satisfied by a set of conjugate priors in the multivariate exponential families if the set $\mathcal{M}$ includes finitely additive probabilities obtained as limits of truncated exponential functions. The obtained set $\mathcal{M}$ is a model of prior ignorance with respect to the functions (queries) that are commonly used for statistical inferences and, because of conjugacy, it is tractable and easy to elicit. Applications of the model to some practical statistical problems show the effectiveness of the approach.

Keywords:

• prior ignorance
• exponential families
• lower and upper expectations

AMS Subject Classification:

• 62C10
• 62F15

Acknowledgements

The authors would like to thank the anonymous referees for comments and constructive criticism that helped us to improve the presentation of the paper.

Notes

1. More precisely, $\mathcal{F}$ denotes a semigroup of transformations of $\mathcal{W}$. That is, each $f\in \mathcal{F}$ maps $\mathcal{W}$ into itself, and the composition f₁f₂ defined by f₁(f₂(w)) is in $\mathcal{W}$ whenever $f_1,f_2\in \mathcal{F}$. The semigroup $\mathcal{F}$ is Abelian if f₁f₂=f₂f₁ whenever $f_1,f_2\in \mathcal{F}$.

2. In this paper we mainly focus on translation invariance. However, for multivariate models, we will impose other invariance properties: invariance to permutations and invariance to representation.

3. Note that I_{A} is the indicator function of set A, that is, I_{A}(x)=1 if x∈A and zero otherwise.

4. Equivalently, if −g and −g(f) belong to ${\mathcal{G}}_1$ then $\underset{\_}{E}[-g(f)]=\underset{\_}{E}[-g]$, which implies that $\overline{E}[g(f)]=\overline{E}[g]$ being $\overline{E}[g]=-\underset{\_}{E}[-g]$ for any g.

5. We point the reader to [16, Chapter 20] for a general discussion about dominated priors. When the likelihood belongs to the exponential families (the focus of this paper), as dominated prior we may consider any proper conjugate prior, the improper uniform or other sufficiently regular priors. The posterior becomes asymptotically Normal in these cases.

6. Let the least term φ of a sequence be a term which is smaller than all but a finite number of the terms which are equal to φ. Then φ is called the lower limit of the sequence.

7. The differences are due to the Jacobians of the transformations.

8. With sufficiently smooth RVBFs, we mean integrable w.r.t. the kernel $\exp (n({\hat{y}}_n^{\mathrm{T}}w-b(w)))\exp (\ell w)$ for any ℓ∈[−c,c], $n\in \mathbb{N}$ and ${\hat{y}}_n\in Cl({\mathcal{Y}}_0),$ with support in $\mathcal{W}$ and continuous on a neighbourhood of the point where the posterior relative to the improper uniform prior concentrates for $n\to \mathrm{\infty }$.

9. This holds for any ℓ∈(0,c]. All ℓ∈(0,c] are equivalent w.r.t. this property, since all $\exp (\ell w)$ are increasing in $\mathcal{W}$ for ℓ>0.

10. Notice that this behaviour in general is not monotone and depends on how ${\hat{y}}_n$ converges with the number of observations.

11. This also holds for the exponential distribution for n₀<−1.

12. Since the priors in $\mathcal{M}$ are all countably additive, it also satisfies strong coherence as defined in [6, Chapter 7].

13. In the formulation we need to impose the additional constraint that the argument of the square root is positive. This implies that the parameters ℓ_·1 and ℓ_·2 cannot vary independently in [−c_·1,c_·1] and, respectively, [−c_·2,c_·2]. However, for suitably large n and non-degenerate distributions, the argument of the square root is usually positive for any ℓ_·1 and ℓ_·2 in the intervals.

14. For Δ=0 the plot actually reports the Type I error.

15. We have chosen this interval in analogy with that of the IEM test. The distribution of the p-values for a different boundary of the ‘no decision zone’ can easily be deduced from Figure 2(c) and 2(d).

16. These lower and upper probabilities are obtained by densities in $\mathcal{M}$ approaching the extreme priors in Equations (42)–(43).

Additional information

Funding

This work was partly supported by the Swiss NSF [grant number 200021146606/1], [grant number 200020137680/1].