A posterior convergence rate theorem for general Markov chains

Abstract

This paper establishes a posterior convergence rate theorem for general Markov chains. Our approach is based on the Hausdorff α-entropy introduced by Xing (Electronic Journal of Statistics 2:848–62, 2008) and Xing and Ranneby (Journal of Statistical Planning and Inference 139 (7):2479–89, 2009). As an application we illustrate our results on a non linear autoregressive model.

Keywords:

• Density function
• Hausdorff entropy
• Hellinger metric
• Markov chain
• posterior distribution
• rate of convergence

AMS CLASSIFICATION:

• 62G20
• 62G07
• 62F15

1. Introduction

The aim of this paper is to study the asymptotic behavior of posterior distributions based on observations which arise from Markov chains. Let $X_0,X_1,\dots$ be a Markov chain with transition density $p_{\theta }(y|x)$ and initial density $q_{\theta }(x_0)$ with respect to some σ-finite measure μ on a measurable space $(\mathfrak{X},\mathcal{A}).$ We assume that the function $x\mapsto q_{\theta }(x)$ and the 2-variable function $(x,y)\mapsto p_{\theta }(y|x)$ are measurable for all parameters θ in the parameter set $\Theta .$ So the joint distribution $P_{\theta }^{(n)}$ of $X^{(n)}=\{X_0,X_1,\dots ,X_n\}$ has a density given by $p_{\theta }^{(n)}(x^{(n)})=q_{\theta }(x_0){\prod }_{i=1}^n{p}_{\theta }(x_i|x_{i-1})$ relative to the product measure $\mu (x_0)\mu (x_1)\dots \mu (x_n),$ where the parameter θ does not depend on the size of $X^{(n)}.$ Denote by θ₀ the true parameter generating the observations $X_0,X_1,\dots .$ Note that any semimetric $d((q_{{\theta }_1},p_{{\theta }_1}),(q_{{\theta }_2},p_{{\theta }_2}))$ on the product space of the initial densities and the transition densities induces naturally a semimetric $d({\theta }_1,{\theta }_2)=d((q_{{\theta }_1},p_{{\theta }_1}),(q_{{\theta }_2},p_{{\theta }_2}))$ on Θ when the mapping $\theta \mapsto (q_{\theta },p_{\theta })$ is one-to-one, which is assumed in the paper. Given a prior Π on Θ, the posterior distribution $\Pi (· |X_0,X_1,\dots ,X_n)$ is a random probability measure given by $$\Pi (B |X_0,X_1,\dots ,X_n)=\frac{{\int }_B p_{\theta }^{(n)}(X^{(n)}) \Pi (d\theta )}{{\int }_{\Theta } p_{\theta }^{(n)}(X^{(n)}) \Pi (d\theta )}=\frac{{\int }_B R_{\theta }^{(n)}(X^{(n)}) \Pi (d\theta )}{{\int }_{\Theta } R_{\theta }^{(n)}(X^{(n)}) \Pi (d\theta )}$$ for each measurable subset B in Θ, where $R_{\theta }^{(n)}(X^{(n)})=p_{\theta }^{(n)}(X^{(n)})/p_{{\theta }_0}^{(n)}(X^{(n)})$ stands for the likelihood ratio.

Recall that the posterior distribution $\Pi (· |X_0,X_1,\dots ,X_n)$ is said to be convergent almost surely at a rate at least ${\epsilon }_n$ if there exists r > 0 such that $$\Pi (\theta \in \Theta : d(\theta ,{\theta }_0)\ge r{\epsilon }_n |X_0,X_1,\dots ,X_n)\to 0\hspace{1em}\text{almost surely as}\hspace{1em}n\to \infty .$$ Posterior consistency is an important issue in Bayesian analysis. Much works were concerned with the asymptotic behavior of posterior distributions for independent and identically distributed observations, see, for instance, Barron, Schervish, and Wasserman (1999), Shen and Wasserman (2001), Ghosal and van der Vaart (2007), Walker, Lijoi, and Prunster (2007), Walker (2003), Walker (2004), Xing and Ranneby (2009), Xing (2011a) and Xing (2011b). An old well-known approach is based on the existence of uniformly consistent tests. In this paper we use an integration condition together with the Hausdorff α-entropy to study convergence rates of posteriors when the observations are not independent and identically distributed. The integration condition and the Hausdorff α-entropy have an advantage in applications, because they both are prior-dependent. The Hausdorff α-entropy condition was introduced in Xing (2008) and Xing and Ranneby (2009) and it is weaker than the metric entropy condition. By means of the integration condition and the Hausdorff α-entropy, we establish a posterior convergence rate theorem for general Markov chains. As applications we discuss the posterior rate of convergence for the non linear autoregressive model.

The layout of this paper is as follows. In Sec. 2 we present a prior-dependent integration inequality and show one type of general posterior convergence rate theorem for Markov chains. In Sec. 3 we illustrate our result by finding a posterior convergence rate for non linear autoregression model. The technical proofs are collected in the Appendix.

2. A convergence rate theorem for Markov chains

In this section we introduce a prior-dependent integration condition to consistency of posterior distributions. Together with the Hausdorff α-entropy, the integration condition plays a central roll in study of the Bayesian convergence rate.

Recall that the Hausdorff α-entropy $J(\delta ,{\Theta }_1,\alpha ,d)$ for a subset ${\Theta }_1\subset \Theta$ is the logarithm of the minimal sum of αth power of prior masses of balls of d-radius $\le \delta$ needed to cover ${\Theta }_1,$ see Xing (2008) and Xing and Ranneby (2009) for the details of the Hausdorff α-entropy. For simplicity of notations, we define the Hausdorff α-constant $C(\delta ,{\Theta }_1,\alpha ,d):=e^{J(\delta ,{\Theta }_1,\alpha ,d)}$ of any subset ${\Theta }_1$ of Θ. Observe that $C(\delta ,{\Theta }_1,\alpha ,d)$ depends on the prior Π. It was proved in Xing and Ranneby (2009) that the inequality $$\Pi {({\Theta }_1)}^{\alpha }\le C(\delta ,{\Theta }_1,\alpha ,d)\le \Pi {({\Theta }_1)}^{\alpha } N{(\delta ,{\Theta }_1,d)}^{1-\alpha }$$ holds for any $0\le \alpha \le 1,$ where $N(\delta ,{\Theta }_1,d)$ denotes the minimal number of balls of d-radius $\le \delta$ needed to cover ${\Theta }_1\subset \Theta .$ We shall adopt the following Hellinger type semimetrics. $$\begin{array}{c}H(p_{{\theta }_1}(y|x),p_{{\theta }_2}(y|x))={({\int }_{\mathfrak{X}}{\int }_{\mathfrak{X}}(\sqrt{p_{{\theta }_1}(y|x)}-\sqrt{p_{{\theta }_2}(y|x)} )}^2 d\mu (y)d\nu (x){)}^{1/2},\\ H(q_{{\theta }_1}(x),q_{{\theta }_2}(x))={({\int }_{\mathfrak{X}}(\sqrt{q_{{\theta }_1}(x)}-\sqrt{q_{{\theta }_2}(x)} )}^2 d\mu (x){)}^{1/2},\\ H_{*}(p_{{\theta }_1}(y|x),p_{{\theta }_2}(y|x))\\ ={({\int }_{\mathfrak{X}}{\int }_{\mathfrak{X}}(\sqrt{p_{{\theta }_1}(y|x)}-\sqrt{p_{{\theta }_2}(y|x)} )}^2(\frac{2}{3}\sqrt{\frac{p_{{\theta }_1}(y|x)}{p_{{\theta }_2}(y|x)}}+\frac{1}{3}) d\mu (y)d\nu (x){)}^{1/2},\\ H_{*}(q_{{\theta }_1}(x),q_{{\theta }_2}(x))={({\int }_{\mathfrak{X}}(\sqrt{q_{{\theta }_1}(x)}-\sqrt{q_{{\theta }_2}(x)} )}^2(\frac{2}{3}\sqrt{\frac{q_{{\theta }_1}(x)}{q_{{\theta }_2}(x)}}+\frac{1}{3}) d\mu (x){)}^{1/2}.\end{array}$$ Denote $$W_n^1({\theta }_0,\epsilon )=\{\theta \in \Theta : H_{*}{(p_{{\theta }_0},p_{\theta })}^2+\frac{1}{n}{H}_{*}{(q_{{\theta }_0},q_{\theta })}^2\le {\epsilon }^2\}.$$ By means of the metric $d(\theta ,{\theta }_0):=H(p_{\theta },p_{{\theta }_0}),$ Ghosal and van der Vaart (2007, Theorem 5) gave an in-probability posterior convergence rate theorem for stationary α-mixing Markov chains. Since calculation of the α-mixing coefficients is generally not easy and many processes are neither mixing nor stationary, it seems worth to develop a posterior convergence rate theorem for Markov chains which may be neither stationary nor α-mixing. Now we present an almost sure assertion in this direction. Our result is based on the following prior-dependent integration condition.

Throughout this paper the notation $a\lesssim b$ means $a\le Cb$ for some positive constant C which is universal or fixed in the proof. Write $a\approx b$ if $a\lesssim b$ and $b\lesssim a.$ Denote $P{f}^{\alpha }={\int }_{\mathbb{X}}{f}^{\alpha }dP$ which is the integral of the non negative function f with power α relative to the measure P on $\mathbb{X}.$

Proposition 1.

Suppose that there exist a μ-integrable function r(y) and constants $a_1\ge a_0>0$ with $a_1\ge 1$ such that $d\nu (y)=r(y)d\mu (y)$ and $a_{0}r(y)\le p_{\theta }(y|x)\le a_{1}r(y)$ for all $\theta \in \Theta$ and $x,y\in \mathfrak{X}$. Let $0<\delta <\frac{\sqrt{a_0}}{2\sqrt{a_1}}$ and $0<\alpha <\frac{1}{2}$. Then the inequality$$\begin{array}{c}{P}_{{\theta }_0}^{(n)} {({\int }_{\{\theta \in {\Theta }_1:\mathrm{ }d(\theta ,{\theta }_0)>\epsilon \}}\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta ))}^{\alpha }\\ \le 2 e^{-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^{2}n{\epsilon }^2}C(\delta \epsilon ,\{\theta \in {\Theta }_1: d(\theta ,{\theta }_0)>\epsilon \},\alpha ,d)\end{array}$$holds for all n, $\epsilon >0$ and ${\Theta }_1\subset \Theta$, where $d(\theta ,{\theta }_0)=H(p_{\theta },p_{{\theta }_0}).$

Therefore we have

Theorem 1.

Suppose that all assumptions of Proposition 1 hold and suppose that ${\epsilon }_n>0,n {\epsilon }_n^2\ge c_0 \hspace{0.17em}\text{log}\hspace{0.17em}n$ for all large n and some fixed constant $c_0>0$. Suppose that there exist $c_1<(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2,c_2>\frac{1}{c_0}$ and a sequence of subsets Θ_n on Θ such that$$C(\delta j{\epsilon }_n,\{\theta \in {\Theta }_n:j{\epsilon }_n<d(\theta ,{\theta }_0)\le 2j{\epsilon }_n\},\alpha ,d)\le e^{c_1{j}^{2}n{\epsilon }_n^2} \Pi {(W_n^1({\theta }_0,{\epsilon }_n))}^{\alpha }$$for all large j, n, and$$\sum _{n=1}^{\infty }\frac{e^{n\mathrm{ }{\epsilon }_n^2\mathrm{ }(3a_1+4c_2)} \Pi (\Theta \setminus {\Theta }_n)}{\Pi (W_n^1({\theta }_0,{\epsilon }_n))}<\infty .$$

Then there exists b > 0 such that for each large r and all large n,$$\Pi (\theta \in \Theta : d(\theta ,{\theta }_0)\ge r {\epsilon }_n| X_0,X_1,\dots ,X_n)\le e^{-bn{\epsilon }_n^2}\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\text{almost}\mathrm{ }\text{surely}.$$

By choosing $\delta =\frac{\sqrt{a_0}}{4\sqrt{a_1}}$ and $\alpha =\frac{1}{4}$ we can easily get

Corollary 1.

Suppose that there exist a μ-integrable function r(y) and constants $a_1\ge a_0>0$ such that $d\nu (y)=r(y)d\mu (y)$ and $a_{0}r(y)\le p_{\theta }(y|x)\le a_{1}r(y)$ for all $\theta \in \Theta$ and $x,y\in \mathfrak{X}$. Suppose that ${\epsilon }_n>0,n {\epsilon }_n^2\ge c_0 \hspace{0.17em}\text{log}\hspace{0.17em}n$ for all large n and some fixed constant $c_0>0$. Suppose that there exist $c_1,c_2,c_3$ with $3c_1+c_2<a_0/16$ and $c_3>1/c_0$ and a sequence of subsets Θ_n on Θ such that for all large j and n,

1. $N(\frac{\sqrt{a_0}}{4\sqrt{a_1}}j{\epsilon }_n,\{\theta \in {\Theta }_n:j{\epsilon }_n<d(\theta ,{\theta }_0)\le 2j{\epsilon }_n\},d)\le e^{c_1{j}^{2}n{\epsilon }_n^2};$

2. $\Pi (\theta \in {\Theta }_n:j{\epsilon }_n<d(\theta ,{\theta }_0)\le 2j{\epsilon }_n)\le e^{c_2{j}^{2}n{\epsilon }_n^2} \Pi (W_n^1({\theta }_0,{\epsilon }_n));$

3. ${\sum }_{n=1}^{\infty }\frac{e^{n\mathrm{ }{\epsilon }_n^2\mathrm{ }(3a_1+4c_3)} \Pi (\Theta \setminus {\Theta }_n)}{\Pi (W_n^1({\theta }_0,{\epsilon }_n))}<\infty .$

Then there exists b > 0 such that for each large r and all large n,$$\Pi (\theta \in \Theta : d(\theta ,{\theta }_0)\ge r {\epsilon }_n| X_0,X_1,\dots ,X_n)\le e^{-bn{\epsilon }_n^2}\mathrm{\hspace{1em}}\mathit{\text{almost}}\mathrm{ }\mathit{\text{surely}}.$$

3. Non linear autoregression

In this section we discuss an application of our theorems. By means of Corollary 1, we improve on the posterior rate of convergence for the non linear autoregressive model in Ghosal and van der Vaart (2007).

We observe $X_1,X_2,\dots ,X_n$ of a time series $\{X_t:t\in \mathbb{Z}\}$ given by $$X_i=f(X_{i-1})+{\xi }_i\mathrm{\hspace{1em}}\mathrm{\hspace{1em}}\mathit{for}\hspace{1em}i=1,2,\dots ,n,$$ where ${\xi }_1,{\xi }_2,\dots ,{\xi }_n$ are i.i.d. random variables with the standard normal distribution and the unknown regression function f is in the space $\mathcal{F}$ which consists of all functions f with $\underset{x\in \mathbb{R}}{\sup }|f(x)|\le M$ for some fixed positive constant M. Let $q_f(x)$ be the density of X₀ relative to the Lebesgue measure $d\mu$ on $\mathbb{R}.$ So $X_0,X_1,\dots$ can be considered as a Markov chain generated by the transition density $p_f(y|x)=\varphi (y-f(x))$ with $\varphi (x)={(2\pi )}^{-1/2}{e}^{-x^2/2}$ and the initial density $q_f(x).$ Since $\varphi (x)$ is a strictly positive continuous function tending to zero as $x\to \pm \infty ,$ there exist two constants $0<a_0<1<a_1$ depending only on M such that $a_0\varphi (y)\le p_f(y|x)\le a_1\varphi (y)$ for all $f\in \mathcal{F}$ and $-\infty <y, x<\infty .$ Assume that there exists a constant N > 0 such that the set of initial densities of the Markov chain satisfies $H_{*}(q_{f_1},q_{f_2})\le N$ for all initial densities $q_{f_1}$ and $q_{f_2}.$ For instance, all of the initial densities with $a_0\varphi (x)\le q_f(x)\le a_1\varphi (x)$ satisfy $H_{*}(q_{f_1},q_{f_2})\le \sqrt{2}{(a_1/a_0)}^{1/4}$ and hence form a set with the requirement. Define a measure $d\nu =\varphi d\mu$ in $\mathbb{R}$ and a norm $\left|\right|f|{|}_2={({\int }_{\mathbb{R}}|f{|}^{2}d\nu )}^{1/2}$ on $\mathcal{F}.$ Assume that the true regression function $f_0\in \mathcal{F}$ belongs to the Lipschitz continuous space Lip_M, which consists of all functions f on $(-\infty ,\infty )$ satisfying $|f(x)-f(y)|\le L |x-y|$ for all $-\infty <x, y<\infty ,$ where L is a fixed positive constant. When the Markov chain is stationary, Ghosal and van der Vaart (2007, Section 7.4) constructed a prior on the regression functions and obtained the in-probability posterior convergence rate $n^{-1/3}{(\hspace{0.17em}\text{log}\hspace{0.17em}n)}^{1/2},$ which is the minimax rate times the logarithmic factor ${(\hspace{0.17em}\text{log}\hspace{0.17em}n)}^{1/2}.$ In the following we shall apply Corollary 1 to get the posterior convergence rate $n^{-1/3}{(\hspace{0.17em}\text{log}\hspace{0.17em}n)}^{1/6}$ in the almost sure sense for a general Markov chain defined as above.

First, we note that for any $f\in \mathcal{F},$ $$\begin{array}{c}{H}_{*}{(p_{f_0},p_f)}^2+\frac{1}{n} H_{*}{(q_{f_0},q_f)}^2\le \sqrt{\frac{a_1}{a_0}}H{(p_{f_0},p_f)}^2+\frac{N^2}{n}\\ =\frac{1}{2}\sqrt{\frac{a_1}{a_0}}{\int }_{-\infty }^{\infty }(1-e^{-\frac{{(f(x)-f_0(x))}^2}{4}}) d\nu (x)+\frac{N^2}{n}\le \frac{\left|\right|f-f_0|{|}_2^2}{8}\sqrt{\frac{a_1}{a_0}}+\frac{N^2}{n},\end{array}$$ where the last inequality follows from the elementary inequality $1-e^{-t}\le t.$ Hence for some small constant $b_1>0$ we have that $W_n^1(f_0,{\epsilon }_n)\supset \{f\in \mathcal{F}:||f-f_0|{|}_2\le b_1 {\epsilon }_n\}$ for all large n. Similarly, $\left|\right|f-f_0|{|}_2\approx H(p_f,p_{f_0})$ hold for all $f\in \mathcal{F}$ with $\left|\right|f-f_0|{|}_2\le 1.$ Hence Corollary 1 works well for the metric $\left|\right|·|{|}_2.$

We also need some basic facts on approximation of Lipschitz continuous functions by means of step functions. Given a finite interval $[-A_n,A_n)$ and a positive integer K_n, we make the partition $[-A_n,A_n)={\cup }_{k=1}^{K_n}{I}_k$ with $I_k=[-A_n+\frac{2A_n(k-1)}{K_n},-A_n+\frac{2A_n·k}{K_n})$ for $k=1,2,\dots ,K_n.$ Write $I_0=\mathbb{R}\setminus [-A_n,A_n).$ The space of step functions relative to the partition is the set of functions $h:[-A_n,A_n)\mapsto \mathbb{R}$ such that h is identically equal to some constant on each I_k for $k=1,2,\dots ,K_n,$ more precisely, $h(x)={\sum }_{k=1}^{K_n}{\beta }_k 1_{I_k}(x)$ for some $\beta =({\beta }_1,{\beta }_2,\dots ,{\beta }_{K_n})\in {[-M,M]}^{K_n}\subset {\mathbb{R}}^{K_n},$ where $1_{I_k}(x)$ denotes the indicator function of I_k. Denote by $f_{\beta }(x)$ the function on $(-\infty ,\infty )$ which is equal to ${\sum }_{k=1}^{K_n}{\beta }_k 1_{I_k}(x)$ on $[-A_n,A_n)$ and vanish outside $[-A_n,A_n).$ Hence $f_{\beta }\in \mathcal{F}$ and $\left|\right|f_{{\beta }_1}-f_{{\beta }_2}|{|}_2=||{\beta }_1-{\beta }_2|{|}_{*},$ where $\left|\right|\beta |{|}_{*}={({\sum }_{k=1}^{K_n}{\beta }_k^2{({\int }_{I_k}d\nu )}^2)}^{1/2}.$ Let Π be the prior on $\mathcal{F}$ which is induced by the map $\beta \mapsto f_{\beta }$ such that all the coordinates β_k of β are chosen to be i.i.d. random variables with the uniform distribution on $[-M,M].$ Hence the support ${\mathcal{F}}_{\Pi }$ of Π consists of all such functions $f_{\beta }.$ Take $A_n=2\sqrt{\hspace{0.17em}\text{log}\hspace{0.17em}(1/{\epsilon }_n)}\approx \sqrt{\hspace{0.17em}\text{log}\hspace{0.17em}n}$ and $K_n=\lfloor \frac{3L{A}_n}{b_1{\epsilon }_n}\rfloor +1$ with ${\epsilon }_n={(\frac{\sqrt{\hspace{0.17em}\text{log}\hspace{0.17em}n}}{n})}^{1/3}.$ Then $K_n\approx {(n\hspace{0.17em}\text{log}\hspace{0.17em}n)}^{1/3}\approx n{\epsilon }_n^2.$ Write ${\beta }_0=({\beta }_{0,1},{\beta }_{0,2},\dots ,{\beta }_{0,K_n})$ for ${\beta }_{0,k}=f_0(-A_n+\frac{2A_{n}k-1}{K_n}).$ Since $f_0\in \mathcal{F}\cap Li{p}_L,$ we have that $f_{{\beta }_0}\in \mathcal{F}$ and $\underset{-A_n\le x<A_n}{\sup }| f_{{\beta }_0}(x)-f_0(x) |\le L{A}_n/K_n\le b_1{\epsilon }_n/3.$ From the triangle inequality and the inequality ${\int }_x^{\infty }\varphi (t)dt\le \varphi (x)/x$ for all x > 0, it follows that for all $f_{\beta }\in {\mathcal{F}}_{\Pi }$ and for all large n, $$\begin{array}{c}| ||f_{\beta }-f_0|{|}_2-\left|\right|f_{\beta }-f_{{\beta }_0}|{|}_2 |\le \left|\right|f_{{\beta }_0}-f_0|{|}_2={({\int }_{-A_n}^{A_n}|f_0-f_{{\beta }_0}{|}^2 d\nu )}^{1/2}\\ +{({\int }_{I_0}{f}_0^2 d\nu )}^{1/2}\le \frac{b_1{\epsilon }_n}{3}{({\int }_{-A_n}^{A_n}d\nu )}^{1/2}+M{(\frac{\varphi (A_n)}{A_n})}^{1/2}\\ \le \frac{b_1{\epsilon }_n}{3}+\frac{M{\epsilon }_n}{{(2\pi )}^{1/4}{A}_n^{1/2}}\le \frac{b_1{\epsilon }_n}{2}.\end{array}$$ Thus for all large j and n, we have $$\begin{array}{c}\frac{\Pi (f_{\beta }\in {\mathcal{F}}_{\Pi }:j{\epsilon }_n<||f_{\beta }-f_0|{|}_2\le 2j{\epsilon }_n)}{\Pi (W_n^1({\theta }_0,{\epsilon }_n))}\le \frac{\Pi (f_{\beta }\in {\mathcal{F}}_{\Pi }:||f_{\beta }-f_0|{|}_2\le 2j{\epsilon }_n)}{\Pi (f_{\beta }\in {\mathcal{F}}_{\Pi }:||f_{\beta }-f_0|{|}_2\le b_1{\epsilon }_n)}\\ \le \frac{\Pi (f_{\beta }\in {\mathcal{F}}_{\Pi }:||f_{\beta }-f_0|{|}_2\le 3j{\epsilon }_n)}{\Pi (f_{\beta }\in {\mathcal{F}}_{\Pi }:||f_{\beta }-f_{{\beta }_0}|{|}_2\le \frac{b_1}{2}{\epsilon }_n)}\\ =\frac{\Pi (\beta \in {[-M,M]}^{K_n}:||\beta -{\beta }_0|{|}_{*}\le 3j{\epsilon }_n)}{\Pi (\beta \in {[-M,M]}^{K_n}:||\beta -{\beta }_0|{|}_{*}\le \frac{b_1}{2}{\epsilon }_n)}.\end{array}$$ Note that the Euclidean volume of the K_n-dimensional ellipsoid $\{\beta \in {\mathbb{R}}^{K_n}:||\beta -{\beta }_0|{|}_{*}\le r\}$ is equal to $r^{K_n}$ times the Euclidean volume of the “unit” K_n-dimensional ellipsoid $\{\beta \in {\mathbb{R}}^{K_n}:||\beta -{\beta }_0|{|}_{*}\le 1\}.$ So the last quotient does not exceed $j^{2K_n}=e^{K_n\hspace{0.17em}\text{log}\hspace{0.17em}(2j)},$ which is less than $e^{c_2{j}^{2}n{\epsilon }_n^2}$ for any given $c_2>0$ and all large j. Hence we have obtained condition (ii) of Corollary 1. Similarly, for all large j and n, we have $$\begin{array}{c}N(\frac{\sqrt{a_0}}{4\sqrt{a_1}}j{\epsilon }_n,\{f_{\beta }\in {\mathcal{F}}_{\Pi }:j{\epsilon }_n<\left|\right|f_{\beta }-f_0|{|}_2\le 2j{\epsilon }_n\},\left|\right|·\left|{|}_2\right)\\ \le N(\frac{\sqrt{a_0}}{4\sqrt{a_1}}j{\epsilon }_n,\{f_{\beta }\in {\mathcal{F}}_{\Pi }:\left|\right|f_{\beta }-f_{{\beta }_0}|{|}_2\le 3j{\epsilon }_n\},\left|\right|·\left|{|}_2\right)\\ \le N(\frac{\sqrt{a_0}}{4\sqrt{a_1}}j{\epsilon }_n,\{\beta \in {[-M,M]}^{K_n}:\left|\right|\beta -{\beta }_0|{|}_{*}\le 3j{\epsilon }_n\},\left|\right|·\left|{|}_{*}\right),\end{array}$$ which, by Lemma 4.1 in Pollard (1990), is less than $b_2^{K_n}=e^{K_n\hspace{0.17em}\text{log}\hspace{0.17em}{b}_2}$ for some constant $b_2>0,$ and therefore condition (i) of Corollary 1 holds for any given $c_1>0.$

Appendix

Proof of Proposition 1.

Our proof is mainly based on some elementary inequalities such as Jensen’ inequality and Hölder’s inequality. It is no restriction to assume that $n=2k$ is an even number. Take non empty disjoint subsets $B_j,j=1,2,\dots ,N,$ of Θ such that ${\sum }_{j=1}^N\Pi {(B_j)}^{\alpha }\le 2 C(\delta \epsilon ,\{\theta \in {\Theta }_1: d(\theta ,{\theta }_0)>\epsilon \},\alpha ,d),{\cup }_{j=1}^N{B}_j=\{\theta \in {\Theta }_1: d(\theta ,{\theta }_0)>\epsilon \}$ and d-diameters of all B_j do not exceed $2 \delta \epsilon .$ Then by the inequality ${(x+y)}^{\alpha }\le x^{\alpha }+y^{\alpha }$ for all $x,y\ge 0,$ we get $$\begin{array}{c}{P}_{{\theta }_0}^{(n)} {({\int }_{\{\theta \in {\Theta }_1:\mathrm{ }d(\theta ,{\theta }_0)>\epsilon \}}\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^{2k}\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta ))}^{\alpha }\\ \le P_{{\theta }_0}^{(n)} \sum _{j=1}^N({\int }_{B_j}\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^{2k}\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta ){)}^{\alpha }\\ =\sum _{j=1}^N\Pi {(B_j)}^{\alpha } P_{{\theta }_0}^{(n)}{(\frac{1}{\Pi (B_j)}{\int }_{B_j}\frac{q_{\theta }(X_0)\prod _{i=1}^{2k}{p}_{\theta }(X_i|X_{i-1})}{q_{{\theta }_0}(X_0)\prod _{i=1}^{2k}{p}_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta ))}^{\alpha }\\ \le 2 C(\delta \epsilon ,\{\theta \in {\Theta }_1: d(\theta ,{\theta }_0)>\epsilon \},\alpha ,d)\\ \underset{1\le j\le N}{\max } P_{{\theta }_0}^{(n)}{(\frac{1}{\Pi (B_j)}{\int }_{B_j}\frac{q_{\theta }(X_0)\prod _{i=1}^{2k}{p}_{\theta }(X_i|X_{i-1})}{q_{{\theta }_0}(X_0)\prod _{i=1}^{2k}{p}_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta ))}^{\alpha }.\end{array}$$ we shall use the notations ${\prod }_{i=1}^0{p}_{\theta }(X_i|X_{i-1})=1$ and $$I_{j,s}=\frac{{\int }_{B_j}{q}_{\theta }(X_0)\prod _{i=1}^{s+1}{p}_{\theta }(X_i|X_{i-1}) \Pi (d\theta )}{{\int }_{B_j}{q}_{\theta }(X_0)\prod _{i=1}^s{p}_{\theta }(X_i|X_{i-1}) \Pi (d\theta )}$$ for $s=0,1,\dots ,2k-1.$ For simplicity we also let $I_{j,s}$ stand for the parameter of the corresponding integral means. Then the last maximum is equal to $$\begin{array}{c}\underset{1\le j\le N}{\max } P_{{\theta }_0}^{(n)}{(\frac{{\int }_{B_j}{q}_{\theta }(X_0)\Pi (d\theta )}{q_{{\theta }_0}(X_0)\Pi (B_j)}\prod _{s=0}^{2k-1}\frac{I_{j,s}}{p_{{\theta }_0}(X_{s+1}|X_s)})}^{\alpha }\\ =\underset{1\le j\le N}{\max } P_{{\theta }_0}^{(n)}\left({(\frac{{\int }_{B_j}{q}_{\theta }(X_0)\Pi (d\theta )}{q_{{\theta }_0}(X_0)\Pi (B_j)}\prod _{t=1}^k\frac{I_{j,2t-1}}{p_{{\theta }_0}(X_{2t}|X_{2t-1})})}^{\alpha }{(\prod _{t=0}^{k-1}\frac{I_{j,2t}}{p_{{\theta }_0}(X_{2t+1}|X_{2t})})}^{\alpha }\right),\end{array}$$ which, by Hölder’s inequality, is less than $$\begin{array}{c}\underset{1\le j\le N}{\max } {(P_{{\theta }_0}^{(n)}{(\frac{{\int }_{B_j}{q}_{\theta }(X_0)\Pi (d\theta )}{q_{{\theta }_0}(X_0)\Pi (B_j)}\prod _{t=1}^k\frac{I_{j,2t-1}}{p_{{\theta }_0}(X_{2t}|X_{2t-1})})}^{2\alpha })}^{\frac{1}{2}}\\ \underset{1\le j\le N}{\max } {(P_{{\theta }_0}^{(n)}{(\prod _{t=0}^{k-1}\frac{I_{j,2t}}{p_{{\theta }_0}(X_{2t+1}|X_{2t})})}^{2\alpha })}^{\frac{1}{2}}:=(\underset{1\le j\le N}{\max } A_{j,k})(\underset{1\le j\le N}{\max } B_{j,k}).\end{array}$$ Take ${\theta }_j\in B_j$ for each j. From Jensen’s inequality and the assumption $a_{0}r(X_s)\le p_{\theta }(X_s|X_{s-1})\le a_{1}r(X_s)$ it turns out that $$\begin{array}{c}d{(I_{j,s},{\theta }_j)}^2={\int }_{\mathfrak{X}}{\int }_{\mathfrak{X}}(\sqrt{I_{j,s}}-\sqrt{p_{{\theta }_j}(X_{s+1}|X_s)} {)}^2 d\mu (X_{s+1})d\nu (X_s)\\ \le {\int }_{B_j}{\int }_{\mathfrak{X}}{\int }_{\mathfrak{X}}(\sqrt{p_{\theta }(X_{s+1}|X_s)}-\sqrt{p_{{\theta }_j}(X_{s+1}|X_s)} {)}^2 d\mu (X_{s+1})\\ \frac{q_{\theta }(X_0)\prod _{i=1}^s{p}_{\theta }(X_i|X_{i-1})}{{\int }_{B_j}{q}_{\theta }(X_0)\prod _{i=1}^s{p}_{\theta }(X_i|X_{i-1}) \Pi (d\theta )} d\nu (X_s)\Pi (d\theta )\\ \le \frac{a_1}{a_0}{\int }_{B_j}{\int }_{\mathfrak{X}}{\int }_{\mathfrak{X}}(\sqrt{p_{\theta }(X_{s+1}|X_s)}-\sqrt{p_{{\theta }_j}(X_{s+1}|X_s)} {)}^2 d\mu (X_{s+1})d\nu (X_s)\\ \frac{q_{\theta }(X_0)\prod _{i=1}^{s-1}{p}_{\theta }(X_i|X_{i-1})}{{\int }_{B_j}{q}_{\theta }(X_0)\prod _{i=1}^{s-1}{p}_{\theta }(X_i|X_{i-1}) \Pi (d\theta )} \Pi (d\theta )\le \frac{4a_1{\delta }^2{\epsilon }^2}{a_0}\\ =\frac{a_1}{a_0}{\int }_{B_j}d{(\theta ,{\theta }_j)}^2\frac{q_{\theta }(X_0)\prod _{i=1}^{s-1}{p}_{\theta }(X_i|X_{i-1})}{{\int }_{B_j}{q}_{\theta }(X_0)\prod _{i=1}^{s-1}{p}_{\theta }(X_i|X_{i-1}) \Pi (d\theta )} \Pi (d\theta )\le \frac{4a_1{\delta }^2{\epsilon }^2}{a_0}\end{array}$$ Thus, $d(I_{j,s},{\theta }_j)\le \frac{2\sqrt{a_1}\delta \epsilon }{\sqrt{a_0}}$ and $d(I_{j,s},{\theta }_0)\ge d({\theta }_j,{\theta }_0)-d(I_{j,s},{\theta }_j)\ge (1-\frac{2\sqrt{a_1}\delta }{\sqrt{a_0}})\epsilon .$ Write $$\begin{array}{c}{A}_{j,k}^2=\\ {\int }_{{\mathfrak{X}}^{2k-1}}({\int }_{\mathfrak{X}}({\int }_{\mathfrak{X}}(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}{)}^{2\alpha } d\mu (X_{2k})) p_{{\theta }_0}(X_{2k-1}|X_{2k-2}) d\mu (X_{2k-1}))\\ {(\frac{{\int }_{B_j}{q}_{\theta }(X_0)\Pi (d\theta )}{q_{{\theta }_0}(X_0)\Pi (B_j)}\prod _{t=1}^{k-1}\frac{I_{j,2t-1}}{p_{{\theta }_0}(X_{2t}|X_{2t-1})})}^{2\alpha }\\ q_{{\theta }_0}(X_0)\prod _{s=0}^{2k-3}{p}_{{\theta }_0}(X_{s+1}|X_s) d\mu (X_0)d\mu (X_1)\dots d\mu (X_{2k-2}).\end{array}$$ Take an non negative integer m with $\frac{2\alpha }{1-2\alpha }\le 2^m<\frac{4\alpha }{1-2\alpha }.$ From Hölder’s inequality it turns out that for each j and k, $$\begin{array}{c}{\int }_{\mathfrak{X}}{\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{2\alpha } d\mu (X_{2k})\\ {\int }_{\mathfrak{X}}{\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{\alpha } {\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{\alpha } d\mu (X_{2k})\\ \le {\left({\int }_{\mathfrak{X}}{\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{\alpha ·\frac{2}{2-2\alpha }} d\mu (X_{2k})\right)}^{\frac{2-2\alpha }{2}} {\left({\int }_{\mathfrak{X}}{\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{\alpha ·\frac{1}{\alpha }} d\mu (X_{2k})\right)}^{\alpha }\\ ={\left({\int }_{\mathfrak{X}}{\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{\frac{2\alpha }{2-2\alpha }} d\mu (X_{2k})\right)}^{\frac{2-2\alpha }{2}},\end{array}$$ which, by repeating the above procedure m more times, does not exceed $$\begin{array}{c}{\left({\int }_{\mathfrak{X}}{\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{\frac{\alpha }{2^m(1-2\alpha )+\alpha }} d\mu (X_{2k})\right)}^{\frac{2^m(1-2\alpha )+\alpha }{2^m}}\\ \le {\left({\int }_{\mathfrak{X}}{\left(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}\right)}^{\frac{1}{2}} d\mu (X_{2k})\right)}^{\frac{\alpha }{2^{m-1}}}.\end{array}$$ Thus we get $$\begin{array}{c}{\int }_{\mathfrak{X}}({\int }_{\mathfrak{X}}(\frac{I_{j,2k-1}}{p_{{\theta }_0}(X_{2k}|X_{2k-1})}{)}^{2\alpha } d\mu (X_{2k})) p_{{\theta }_0}(X_{2k-1}|X_{2k-2}) d\mu (X_{2k-1})\\ \le {\int }_{\mathfrak{X}}(1-\frac{1}{2}{\int }_{\mathfrak{X}}(\sqrt{I_{j,2k-1}}-\sqrt{p_{{\theta }_0}(X_{2k}|X_{2k-1})} {)}^2 d\mu (X_{2k}){)}^{\frac{\alpha }{2^{m-1}}}\\ p_{{\theta }_0}(X_{2k-1}|X_{2k-2}) d\mu (X_{2k-1})\\ \le {(1-\frac{1}{2}{\int }_{\mathfrak{X}}{\int }_{\mathfrak{X}}(\sqrt{I_{j,2k-1}}-\sqrt{p_{{\theta }_0}(X_{2k}|X_{2k-1})} )}^2{p}_{{\theta }_0}(X_{2k-1}|X_{2k-2})d\mu (X_{2k})d\mu (X_{2k-1}){)}^{\frac{\alpha }{2^{m-1}}}\\ \le {(1-\frac{a_0}{2}{\int }_{\mathfrak{X}}{\int }_{\mathfrak{X}}(\sqrt{I_{j,2k-1}}-\sqrt{p_{{\theta }_0}(X_{2k}|X_{2k-1})} )}^2 d\mu (X_{2k})d\nu (X_{2k-1}){)}^{\frac{1}{2}-\alpha }\\ ={(1-\frac{a_{0}d{(I_{j,2k-1},{\theta }_0)}^2}{2})}^{\frac{1}{2}-\alpha }\le e^{-(1-2\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2{\epsilon }^2}.\end{array}$$ Hence we have $$\begin{array}{c}{A}_{j,k}^2\le e^{-(1-2\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2{\epsilon }^2}{\int }_{{\mathfrak{X}}^{2k-1}}(\frac{{\int }_{B_j}{q}_{\theta }(X_0)\Pi (d\theta )}{q_{{\theta }_0}(X_0)\Pi (B_j)}\prod _{t=1}^{k-1}\frac{I_{j,2t-1}}{p_{{\theta }_0}(X_{2t}|X_{2t-1})}{)}^{2\alpha }\\ q_{{\theta }_0}(X_0)\prod _{s=0}^{2k-3}{p}_{{\theta }_0}(X_{s+1}|X_s) d\mu (X_0)d\mu (X_1)\dots d\mu (X_{2k-2}).\end{array}$$ Repeating the same argument k − 1 times one can get that $$\begin{array}{c}{A}_{j,k}^2\le e^{-(1-2\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^{2}k{\epsilon }^2}{\int }_{\mathfrak{X}}(\frac{{\int }_{B_j}{q}_{\theta }(X_0)\Pi (d\theta )}{q_{{\theta }_0}(X_0)\Pi (B_j)}{)}^{2\alpha }{q}_{{\theta }_0}(X_0) d\mu (X_0)\\ \le e^{-(1-2\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^{2}k{\epsilon }^2}{({\int }_{\mathfrak{X}}\frac{{\int }_{B_j}{q}_{\theta }(X_0)\Pi (d\theta )}{q_{{\theta }_0}(X_0)\Pi (B_j)}{q}_{{\theta }_0}(X_0) d\mu (X_0))}^{2\alpha }\\ =e^{-(1-2\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^{2}k{\epsilon }^2}.\end{array}$$ Similarly, we have $$B_{j,k}^2\le e^{-(1-2\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^{2}k{\epsilon }^2}.$$ Hence we have proved the required inequality and the proof of Proposition 1 is complete. □

To prove Theorem 1 we need the following inequality.

Lemma 1.

If there exists a constant $a_1\ge 1$ such that ${\int }_A{p}_{{\theta }_0}(y|x)d\mu (y)\le a_1{\int }_{A}d\nu (y)$ for all $x\in \mathfrak{X}$ and $A\in \mathcal{A}$, then the inequality$$\begin{array}{c}{P}_{{\theta }_0}^{(n)}({\int }_{\Theta }\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta )\le e^{-n\mathrm{ }{\epsilon }^2\mathrm{ }(3a_1+4c)}\Pi (W_n^1({\theta }_0,\epsilon )))\\ \le e^{-n\mathrm{ }{\epsilon }^2\mathrm{ }c}\end{array}$$holds for all n, $\epsilon >0$ and c > 0.

Proof of Lemma 1.

Without loss of generality, we may assume that $\Pi (W_n^1({\theta }_0,\epsilon ))>0.$ From Jensen’s inequality and Chebyshev’s inequality it follows that $$\begin{array}{c}{P}_{{\theta }_0}^{(n)}({\int }_{\Theta }\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta )\le e^{-n\mathrm{ }{\epsilon }^2\mathrm{ }(3a_1+4c)}\Pi (W_n^1({\theta }_0,\epsilon )) )\\ \le P_{{\theta }_0}^{(n)}\left(e^{n\mathrm{ }{\epsilon }^2\mathrm{ }(\frac{3a_1}{4}+c)}\le {\left(\frac{1}{\Pi \left(W_n^1({\theta }_0,\epsilon )\right)} {\int }_{W_n^1({\theta }_0,\epsilon )}\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta ) \right)}^{-\frac{1}{4}}\right)\\ \le P_{{\theta }_0}^{(n)}\left(e^{n\mathrm{ }{\epsilon }^2\mathrm{ }(\frac{3a_1}{4}+c)}\le \frac{1}{\Pi \left(W_n^1({\theta }_0,\epsilon )\right)} {\int }_{W_n^1({\theta }_0,\epsilon )}{\left(\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})}\right)}^{-\frac{1}{4}} \Pi (d\theta ) \right)\\ \le \frac{{\int }_{W_n^1({\theta }_0,\epsilon )}{P}_{{\theta }_0}^{(n)}{(\frac{q_{{\theta }_0}(X_0)}{q_{\theta }(X_0)}\prod _{i=1}^n\frac{p_{{\theta }_0}(X_i|X_{i-1})}{p_{\theta }(X_i|X_{i-1})})}^{\frac{1}{4}} \Pi (d\theta )}{e^{n\mathrm{ }{\epsilon }^2\mathrm{ }(\frac{3a_1}{4}+c)}\Pi (W_n^1({\theta }_0,\epsilon ))}.\end{array}$$ So it suffices to prove that $P_{{\theta }_0}^{(n)}{(\frac{q_{{\theta }_0}(X_0)}{q_{\theta }(X_0)}{\prod }_{i=1}^n\frac{p_{{\theta }_0}(X_i|X_{i-1})}{p_{\theta }(X_i|X_{i-1})})}^{\frac{1}{4}}\le e^{\frac{3a_1}{4}n{\epsilon }^2}$ for all $\theta \in W_n^1({\theta }_0,\epsilon ).$ We assume without loss of generality that n is an even number, say $n=2k.$ Write $$\frac{q_{{\theta }_0}(X_0)}{q_{\theta }(X_0)}\prod _{i=1}^n\frac{p_{{\theta }_0}(X_i|X_{i-1})}{p_{\theta }(X_i|X_{i-1})}=\frac{q_{{\theta }_0}(X_0)}{q_{\theta }(X_0)}\prod _{j=1}^k\frac{p_{{\theta }_0}(X_{2j}|X_{2j-1})}{p_{\theta }(X_{2j}|X_{2j-1})}\prod _{j=1}^k\frac{p_{{\theta }_0}(X_{2j-1}|X_{2j-2})}{p_{\theta }(X_{2j-1}|X_{2j-2})}.$$ From Hölder’s inequality it then turns out that $$\begin{array}{c}{P}_{{\theta }_0}^{(n)}{(\frac{q_{{\theta }_0}(X_0)}{q_{\theta }(X_0)}\prod _{i=1}^n\frac{p_{{\theta }_0}(X_i|X_{i-1})}{p_{\theta }(X_i|X_{i-1})})}^{\frac{1}{4}}\\ \le {(P_{{\theta }_0}^{(n)}{(\frac{q_{{\theta }_0}(X_0)}{q_{\theta }(X_0)}\prod _{j=1}^k\frac{p_{{\theta }_0}(X_{2j}|X_{2j-1})}{p_{\theta }(X_{2j}|X_{2j-1})})}^{\frac{1}{2}})}^{\frac{1}{2}} {(P_{{\theta }_0}^{(n)}{(\prod _{j=1}^k\frac{p_{{\theta }_0}(X_{2j-1}|X_{2j-2})}{p_{\theta }(X_{2j-1}|X_{2j-2})})}^{\frac{1}{2}})}^{\frac{1}{2}}\\ :=A_k{B}_k.\end{array}$$ Hence by Fubini’s theorem we get that $A_k^2$ is equal to $$\begin{array}{c}{\int }_{{\mathfrak{X}}^{2k+1}}\frac{q_{{\theta }_0}{(X_0)}^{\frac{3}{2}}}{q_{\theta }{(X_0)}^{\frac{1}{2}}}\prod _{j=1}^k(\frac{p_{{\theta }_0}{(X_{2j}|X_{2j-1})}^{\frac{3}{2}}}{p_{\theta }{(X_{2j}|X_{2j-1})}^{\frac{1}{2}}} p_{{\theta }_0}(X_{2j-1}|X_{2j-2}))\\ d\mu (X_0)d\mu (X_1)\dots d\mu (X_{2k})\\ ={\int }_{{\mathfrak{X}}^{2k-1}}({\int }_{\mathfrak{X}}({\int }_{\mathfrak{X}}\frac{p_{{\theta }_0}{(X_{2k}|X_{2k-1})}^{\frac{3}{2}}}{p_{\theta }{(X_{2k}|X_{2k-1})}^{\frac{1}{2}}} d\mu (X_{2k})) p_{{\theta }_0}(X_{2k-1}|X_{2k-2}) d\mu (X_{2k-1}))\\ \frac{q_{{\theta }_0}{(X_0)}^{\frac{3}{2}}}{q_{\theta }{(X_0)}^{\frac{1}{2}}}\prod _{j=1}^{k-1}\frac{p_{{\theta }_0}{(X_{2j}|X_{2j-1})}^{\frac{3}{2}}}{p_{\theta }{(X_{2j}|X_{2j-1})}^{\frac{1}{2}}} p_{{\theta }_0}(X_{2j-1}|X_{2j-2}) d\mu (X_0)d\mu (X_1)\dots d\mu (X_{2k-2}),\end{array}$$ where by the proof of Lemma 1 in Xing (2011b) we have $$\begin{array}{c}{\int }_{\mathfrak{X}}({\int }_{\mathfrak{X}}\frac{p_{{\theta }_0}{(X_{2k}|X_{2k-1})}^{\frac{3}{2}}}{p_{\theta }{(X_{2k}|X_{2k-1})}^{\frac{1}{2}}} d\mu (X_{2k})) p_{{\theta }_0}(X_{2k-1}|X_{2k-2}) d\mu (X_{2k-1})\\ ={\int }_{\mathfrak{X}}(1+\frac{3}{2}{H}_{*}{(p_{{\theta }_0}(·|X_{2k-1}),p_{\theta }(·|X_{2k-1}))}^2) p_{{\theta }_0}(X_{2k-1}|X_{2k-2}) d\mu (X_{2k-1})\\ =1+{\int }_{\mathfrak{X}}\frac{3}{2}{H}_{*}{(p_{{\theta }_0}(·|X_{2k-1}),p_{\theta }(·|X_{2k-1}))}^2 p_{{\theta }_0}(X_{2k-1}|X_{2k-2}) d\mu (X_{2k-1})\\ \le 1+{\int }_{\mathfrak{X}}\frac{3a_1}{2}{H}_{*}{(p_{{\theta }_0}(·|X_{2k-1}),p_{\theta }(·|X_{2k-1}))}^2 d\nu (X_{2k-1})\\ =1+\frac{3a_1}{2}{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2\le e^{\frac{3a_1}{2}{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2}.\end{array}$$ Thus, we have obtained that $A_k\le e^{\frac{3a_1}{4}{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2}{A}_{k-1}.$ Repeating the same argument k − 1 times and using $a_1\ge 1$ one can get $$\begin{array}{c}{A}_k\le e^{\frac{3a_1}{4}k{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2}{({\int }_{\mathfrak{X}}\frac{q_{{\theta }_0}{(X_0)}^{\frac{3}{2}}}{q_{\theta }{(X_0)}^{\frac{1}{2}}} d\mu (X_0))}^{\frac{1}{2}}\\ =e^{\frac{3a_1}{4}k{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2}{(1+\frac{3}{2}{H}_{*}{(q_{{\theta }_0},q_{\theta })}^2)}^{\frac{1}{2}}\le e^{\frac{3}{4}{H}_{*}{(q_{{\theta }_0},q_{\theta })}^2+\frac{3a_1}{4}k{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2}.\end{array}$$ Similarly, we can get that $B_k\le e^{\frac{3a_1}{4}k{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2}.$ Therefore $A_k{B}_k\le e^{\frac{3}{4}{H}_{*}{(q_{{\theta }_0},q_{\theta })}^2+\frac{3a_1}{4}n{H}_{*}{(p_{{\theta }_0},p_{\theta })}^2}\le e^{\frac{3a_1}{4}n{\epsilon }^2}$ for all $\theta \in W_n^1({\theta }_0,\epsilon ),$ and the proof of Lemma 1 is complete. □

Proof of Theorem 1.

Take a constant c such that $c_2>c>1/c_0.$ Hence $e^{-n\mathrm{ }{\epsilon }_n^2\mathrm{ }c}\le e^{-c{c}_0\hspace{0.17em}\text{log}\hspace{0.17em}n}=1/n^{c{c}_0}$ and hence ${\sum }_{n=1}^{\infty }{e}^{-n\mathrm{ }{\epsilon }_n^2\mathrm{ }c}<\infty .$ By Lemma 1 and the first Borel-Cantelli lemma, we get that for almost all $\{X_0,X_1,\dots ,X_n\}$ the inequality $${\int }_{\Theta }\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta )\ge e^{-n\mathrm{ }{\epsilon }_n^2\mathrm{ }(3a_1+4c)}\Pi (W_n^1({\theta }_0,{\epsilon }_n))$$ holds for all large n. But $$\begin{array}{c}{P}_{{\theta }_0}^{(n)}(\Pi (\theta \in \Theta : d(\theta ,{\theta }_0)\ge r {\epsilon }_n|X_0,X_1,\dots ,X_n)\ge e^{-bn{\epsilon }_n^2})\\ \le P_{{\theta }_0}^{(n)}(e^{bn{\epsilon }_n^2}\Pi (\theta \in \Theta \setminus {\Theta }_n: d(\theta ,{\theta }_0)\ge r {\epsilon }_n|X_0,X_1,\dots ,X_n)\ge \frac{1}{2})\\ +P_{{\theta }_0}^{(n)}(e^{bn{\epsilon }_n^2}\Pi (\theta \in {\Theta }_n: d(\theta ,{\theta }_0)\ge r {\epsilon }_n|X_0,X_1,\dots ,X_n)\ge \frac{1}{2}):=a_n+b_n.\end{array}$$ Hence, for large n we have $$\begin{array}{c}{a}_n\le 2e^{bn{\epsilon }_n^2} P_{{\theta }_0}^{(n)}(\Pi (\theta \in \Theta \setminus {\Theta }_n: d(\theta ,{\theta }_0)\ge r {\epsilon }_n|X_0,X_1,\dots ,X_n))\\ \le \frac{2e^{n{\epsilon }_n^2(b+3a_1+4c)}}{\Pi (W_n^1({\theta }_0,{\epsilon }_n))}{P}_{{\theta }_0}^{(n)}\left({\int }_{\{\theta \in \Theta \setminus {\Theta }_n:\mathrm{ }d(\theta ,{\theta }_0)\ge r{\epsilon }_n\}}\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta )\right)\\ \le \frac{2e^{n{\epsilon }_n^2(b+3a_1+4c)} \Pi (\Theta \setminus {\Theta }_n)}{\Pi (W_n^1({\theta }_0,{\epsilon }_n))},\end{array}$$ which, by the assumption of Theorem 1, implies that ${\sum }_{n=1}^{\infty }{a}_n<\infty$ if the constant b is small enough.

On the other hand, let $D_j=\{\theta \in {\Theta }_n:j{\epsilon }_n<d(\theta ,{\theta }_0)\le 2j{\epsilon }_n\}$ and let $[r]$ be the largest integer less than or equal to the constant r. Then, by the inequality ${(x+y)}^{\alpha }\le x^{\alpha }+y^{\alpha }$ for all $x,y\ge 0$ and $0<\alpha <1,$ we get $$\begin{array}{c}{b}_n=P_{{\theta }_0}^{(n)}(2^{\alpha }{e}^{\alpha bn{\epsilon }_n^2} \Pi {(\theta \in {\Theta }_n: d(\theta ,{\theta }_0)\ge r {\epsilon }_n|X_0,X_1,\dots ,X_n)}^{\alpha }\ge 1)\\ \le 2^{\alpha }{e}^{\alpha bn{\epsilon }_n^2} P_{{\theta }_0}^{(n)}(\Pi {(\theta \in {\Theta }_n: d(\theta ,{\theta }_0)\ge r {\epsilon }_n|X_0,X_1,\dots ,X_n)}^{\alpha })\\ \le 2^{\alpha }{e}^{\alpha bn{\epsilon }_n^2} \sum _{j=\left[r\right]}^{\infty }{P}_{{\theta }_0}^{(n)}(\Pi {(D_j|X_0,X_1,\dots ,X_n)}^{\alpha })\\ \le \frac{2^{\alpha }{e}^{\alpha n{\epsilon }_n^2(b+3a_1+4c)}}{\Pi {(W_n^1({\theta }_0,{\epsilon }_n))}^{\alpha }}\sum _{j=\left[r\right]}^{\infty }{P}_{{\theta }_0}^{(n)}{\left({\int }_{D_j}\frac{q_{\theta }(X_0)}{q_{{\theta }_0}(X_0)}\prod _{i=1}^n\frac{p_{\theta }(X_i|X_{i-1})}{p_{{\theta }_0}(X_i|X_{i-1})} \Pi (d\theta )\right)}^{\alpha }\end{array}$$ which, by Proposition 1 and the inequality assumption of Theorem 1, does not exceed $$\begin{array}{c}\frac{2^{\alpha }{e}^{\alpha n{\epsilon }_n^2(b+3a_1+4c)}}{\Pi {(W_n^1({\theta }_0,{\epsilon }_n))}^{\alpha }}\sum _{j=\left[r\right]}^{\infty }2 e^{-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^{2}n{(j{\epsilon }_n)}^2} e^{c_1{j}^{2}n{\epsilon }_n^2} \Pi {(W_n^1({\theta }_0,{\epsilon }_n))}^{\alpha }\\ =2^{1+\alpha }{e}^{\alpha n{\epsilon }_n^2(b+3a_1+4c)}\sum _{j=\left[r\right]}^{\infty }{e}^{\left(c_1-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2\right)n{\epsilon }_n^2{j}^2}\\ \le 2^{1+\alpha }{e}^{\alpha n{\epsilon }_n^2(b+3a_1+4c)}\sum _{j=\left[r\right]}^{\infty }{e}^{\left(c_1-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2\right)n{\epsilon }_n^{2}j}\\ =\frac{2^{1+\alpha }{e}^{\alpha n{\epsilon }_n^2(b+3a_1+4c)+\left(c_1-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2\right)n{\epsilon }_n^2\left[r\right]}}{1-e^{\left(c_1-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2\right)n{\epsilon }_n^2}}\\ \le \frac{2^{1+\alpha }{n}^{c_0\alpha (b+3a_1+4c)+c_0\left(c_1-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2\right)\left[r\right]}}{1-n^{\left(c_1-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2\right)c_0}}\\ \le 2^{2+\alpha }{n}^{c_0\alpha (b+3a_1+4c)+c_0\left(c_1-(\frac{1}{2}-\alpha ){(\frac{\sqrt{a_0}}{2}-\sqrt{a_1}\delta )}^2\right)\left[r\right]},\end{array}$$ where the next last inequality holds for all large r and the last inequality holds for all large n. Since the last exponent of n is strictly less than −1 if r is large enough, we obtain that ${\sum }_{n=1}^{\infty }{b}_n<\infty$ if the constant r is large enough. Hence, by the first Borel-Cantelli lemma we obtain that for almost all $X_0,X_1,\dots ,X_n,$ $$\Pi (\theta \in \Theta : d(\theta ,{\theta }_0)\ge r {\epsilon }_n| X_0,X_1,\dots ,X_n)\le e^{-bn{\epsilon }_n^2}$$ if n is large enough. The proof of Theorem 1 is completed. □