Exponential inequalities for self-normalized martingales

Abstract

In this paper, we establish several exponential inequalities for martingales and self-normalized martingales, which improve some known results.

1 Introduction

A prototypical example of self-normalized random variables is Student’s t-statistic which replaces the population standard deviation σ in the standardized sample mean $\sqrt{n}({\overline{X}}_n-\mu )/\sigma$ by the sample standard deviation. Generally, a self-normalized process is of the form $A_t/B_t$, in which $B_t$ is a random variable that estimates some dispersion measure of $A_t$. If $A_t$ is normalized by nonrandom $b_t$ instead, the moment conditions are needed. However, for self-normalized process, the moment conditions can be deleted, for example, Shao [1] obtained the large deviation of self-normalized sums of i.i.d. random variables without moment conditions. In addition, there has been increasing interest in limit theorems and moment bounds for self-normalized sums of i.i.d. zero-mean random variables $X_i$. Bentkus and Götze [2] gave a Berry-Esseen bound for Student’s t-statistic. Giné et al. [3] proved that the t-statistic has a limiting standard normal distribution if and only if $X_1$ is in the domain of attraction of a normal law. We refer to De la Peña et al. [4] for the comprehensive review of the state of the art of the theory and it applications in statistical inference.

In this paper, we concentrate on the exponential inequalities of the self-normalized martingale. Let $(M_n)$ be a locally square integrable real martingale adapted to a filtration $\mathbb{F}=({\mathcal{F}}_n)$ with $M_0=0$. The predictable quadratic variation and the total quadratic variation of $(M_n)$ are, respectively, given by

$${〈M〉}_n=\sum _{k=1}^n\mathbb{E}[\mathrm{\Delta }{(M_k)}^2|{\mathcal{F}}_{k-1}]$$

and

$${[M]}_n=\sum _{k=1}^n\mathrm{\Delta }{(M_k)}^2,$$

where $\mathrm{\Delta }{M}_n=M_n-M_{n-1}$. In [5], Bercu and Touati established the following results without any assumptions on $(M_n)$.

Theorem 1.1 [[5], Theorem 2.1]

Let $(M_n)$ be a locally square integrable martingale. Then, for all $x,y>0$,

$$\mathbb{P}(|M_n|\ge x,{[M]}_n+{〈M〉}_n\le y)\le 2exp(-\frac{x^2}{2y}).$$

(1)

Theorem 1.2 [[5], Theorem 2.2]

Let $(M_n)$ be a locally square integrable martingale. Then, for all $x,y>0$, $a>0$ and $b>0$,

$$\mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,{〈M〉}_n\ge {[M]}_n+y)\le 2exp(-x^2(ab+\frac{b^{2}y}{2})).$$

(2)

Moreover, we also have

$$\begin{array}{c}\mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,{[M]}_n\le y{〈M〉}_n)\hfill \\ \le 2\underset{p>1}{inf}{\left(\mathbb{E}[exp(-(p-1)\frac{x^2}{(1+y)}(ab+\frac{b^2}{2}{〈M〉}_n))]\right)}^{\frac{1}{p}}.\hfill \end{array}$$

It is necessary to point out that, to calculate the above exponential bounds, the following canonical assumption is important. For a pair of random variables $(A,B)$ with $B>0$, if the following inequality holds:

$$\mathbb{E}[exp(\lambda A-\frac{{\lambda }^2{B}^2}{2})]\le 1,\lambda \in \mathbb{R},$$

then $(A,B)$ are called satisfying canonical assumption. For such a pair $(A,B)$, De la Pen̂a and Pang [6] proved the following exponential bounds (in their inequalities, there are some misprints, so we state a correction as follows).

Theorem 1.3 [[6], Theorem 2.1]

Let $(A,B)$ be a pair of random variables with $B>0$ in the probability space $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ satisfying the canonical assumption. Suppose $\mathbb{E}{|A|}^p<\mathrm{\infty }$ for some $p>1$. Then, for any $x>0$ and for $q\ge 1$ such that $\frac{1}{p}+\frac{1}{q}=1$,

$$\mathbb{P}(\frac{|A|}{\sqrt{\frac{2q-1}{q}(B^2+{[\mathbb{E}{|A|}^p]}^{\frac{2}{p}})}}\ge x)\le {\left(\frac{q}{2q-1}\right)}^{\frac{q}{2(2q-1)}}{x}^{-\frac{q}{2q-1}}{e}^{-\frac{x^2}{2}}.$$

In particular, if $\mathbb{E}{A}^2<\mathrm{\infty }$,

$$\mathbb{P}(\frac{|A|}{\sqrt{\frac{3}{2}(B^2+\mathbb{E}[{|A|}^2])}}\ge x)\le {\left(\frac{2}{3}\right)}^{\frac{1}{3}}{x}^{-\frac{2}{3}}{e}^{-\frac{x^2}{2}},\mathit{\text{for }}x>0,$$

and if $\mathbb{E}|A|<\mathrm{\infty }$,

$$\mathbb{P}(\frac{|A|}{\sqrt{2(B^2+{(\mathbb{E}|A|)}^2)}}\ge x)\le 2^{-\frac{1}{4}}{x}^{-\frac{1}{2}}{e}^{-\frac{x^2}{2}},\mathit{\text{for }}x>0.$$

Moreover, if B satisfies $\mathbb{E}[B^2]=\mathbb{E}[A^2]<\mathrm{\infty }$, the upper bound becomes

$$min[2^{\frac{1}{3}}{e}^{-\frac{x^2}{2}},{(2/3)}^{\frac{1}{3}}{x}^{-\frac{2}{3}}{e}^{-\frac{x^2}{2}}],\mathit{\text{for }}x>0.$$

Here we want to mention the works in Bercu [7], which provided some other different inequality forms.

The purpose of this paper is to establish several exponential inequalities motivated by the above works. In Section 2, we shall propose exponential inequalities for martingales and self-normalized martingales, and, in the last section, the deviation bound of the least-square estimate of the unknown parameter in a linear regressive model is established.

2 Exponential inequalities

2.1 Exponential inequalities for martingales

In this subsection we will give some exponential inequalities involving ${〈M〉}_n$ and ${[M]}_n$.

Theorem 2.1 Let $(M_n)$ be a locally square integrable martingale. Then, for all $x,y>0$,

$$\mathbb{P}(|M_n|\ge x,\frac{1}{2}{[M]}_n+{〈M〉}_n\le y)\le 2exp(-\frac{3x^2}{4y}).$$

(3)

Remark 2.1 There is no assumption on $(M_n)$ in the above result. Since, for any $x,y>0$,

$$\begin{array}{c}\mathbb{P}(|M|\ge x,{[M]}_n+{〈M〉}_n\le y)\hfill \\ \le \mathbb{P}(|M_n|\ge x,\frac{1}{2}{[M]}_n+{〈M〉}_n\le y)\hfill \\ \le 2exp(-\frac{3x^2}{4y})\le 2exp(-\frac{x^2}{2y}),\hfill \end{array}$$

the inequality (3) is more precise than the inequality (1). Therefore, Theorem 2.1 improves the works of Bercu and Touati [[5], Theorem 2.1].

For self-normalized martingales, we can obtain the following inequality which improves the well-known Theorem 1.1.

Theorem 2.2 Let $(M_n)$ be a locally square integrable martingale. Then, for all $x,y>0$, $a\ge 0$, $b\ge 0$,

$$\mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,{〈M〉}_n\ge \frac{1}{4}{[M]}_n+y)\le 2exp(-x^2(ab+\frac{2b^2}{3}y)).$$

(4)

Moreover, we also have the result

$$\begin{array}{c}\mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,\frac{1}{2}{[M]}_n\le y{〈M〉}_n)\hfill \\ \le 2\underset{p>1}{inf}{\left(\mathbb{E}[exp(-(p-1)\frac{x^2}{(1+y)}(ab+\frac{2b^2}{3}{〈M〉}_n))]\right)}^{\frac{1}{p}}.\hfill \end{array}$$

(5)

Remark 2.2 Since, for any $x,y>0$, $a\ge 0$, and $b>0$,

$$\begin{array}{c}\mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,{〈M〉}_n\ge {[M]}_n+y)\hfill \\ \le \mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,{〈M〉}_n\ge \frac{1}{4}{[M]}_n+y)\hfill \\ \le 2exp(-x^2(ab+\frac{2b^2}{3}y))\hfill \\ \le 2exp(-x^2(ab+\frac{b^{2}y}{2})),\hfill \end{array}$$

the inequality (4) is better than the inequality (2). Similarly, we have

$$\begin{array}{c}\mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,{[M]}_n\le y{〈M〉}_n)\hfill \\ \le \mathbb{P}(\frac{|M_n|}{a+b{〈M〉}_n}\ge x,\frac{1}{2}{[M]}_n\le y{〈M〉}_n)\hfill \\ \le 2\underset{p>1}{inf}{\left(\mathbb{E}[exp(-(p-1)\frac{x^2}{(1+y)}(ab+\frac{2b^2}{3}{〈M〉}_n))]\right)}^{\frac{1}{p}}\hfill \\ \le 2\underset{p>1}{inf}{\left(\mathbb{E}[exp(-(p-1)\frac{x^2}{(1+y)}(ab+\frac{b^2}{2}{〈M〉}_n))]\right)}^{\frac{1}{p}}.\hfill \end{array}$$

Hence, Theorem 2.2 improves Theorem 1.2.

From Lemma 2.2 in Section 2, we know that ${〈M〉}_n$ and ${[M]}_n$ satisfy the canonical assumption, i.e.

$$\mathbb{E}\{exp(t{M}_n-\frac{t^2}{2}[\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3}])\}\le 1.$$

Therefore, we can obtain the following theorem.

Theorem 2.3 Let $(X_k,k\ge 1)$ be a martingale different sequence with respect to the filtration $\mathbb{F}=\{{\mathcal{F}}_k:k\ge 1\}$ and suppose that $\mathbb{E}{X}_k^2<\mathrm{\infty }$ for all $k\ge 1$. Then, for $\lambda \in \mathbb{R}$,

$$\mathbb{E}\{exp(t{M}_n-\frac{t^2}{2}[\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3}])\}\le 1,$$

and for $x>0$,

$$\mathbb{P}(\frac{|M_n|}{\sqrt{\frac{3}{2}(\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3}+\mathbb{E}[M_n^2])}}\ge x)\le {\left(\frac{2}{3}\right)}^{-1/3}{x}^{-2/3}{e}^{-\frac{x^2}{2}},$$

(6)

where $M_n={\sum }_{k=1}^n{X}_k$, ${[M]}_n={\sum }_{k=1}^n{X}_k^2$ and ${〈M〉}_n={\sum }_{k=1}^n\mathbb{E}(X_k^2|{\mathcal{F}}_{k-1})$.

Remark 2.3 By Fatou’s lemma, it is easy to see that the above results still hold for any stopping time T with respect the filtration with $T<\mathrm{\infty }$ a.s.

In [[6], Theorem 3.1], De la Peña and Pang obtained the following inequality: Let T be any stopping time with respect the filtration and assume $T<\mathrm{\infty }$ almost surely. Then, for all $x>0$,

$$\mathbb{P}(\frac{|M_T|}{\sqrt{\frac{3}{2}({[M]}_T+{〈M〉}_T+\mathbb{E}[M_T^2])}}\ge x)\le {\left(\frac{2}{3}\right)}^{\frac{1}{3}}{x}^{-\frac{2}{3}}{e}^{-\frac{x^2}{2}}.$$

(7)

By comparing the inequalities (6) and (7), we know that the inequality (6) is better than the inequality (7). More precisely, we have

$$\begin{array}{c}\mathbb{P}(\frac{|M_T|}{\sqrt{\frac{3}{2}({[M]}_T+{〈M〉}_T+\mathbb{E}[M_T^2])}}\ge x)\hfill \\ \le \mathbb{P}(\frac{|M_T|}{\sqrt{\frac{3}{2}(\frac{{[M]}_T}{3}+\frac{2{〈M〉}_T}{3}+\mathbb{E}[M_T^2])}}\ge x)\le {\left(\frac{2}{3}\right)}^{\frac{1}{3}}{x}^{-\frac{2}{3}}{e}^{-\frac{x^2}{2}}.\hfill \end{array}$$

The following result may be of independent interest.

Theorem 2.4 Let $(A,B)$ be a pair of random variables with $B>0$ in the probability space $(\mathrm{\Omega },\mathcal{F},P)$ satisfying the canonical assumption. For every $b>0$, $S\ge 1$ and $\lambda \ge 1$,

$$\mathbb{P}(|A|>\lambda \sqrt{B},b\le \sqrt{B}\le bS)\le 2\sqrt{e}(1+2\lambda logS)e^{-{\lambda }^2/2}.$$

From Theorem 2.4, we have the following.

Corollary 2.1 Let $(M_n)$ be a locally square integrable martingale. Then, for every $b>0$, $S\ge 1$ and $\lambda \ge 1$, we have

$$\begin{array}{c}\mathbb{P}(|M_n|>\lambda \sqrt{\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3}},b\le \sqrt{\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3}}\le bS)\hfill \\ \le 2\sqrt{e}(1+2\lambda logS)e^{-{\lambda }^2/2}.\hfill \end{array}$$

2.2 Proofs of main results

We start with the following basic lemma.

Lemma 2.1 Let X be a square integrable random variable with $\mathbb{E}X=0$ and $0<{\sigma }^2:=\mathbb{E}{X}^2<\mathrm{\infty }$. Then, for all $t\in \mathbb{R}$, we have

$$\mathbb{E}[exp(tX-\frac{t^2}{6}{X}^2)]\le 1+\frac{t^2}{3}{\sigma }^2.$$

(8)

Proof First, we shall prove

$$exp(x-\frac{x^2}{6})\le 1+x+\frac{x^2}{3}.$$

(9)

Let

$$G(x)=exp(x-\frac{x^2}{6})-1-x-\frac{x^2}{3},$$

then, by a straightforward calculation, we have

$$G^{\prime }(x)=(1-\frac{x}{3})exp(x-\frac{x^2}{6})-1-\frac{2}{3}x$$

and

$$\begin{array}{rl}{G}^{″}(x)& =-\frac{1}{3}exp(x-\frac{x^2}{6})+{(1-\frac{x}{3})}^{2}exp(x-\frac{x^2}{6})-\frac{2}{3}\\ =-\frac{2}{3}[1-(1-x+\frac{1}{6}{x}^2)exp(x-\frac{x^2}{6})].\end{array}$$

Since

$$exp(\frac{x^2}{6}-x)>1+\frac{x^2}{6}-x,$$

it follows that

$$exp(x-\frac{x^2}{6})\cdot (1-x+\frac{x^2}{6})<1.$$

So we get $G^{″}(x)<0$, which means that $G(x)$ is concave function. Because $G(0)=0$ and $G^{\prime }(0)=0$, we can prove $G(x)\le G(0)$, i.e. $exp(x-\frac{x^2}{6})\le 1+x+\frac{x^2}{3}$. Finally we prove Lemma 2.1 by (9). □

In order to prove our main results, we firstly introduce Lemma 2.2.

Lemma 2.2 Let $(M_n)$ be a locally square integrable martingale. For all $t\in \mathbb{R}$ and $n\ge 0$, denote

$$V_n(t)=exp(t{M}_n-\frac{t^2}{6}{[M]}_n-\frac{t^2}{3}{〈M〉}_n).$$

(10)

Then, for all $t\in \mathbb{R}$, $(V_n(t))$ is a positive supermartingale with $\mathbb{E}[V_n(t)]\le 1$.

Proof For all $t\in \mathbb{R}$ and $n\ge 0$, we have

$$V_n(t)=V_n(t-1)exp(t{M}_n-\frac{t^2}{6}{[M]}_n-\frac{t^2}{3}{〈M〉}_n),$$

where $\mathrm{\Delta }{M}_n=M_n-M_{n-1}$, $\mathrm{\Delta }{[M]}_n=\mathrm{\Delta }{M}^2$ and $\mathrm{\Delta }{〈M〉}_n=\mathbb{E}[\mathrm{\Delta }{M}^2|{\mathcal{F}}_{n-1}]$. Hence, we deduce from Lemma 2.1 that, for all $t\in \mathbb{R}$,

$$\begin{array}{rl}\mathbb{E}[V_n(t)|{\mathcal{F}}_{n-1}]& \le V_n(t-1)exp(-\frac{t^2}{3}\mathrm{\Delta }{〈M〉}_n)\cdot (1+\frac{t^2}{3}\mathrm{\Delta }{〈M〉}_n)\\ =V_n(t-1).\end{array}$$

As a result, for all $t\in \mathbb{R}$, $(V_n(t))$ is a positive supermartingale, i.e. for all $n\ge 1$, $\mathbb{E}[V_n(t)]\le \mathbb{E}[V_{n-1}(t)]$, which implies that $\mathbb{E}[V_n(t)]\le \mathbb{E}[V_0(t)]=1$. □

Next, we start to prove Theorem 2.1 and Theorem 2.2 inspired by the original article of Bercu and Touati [5].

Proof of Theorem 2.1 First of all, according to the condition of Lemma 2.2, we denote

$$Z_n=\frac{1}{2}{[M]}_n+{〈M〉}_n.$$

(11)

For all $x,y\ge 0$ let $A_n=\{|M_n|\ge x,Z_n\le y\}$. We define $A_n=A_n^{+}\cup A_n^{-}$ where $A_n^{+}=\{M_n\ge x,Z_n\le y\}$ and $A_n^{-}=\{M_n\le -x,Z_n\le y\}$. By Markov’s inequality, we have, for all $t\ge 0$,

$$\begin{array}{rl}\mathbb{P}\left(A_n^{+}\right)& \le \mathbb{E}[exp(\frac{t}{2}{M}_n-\frac{t}{2}x)I_{A_n^{+}}]\\ \le \mathbb{E}[exp(\frac{t}{2}{M}_n-\frac{t^2}{6}{Z}_n)\cdot exp(\frac{t^2}{6}{Z}_n-\frac{t}{2}x)I_{A_n^{+}}]\\ \le exp(\frac{t^2}{6}y-\frac{t}{2}x)\sqrt{\mathbb{E}[V_n(t)\mathbb{P}\left(A_n^{+}\right)]}.\end{array}$$

Hence, we deduce from Lemma 2.2 that, for all $t>0$,

$$\mathbb{P}\left(A_n^{+}\right)\le exp(\frac{t^2}{6}y-\frac{t}{2}x)\sqrt{\mathbb{P}\left(A_n^{+}\right)}.$$

(12)

Dividing both sides of (12) by $\sqrt{\mathbb{P}(A_n^{+})}$ and choosing the value $t=\frac{3x}{2y}$, we find that

$$\mathbb{P}\left(A_n^{+}\right)\le exp(-\frac{3x^2}{2y}).$$

We also have the same upper bound for $\mathbb{P}(A_n^{-})$, immediately leading to the result of (3). □

Proof of Theorem 2.2 We are going to list the proof of Theorem 2.2 in the special case $a=0$ and $b=1$. For all $x,y>0$, let

$$B_n=\{|M_n|\ge x{〈M〉}_n,{〈M〉}_n-\frac{1}{4}{[M]}_n\ge y\}=B_n^{+}\cup B_n^{-},$$

where

$$\begin{array}{c}{B}_n^{+}=\{M_n\ge x{〈M〉}_n,{〈M〉}_n-\frac{1}{4}{[M]}_n\ge y\},\hfill \\ B_n^{-}=\{M_n\le -x{〈M〉}_n,{〈M〉}_n-\frac{1}{4}{[M]}_n\ge y\}.\hfill \end{array}$$

By Cauchy-Schwarz’s inequality, we have, for all $t>0$,

$$\begin{array}{rl}\mathbb{P}\left(B_n^{+}\right)& \le \mathbb{E}[exp(\frac{t}{2}{M}_n-\frac{tx}{2}{〈M〉}_n)I_{B_n^{+}}]\\ \le \mathbb{E}[exp(\frac{t}{2}{M}_n-\frac{t^2}{6}{Z}_n)exp(\frac{t}{6}(t-3x){〈M〉}_n+\frac{t^2}{12}{[M]}_n)I_{B_n^{+}}]\\ \le \mathbb{E}{[exp(\frac{t}{3}(t-3x){〈M〉}_n+\frac{t^2}{6}{[M]}_n)I_{B_n^{+}}]}^{1/2},\end{array}$$

(13)

where $Z_n$ is defined in (11). Consequently, we obtain from (13) with the particular choice $t=x$ that

$$\mathbb{P}\left(B_n^{+}\right)\le exp(-\frac{x^{2}y}{3})\sqrt{\mathbb{P}\left(B_n^{+}\right)}.$$

(14)

Therefore, if we divide both sides of (14) by $\sqrt{\mathbb{P}(B_n^{+})}$, we find that

$$\mathbb{P}\left(B_n^{+}\right)\le exp(-\frac{2x^{2}y}{3}).$$

The same upper bound holds for $\mathbb{P}(B_n^{-})$, which clearly implies (4). Furthermore, for all $x,y>0$, let

$$C_n=\{|M_n|\ge x{〈M〉}_n,\frac{1}{2}{[M]}_n\le y{〈M〉}_n\}=C_n^{+}\cup C_n^{-},$$

where

$$\begin{array}{c}{C}_n^{+}=\{M_n\ge x{〈M〉}_n,\frac{1}{2}{[M]}_n\le y{〈M〉}_n\},\hfill \\ C_n^{-}=\{M_n\le -x{〈M〉}_n,\frac{1}{2}{[M]}_n\le y{〈M〉}_n\}.\hfill \end{array}$$

By Hölder’s inequality, we have, for all $t>0$ and $q>1$,

$$\begin{array}{rl}\mathbb{P}\left(C_n^{+}\right)& \le \mathbb{E}[exp(\frac{t}{q}{M}_n-\frac{tx}{q}{〈M〉}_n)I_{C_n^{+}}]\\ \le \mathbb{E}[exp(\frac{t}{q}{M}_n-\frac{t^2}{3q}{Z}_n)exp(\frac{t}{3q}(t-3x+ty){〈M〉}_n)I_{C_n^{+}}]\\ \le {\left(\mathbb{E}[exp(\frac{tp}{3q}(t-3x+ty){〈M〉}_n)]\right)}^{\frac{1}{p}}.\end{array}$$

(15)

Consequently, as $\frac{p}{q}=p-1$, we can deduce from (15) and the particular choice $t=\frac{x}{1+y}$ that

$$\begin{array}{rl}\mathbb{P}\left(C_n^{+}\right)& \le \underset{p>1}{inf}{\left(\mathbb{E}[exp\left(\frac{p-1}{3}(\frac{x^2}{{(1+y)}^2}-\frac{3x^2}{(1+y)}+\frac{x^{2}y}{{(1+y)}^2}){〈M〉}_n\right)]\right)}^{\frac{1}{p}}\\ \le \underset{p>1}{inf}{\left(\mathbb{E}[exp(-(p-1)\cdot \frac{2x^2}{3(1+y)}{〈M〉}_n)]\right)}^{1/p}.\end{array}$$

We also find the same upper bound for $\mathbb{P}(C_n^{-})$, which completes the proof of Theorem 2.2. □

Proof of Theorem 2.3 Because $(X_k,k\ge 1)$ is a martingale different sequence, it satisfies the canonical assumption, i.e. $\mathbb{E}\{exp(t{M}_n-\frac{t^2}{2}[\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3}])\}\le 1$. Putting

$$A=M_n,B^2=\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3},$$

then, according to the canonical assumption and Fubini’s theorem, for any $C>0$, we have

$$\begin{array}{rl}1& \ge {\int }_R\frac{1}{\sqrt{2\pi }{C}^{-\frac{1}{2}}}{e}^{-\frac{{\lambda }^2}{2C^{-1}}}\mathbb{E}[exp(\lambda A-\frac{{\lambda }^2{B}^2}{2})]d\lambda \\ =\mathbb{E}[C^{\frac{1}{2}}exp\left(\frac{A^2}{2(B^2+C)}\right){\int }_R\frac{1}{\sqrt{2\pi }}exp(-\frac{{(\lambda +\frac{A}{B^2+C})}^2}{\frac{2}{B^2+C}})d\lambda ]\\ =\mathbb{E}[{\left(\frac{C}{B^2+C}\right)}^{\frac{1}{2}}exp\left(\frac{A^2}{2(B^2+C)}\right)].\end{array}$$

For any measurable set $G\in \mathbb{F}$, by Markov’s inequality, we get

$$\begin{array}{c}\mathbb{P}(\frac{|A|}{\sqrt{B^2+C}}\ge x,G)\hfill \\ =\mathbb{P}(\frac{{|A|}^2}{4(B^2+C)}\ge \frac{x^2}{4},G)\hfill \\ \le \mathbb{P}(\frac{{|A|}^{\frac{1}{2}}}{{(B^2+C)}^{\frac{1}{4}}}{e}^{\frac{A^2}{4(B^2+C)}}\ge x^{\frac{1}{2}}{e}^{\frac{x^2}{4}},G)\hfill \\ \le x^{-1/2}{e}^{-\frac{x^2}{4}}\mathbb{E}\left[{\left(\frac{A^2}{B^2+C}\right)}^{\frac{1}{4}}{e}^{\frac{A^2}{4(B^2+C)}}{I}_G\right].\hfill \end{array}$$

(16)

By using Hölder’s inequality, we get

$$\begin{array}{c}\mathbb{E}\left[{\left(\frac{A^2}{B^2+C}\right)}^{\frac{1}{4}}{e}^{\frac{A^2}{4(B^2+C)}}{I}_G\right]\hfill \\ =\mathbb{E}\left[\frac{C^{\frac{1}{4}}}{{|A|}^{\frac{1}{2}}}{\left(\frac{A^2}{B^2+C}\right)}^{\frac{1}{4}}{e}^{\frac{A^2}{4(B^2+C)}}\frac{{|A|}^{\frac{1}{2}}}{C^{\frac{1}{4}}}{I}_G\right]\hfill \\ \le {\left(\mathbb{E}[{\left(\frac{C}{B^2+C}\right)}^{\frac{1}{2}}exp\left(\frac{A^2}{2(B^2+C)}\right)]\right)}^{\frac{1}{2}}{\left(\mathbb{E}\left[\frac{|A|}{C^{\frac{1}{2}}}{I}_G\right]\right)}^{\frac{1}{2}}\hfill \\ \le {\left(\mathbb{E}\left[\frac{|A|}{C^{\frac{1}{2}}}{I}_G\right]\right)}^{\frac{1}{2}}.\hfill \end{array}$$

Let $C=\mathbb{E}[{|A|}^2]$, and by using Hölder’s inequality, we have the following inequality:

$$\mathbb{E}\left[\frac{|A|}{C^{\frac{1}{2}}}{I}_G\right]\le {\left(\mathbb{E}\left[\frac{{|A|}^2}{C}\right]\right)}^{\frac{1}{2}}\mathbb{P}{(G)}^{\frac{1}{2}}=\mathbb{P}{(G)}^{\frac{1}{2}}.$$

Hence, from (16), it follows that

$$\mathbb{P}(\frac{|A|}{\sqrt{B^2+\mathbb{E}{|A|}^2}}\ge x,G)\le x^{-1/2}{e}^{-\frac{x^2}{4}}\mathbb{P}{(G)}^{\frac{1}{4}}.$$

Now letting $G=\{\frac{|A|}{\sqrt{B^2+\mathbb{E}{|A|}^2}}\ge x\}$, we will have

$$\mathbb{P}(\frac{|A|}{\sqrt{B^2+\mathbb{E}{|A|}^2}}\ge x)\le x^{-2/3}{e}^{-\frac{x^2}{3}}.$$

So we can get the inequality

$$\mathbb{P}(\frac{|M_n|}{\sqrt{\frac{3}{2}(\frac{{[M]}_n}{3}+\frac{2{〈M〉}_n}{3}+\mathbb{E}[M_n^2])}}\ge x)\le {\left(\frac{2}{3}\right)}^{-1/3}{x}^{-2/3}{e}^{-\frac{x^2}{2}}.$$

 □

Proof of Theorem 2.4 Given $a>1$, let $b_k=b{a}_k$ and define random events $C_k=\{b_k\le \sqrt{B}\le b_{k+1}\}$, $k=0,1,2,\dots ,K$, where K denotes the integer part of ${log}_{a}S$. Since the pair $(A,B)$ satisfies the canonical assumption, we have

$$\begin{array}{rl}1& \ge \mathbb{E}exp(\frac{\lambda }{b_k}A-\frac{{\lambda }^2}{2b_k^2}B)I_{\{A>\lambda \sqrt{B},C_k\}}\\ \ge \mathbb{E}exp(\frac{{\lambda }^2}{b_k}\sqrt{B}-\frac{{\lambda }^2}{2b_k^2}B)I_{\{A>\lambda \sqrt{B},C_k\}}\\ \ge \mathbb{E}\underset{b_k\le v\le b_{k+1}}{inf}exp(\frac{{\lambda }^2}{b_k}v-\frac{{\lambda }^2}{2b_k^2}{v}^2)I_{\{A>\lambda \sqrt{B},C_k\}}\\ =exp\left({\lambda }^2(a-\frac{a^2}{2})\right)\mathbb{P}\{A>\lambda \sqrt{B},C_k\},\end{array}$$

which implies

$$\mathbb{P}(A>\lambda \sqrt{B},C_k)\le exp(-{\lambda }^2(a-\frac{a^2}{2})).$$

Therefore, we conclude that

$$\begin{array}{c}\mathbb{P}(|A|>\lambda \sqrt{B},b\le \sqrt{B}\le bS)\hfill \\ \le 2\sum _{k=0}^K\mathbb{P}(|A|>\lambda \sqrt{B},b\le \sqrt{B}\le bS,C_k)\hfill \\ \le 2(1+{log}_{a}S)exp(-{\lambda }^2(a-\frac{a^2}{2})).\hfill \end{array}$$

(17)

We need to choose $a>1$ such that the bound in (17) is possibly small. Now, by taking $a=1+1/\lambda$, we have

$${\lambda }^2(a-\frac{a^2}{2})=\frac{1}{2}({\lambda }^2-1).$$

(18)

Since $log(1+1/\lambda )\ge 1/(2\lambda )$ for $\lambda \ge 1$, then we have $(1+{log}_{a}S)\le (1+2\lambda logS)$, which, together with (18), yields the desired result. □

3 Linear regressions

In this section, let us consider the deviation inequality of the least-squares estimate of the unknown parameter in linear regressive model. For all $n\ge 0$, let

$$X_{n+1}=\theta {\varphi }_n+{\epsilon }_{n+1},$$

(19)

where $X_n$, ${\varphi }_n$, and ${\epsilon }_n$ are the observation, the regression variable, and the driven noise, respectively. Suppose that $({\varphi }_n)$ is a sequence of independent and identically distributed random variables, and $({\epsilon }_n)$ is a sequence of identically distributed random variables with mean zero and variance ${\sigma }^2>0$. Furthermore, assume that $({\epsilon }_{n+1})$ is independent of ${\mathcal{F}}_n$ where ${\mathcal{F}}_n=\sigma ({\varphi }_i,{\epsilon }_j;0\le i\le n-1,1\le j\le n)$. The least-squares estimate of the unknown parameter θ is given by

$${\hat{\theta }}_n=\frac{{\sum }_{k=1}^n{\varphi }_{k-1}{X}_k}{{\sum }_{k=1}^n{\varphi }_{k-1}^2},$$

which yields from (19)

$${\hat{\theta }}_n-\theta ={\sigma }^2\frac{M_n}{{〈M〉}_n},$$

(20)

where

$$M_n=\sum _{k=1}^n{\varphi }_{k-1}{\epsilon }_k\text{and}{〈M〉}_n={\sigma }^2\sum _{k=1}^n{\varphi }_{k-1}^2.$$

Let H and L be the cumulant generating functions of the sequence $({\varphi }_n^2)$ and $({\epsilon }_n^2)$, respectively given, for all $t\in \mathbb{R}$, by

$$H(t)=log\mathbb{E}[exp\left(t{\varphi }_n^2\right)]\text{and}L(t)=log\mathbb{E}[exp\left(t{\epsilon }_n^2\right)].$$

Bercu and Touati [5] obtained the following result.

Corollary 3.1 [[5], Corollary 5.1]

Assume that L is finite on some interval $[0,c]$ with $c>0$ and denote by I its Fenchel-Legendre transform on $[0,c]$,

$$I(x)=\underset{0\le t\le c}{sup}\{xt-L(t)\}.$$

Then, for all $n\ge 1$, $x>0$ and $y>0$, we have

$$\mathbb{P}(|{\hat{\theta }}_n-\theta |\ge x)\le 2\underset{p>1}{inf}exp\left(\frac{n}{p}H(-\frac{(p-1)x^2}{2{\sigma }^2(1+y)})\right)+exp(-nI\left(\frac{{\sigma }^{2}y}{n}\right)).$$

(21)

Now, we give the following theorem.

Theorem 3.1 Under the conditions of Corollary  3.1, for all $n\ge 1$, $x>0$, and $y>0$, we have

$$\mathbb{P}(|{\hat{\theta }}_n-\theta |\ge x)\le 2\underset{p>1}{inf}exp\left(\frac{n}{p}H(-\frac{2(p-1)x^2}{3{\sigma }^2(1+y)})\right)+exp(-nI\left(\frac{2{\sigma }^{2}y}{n}\right)).$$

(22)

Remark 3.1 Obviously, the upper bound in (22) is better than the bound (21).

Proof From (20), for all $n\ge 1$, $x>0$ and $y>0$, we get

$$\begin{array}{rl}\mathbb{P}(|{\hat{\theta }}_n-\theta |\ge x)& =\mathbb{P}(|M_n|\ge \frac{x}{{\sigma }^2}{〈M〉}_n)\\ \le \mathbb{P}(|M_n|\ge \frac{x}{{\sigma }^2}{〈M〉}_n,\frac{1}{2}{[M]}_n\le y{〈M〉}_n)+\mathbb{P}(\frac{1}{2}{[M]}_n\ge y{〈M〉}_n).\end{array}$$

By using the inequality (5), it follows that

$$\begin{array}{c}\mathbb{P}(|M_n|\ge \frac{x}{{\sigma }^2}{〈M〉}_n,\frac{1}{2}{[M]}_n\le y{〈M〉}_n)\hfill \\ \le 2\underset{p>1}{inf}{\left(\mathbb{E}[exp(-(p-1)\frac{2x^2}{3{\sigma }^4(1+y)}{〈M〉}_n)]\right)}^{\frac{1}{p}}\hfill \\ =2\underset{p>1}{inf}exp\left(\frac{n}{p}H(-\frac{2(p-1)x^2}{3{\sigma }^2(1+y)})\right).\hfill \end{array}$$

Furthermore, since, for any $t\in [0,c]$,

$$\begin{array}{c}\mathbb{P}(\frac{1}{2}{[M]}_n\ge y{〈M〉}_n)\hfill \\ =\mathbb{P}(\sum _{k=1}^n{\varphi }_{k-1}^2{\epsilon }_k^2\ge 2y{\sigma }^2\sum _{k=1}^n{\varphi }_{k-1}^2)\hfill \\ \le \mathbb{P}(\sum _{k=1}^n{\epsilon }_k^2\ge 2y{\sigma }^2)\le exp(-2y{\sigma }^{2}t)\mathbb{E}exp\left(t\sum _{k=1}^n{\epsilon }_k^2\right)\hfill \\ \le exp(-2y{\sigma }^{2}t+nL(t))\le exp(-nI\left(\frac{2y{\sigma }^2}{n}\right)),\hfill \end{array}$$

then, from the above discussions, the desired results can be obtained. □