Limit theorems for delayed sums of random sequence

Abstract

For a sequence of arbitrarily dependent random variables (X _n )_n∈Nand Borel sets (B _n )_n∈N, on real line the strong limit theorems, represented by inequalities, i.e. the strong deviation theorems of the delayed average $S_{n.k_n}\left(\omega \right)$ are investigated by using the notion of asymptotic delayed log-likelihood ratio. The results obtained popularizes the methods proposed by Liu.

Mathematics Subject Classification 2000: Primary, 60F15.

1. Introduction

Let (a _n )_n∈Nbe a sequence of real numbers and let (k _n )_n∈Nbe a sequence of positive integers. The numbers

$${\rho }_{n,k_n}=\left\{{\displaystyle \sum _{j=1}^{k_n}}{a}_{n+j-1}\right\}/k_n$$

are called the (forward) delayed first arithmetic means (See [1]). In [2], using the limiting behavior of delayed average, Chow found necessary and sufficient conditions for the Borel summability of i.i.d. random variables and also obtained very simple proofs of a number of well-known results such as the Hsu-Robbins-Spitzer-Katz theorem. In [3], Lai studied the analogues of the law of the iterated logarithim for delayed sums of independent random variables. Recently, Chen [4] has presented an accurate description the limiting behavior of delayed sums under a non-identically distribution setup, and has deduced Chover-type laws of the iterated logarithm for them.

Our aim in this article is to establish strong deviation theorems (limit theorem expressed by inequalities, see [5]) of delayed average for the dependent absolutely continuous random variables. By using the notion of asymptotic delayed log-likelihood ratio, we extend the analytic technique proposed by Liu [5] to the case of delayed sums. The crucial part of the proof is to construct a delayed likelihood ratio depending on a parameter, and then applies the Borel-Cantelli lemma.

Throughout, let (X _n )_n∈Nbe a sequence of absolutely continuous random variables on a fixed probability space $\left\{\Omega ,\mathcal{F},P\right\}$ with the joint density function g^{1, n}(x₁,..., x _n ), n ∈ N, and f _j (x), j = 1, 2,... be the the marginal density function of random variable X _j . (k _n )_n∈Nbe a subsequence of positive integers, such that, for every ε > 0, ${\sum }_{n=1}^{\infty }\mathsf{\text{exp}}\left(-k_n\epsilon \right)<\infty$.

Definition 1. The delayed likelihood ratio is defined by

$${\mathcal{L}}_n\left(\omega \right)=\left\{\begin{array}{cc}\frac{{\Pi }_{j=n}^{n+k_n-1}{f}_j\left(X_j\right)}{{g^{n,n+k_n-1}}^{{}_{\left(X_n,...,X_{n+{}_{kn}-1}\right)}}},\hfill & \mathsf{\text{if}}\mathsf{\text{denominator}}>0\hfill \\ 0,\hfill & \mathsf{\text{otherwise}}\hfill \end{array}\right.$$

(1.1)

Let

$$\mathcal{L}\left(\omega \right)=-\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}\frac{1}{k_n}\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)$$

(1.2)

$\mathcal{L}\left(\omega \right)$ is called asymptotic delayed log-likelihood ratio, where $g^{n,n+k_n-1}\left(x_n,\dots ,{x_n}_{+k_n-1}\right)$ denotes the joint density function of random vector $\left(X_n,\dots ,X_{n+k_n-1}\right)$, ω is a sample point (with log 0 = -∞).

It will be shown in Lemma 1 that $\mathcal{L}\left(\omega \right)\ge 0$a.e. in any case.

Remark 1. It will be seen below that $\mathcal{L}\left(\omega \right)$ has the analogous properties of the likelihood ratio in [5], Although $\mathcal{L}\left(\omega \right)$ is not a proper metric among probability measures, we nevertheless consider it as a measure of "discrimination" between the dependence (their joint distribution) and independence (the product of their marginals). Obviously, ${\mathcal{L}}_n\left(\omega \right)=1$, a.e. n∈ N if (X _n )_n∈Nis independent. In view of the above discussion of the asymptotic logarithmic delayed likelihood ratio, it is natural for us to think of $\mathcal{L}\left(\omega \right)$ as a measure how far (the random deviation) of (X _n )_n∈Nis from being independent and how dependent they are. The closer $\mathcal{L}\left(\omega \right)$ approaches to 0, the smaller the deviation is.

Lemma 1. Let${\mathcal{L}}_n\left(\omega \right)$be define as above, then

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)\le 0,a.e.$$

(1.3)

Proof. Let $B=\left\{\left(x_n,\dots ,x_{n+k_n-1}\right):g^{n,n+k_n-1}\left(x_n,\dots ,x_{n+k_n-1}\right)>0\right\}.$ Since

$$\begin{array}{c}E\left[{\mathcal{L}}_n\left(\omega \right)\right]\hfill \\ =\int \cdots \underset{\left(x_n,\dots ,x_{n+k_n-1}\right)\in B}{\int }\frac{{\prod }_{j=n}^{n+k_n-1}{f}_j\left(x_j\right)}{g^{n,n+k_n-1}\left(x_n,\dots ,x_{n+k_n-1}\right)}\hfill \\ .g^{n,n+k_n-1}\left(x_n,\dots ,x_{n+k_n-1}\right)d{x}_n\dots d{x}_{n+k_n-1}\hfill \\ =\int \cdots \underset{\left(x_n,\dots ,x_{n+k_n-1}\right)\in B}{\int }{\displaystyle \prod _{j=n}^{n+k_n-1}}{f}_j\left(x_j\right)d{x}_n\dots d{x}_{n+k_n-1}\hfill \\ \le \int \cdots \underset{\left(x_n,\dots ,x_{n+k_n-1}\right)\in R^{k_n}}{\int }{\displaystyle \prod _{j=n}^{n+k_n-1}}{f}_j\left(x_j\right)d{x}_n\dots d{x}_{n+k_n-1}=1.\hfill \end{array}$$

From Markov inequality, for every ε > 0, we have

$$P\left[\frac{1}{k_n}\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)\ge \epsilon \right]=P\left[{\mathcal{L}}_n\left(\omega \right)\ge \mathsf{\text{exp}}\left(k_n\epsilon \right)\right]\le 1\cdot \mathsf{\text{exp}}\left(-k_n\epsilon \right).$$

Hence

$${\displaystyle \sum _{n=1}^{\infty }}P\left[\frac{1}{k_n}\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)\ge \epsilon \right]\le {\displaystyle \sum _{n=1}^{\infty }}\mathsf{\text{exp}}\left(-k_n\epsilon \right)<\infty .$$

By Borel-Cantelli lemma, we have

$$P\left[\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)\ge 2\epsilon \right]=0,$$

for any ε > 0, (1.3) follows immediately. □

2. Main results and proofs

Theorem 1. Let (X _n )_n∈N, ${\mathcal{L}}_n\left(\omega \right)$, $\mathcal{L}\left(\omega \right)$be defined as above, (B _n )_n∈Nbe a sequence of Borel sets of the real line. Let $S_{n,k_n}\left(\omega \right)=\frac{1}{k_n}{\sum }_{j=n}^{n+k_n-1}{1}_{B_j}\left(X_j\right)$, and assume

$$c=\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}{\displaystyle \sum _{j=n}^{n+k_n-1}}P\left(X_j\in B_j\right),$$

(2.1)

then

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}{S}_{n,k_n}\left(\omega \right)\le {\left(\sqrt{\mathcal{L}\left(\omega \right)}+\sqrt{c}\right)}^2,a.e.$$

(2.2)

where$1_{B_n}\left(\cdot \right)$be the indicator function of B _n .

Proof. Assume s > 0 to be a constant, and let

$$h_j\left(x_j\right)=\frac{{s^{1_{B_j}}}^{\left(x_j\right)}{f}_j\left(x_j\right)}{1+\left(s-1\right){\int }_{B_j}{f}_j\left(x_j\right)d{x}_j},j=1,2,\dots$$

(2.3)

It is not difficult to see that $\int h_j\left(x_j\right)d{x}_j=1$, j = 1, 2,... Let

$${\Lambda }_n\left(s,\omega \right)=\left\{\begin{array}{cc}\frac{{\Pi }_{j=n}^{n+k_n-1}{h}_j\left(X_j\right)}{g^{n,n+k_n-1}\left({X_n}_{,...,}{X}_{n+k_n-1}\right)},\hfill & \mathsf{\text{if}}\mathsf{\text{denominator}}>0\hfill \\ 0,\hfill & \mathsf{\text{otherwise}}\hfill \end{array}\right.$$

(2.4)

From Lemma 1, there exists $A\left(s\right)\in \mathcal{F}$, P(A(s)) = 1, such that

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\mathsf{\text{log}}{\Lambda }_n\left(s,\omega \right)\le 0,\omega \in A\left(s\right)$$

(2.5)

Since ${\int }_{B_j}{f}_j\left(x_j\right)d{x}_j=P\left(X_j\in B_j\right)$, by (2.3) we have

$$\begin{array}{c}{\displaystyle \prod _{j=n}^{n+k_n-1}}{h}_j\left(x_j\right)\hfill \\ ={\displaystyle \prod _{j=n}^{n+k_n-1}}\frac{s^{1_{B_j}\left(x_j\right)}{f}_j\left(x_j\right)}{1+\left(s-1\right){\int }_{B_j}{f}_j\left(x_j\right)d{x}_j}\hfill \\ =s^{{\sum }_{j=n}^{n+k_n-1}{1}_{B_j}\left(x_j\right)}\prod _{j=n}^{n+k_n-1}\frac{f_j\left(x_j\right)}{1+\left(s-1\right)P\left(X_j\in B_j\right)}\hfill \end{array}$$

(2.6)

It follows from (1.1), (2.4) and (2.6) that

$$\mathsf{\text{log}}{\Lambda }_n\left(s,\omega \right)={\displaystyle \sum _{j=n}^{n+k_n-1}}{1}_{B_j}\left(X_j\right)\mathsf{\text{log}}s-{\displaystyle \sum _{j=n}^{n+k_n-1}}\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]+\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)$$

(2.7)

(2.5) and (2.7) yield

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left(\mathsf{\text{log}}s{\displaystyle \sum _{j=n}^{n+k_n-1}}{1}_{B_j}\left(X_j\right)-{\displaystyle \sum _{j=n}^{n+k_n-1}}\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]+\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)\right)\le 0,\omega \in A\left(s\right)$$

(2.8)

Let s > 1, dividing the two sides of (2.8) by log s, we have

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}{1}_{B_j}\left(X_j\right)-{\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\mathsf{\text{log}}\left[1+\left(s-\mathsf{\text{1}}\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}+\frac{\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)}{\mathsf{\text{log}}s}\right)\le 0,\omega \in A\left(s\right)$$

(2.9)

By (1.2), (2.9) and the property lim sup _n (a _n - b _n ) ≤ d ⇒ lim sup _n (a _n - c _n ) ≤ lim sup _n (b _n - c _n ) + d, one gets

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}{1}_{B_j}\left(X_j\right)-{\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\mathsf{\text{log}}\left[1+\left(s-\mathsf{\text{1}}\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}\right)\le \frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s},\omega \in A\left(s\right)$$

(2.10)

By (2.10) and the property of the superior above and the inequality 0 < log(1+x) ≤ x(x > 0), we obtain

$$\begin{array}{c}\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}{S}_{n,k_n}\left(\omega \right)\hfill \\ \le \underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}\right)+\frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s}\hfill \\ \le \underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\left(s-1\right)P\left(X_j\in B_j\right)}{\mathsf{\text{log}}s}\right)+\frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s}\hfill \\ \le c\left(\frac{s-1}{\mathsf{\text{log}}s}\right)+\frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s},\omega \in A\left(s\right)\hfill \end{array}$$

(2.11)

(2.11) and the inequality $1-\frac{1}{s}<\mathsf{\text{log}}s\left(s>1\right)$ imply

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}{S}_{n,k_n}\left(\omega \right)\le c\cdot s+\frac{s\mathcal{L}\left(\omega \right)}{s-1},\omega \in A\left(s\right)$$

(2.12)

Let D be a set of countable real numbers dense in the interval (1, +∞), and let A* = ∩_s∈DA(s), g(s, x) = cs + sx/(s - 1), then we have by (2.12)

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}{S}_{n,k_n}\left(\omega \right)\le g\left(s,\mathcal{L}\left(\omega \right)\right),\omega \in A^{*},s\in D$$

(2.13)

Let c > 0, it easy to see that if $\mathcal{L}\left(\omega \right)>0$, a.e., then, for fixed ω, $g\left(s,\mathcal{L}\left(\omega \right)\right)$as a function of s attains its smallest value $g\left(1+\sqrt{\mathcal{L}\left(\omega \right)/c},\mathcal{L}\left(\omega \right)\right)=2\sqrt{c\mathcal{L}\left(\omega \right)}+\mathcal{L}\left(\omega \right)+c$ on the interval (1, +∞), and g(s, 0) is increasing on the interval (1, +∞) and lim_s→1+ g(s, 0) = 0. For each ω ∈ A* ∩ A(1), if $\mathcal{L}\left(\omega \right)\ne \infty$, take κ _n (ω) ∈ D, n = 1, 2,..., such that ${\kappa }_n\left(\omega \right)\to 1+\sqrt{\mathcal{L}\left(\omega \right)/c}$. We have by the continuity of $g\left(s,\mathcal{L}\left(\omega \right)\right)$ with respect to s,

$$\underset{n\to +\infty }{\mathsf{\text{lim}}}g\left({\kappa }_n\left(\omega \right),\mathcal{L}\left(\omega \right)\right)={\left(\sqrt{\mathcal{L}\left(\omega \right)}+\sqrt{c}\right)}^2,$$

(2.14)

By (2.13), we obtain

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}{S}_{n,k_n}\le g\left({\kappa }_n\left(\omega \right),\mathcal{L}\left(\omega \right)\right),n=1,2,\dots$$

(2.15)

(2.14) and (2.15) imply

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}{S}_{n,k_n}\left(\omega \right)\le {\left(\sqrt{\mathcal{L}\left(\omega \right)}+\sqrt{c}\right)}^2,\omega \in A^{*}\cap A\left(1\right)$$

(2.16)

If $\mathcal{L}\left(\omega \right)=\infty$, (2.16) holds trivially. Since P (A* ∩ A(1)) = 1, (2.2) holds by (2.16), when c > 0.

When c = 0, we have by letting s = e in (2.11),

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}{S}_{n,k_n}\left(\omega \right)\le \mathcal{L}\left(\omega \right),\omega \in A\left(e\right)$$

(2.17)

since P (A(e)) = 1, (2.2) also holds by (2.17) when c = 0. □

Theorem 2. Let (X _n )_n∈N, ${\mathcal{L}}_n\left(\omega \right)$, $\mathcal{L}\left(\omega \right)$, (B _n )_n∈N, $S_{n,k_n}\left(\omega \right)$be defined as in Theorem 1 and assume

$$c^{\prime }=\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}\frac{1}{k_n}{\displaystyle \sum _{j=n}^{n+k_n-1}}P\left(X_j\in B_j\right),$$

(2.18)

then, if$0\le \mathcal{L}\left(\omega \right)\le c^{\prime }$a.e., then

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}{S}_{n,k_n}\left(\omega \right)\ge c^{\prime }-2\sqrt{c^{\prime }\mathcal{L}\left(\omega \right),a.e.}$$

(2.19)

Proof. Let 0 <s < 1, dividing the two sides of (2.8) by log s, we have

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}{1}_{B_j}\left(X_j\right)-{\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}+\frac{\mathsf{\text{log}}{\mathcal{L}}_n\left(\omega \right)}{\mathsf{\text{log}}s}\right)\ge 0,\omega \in A\left(s\right)$$

(2.20)

By (1.2), (2.20) and the property lim inf _n (a _n - b _n ) ≥ d ⇒ lim inf _n (a _n - c _n ) ≥ lim inf _n (b _n - c _n ) + d, one gets

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}{1}_{B_j}\left(X_j\right)-{\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}\right)\ge \frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s},\omega \in A\left(s\right)$$

(2.21)

By (2.21) and the property of the inferior above and the inequality log(1+ x) ≤ x(-1 <x ≤ 0), we obtain

$$\begin{array}{c}\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}{S}_{n,k_n}\left(\omega \right)\hfill \\ \ge \underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}\right)+\frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s}\hfill \\ \ge \underset{n}{\mathsf{\text{lim}}}\mathsf{\text{inf}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\left(s-1\right)P\left(X_j\in B_j\right)}{\mathsf{\text{log}}s}\right)+\frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s}\hfill \\ \ge c^{\prime }\left(\frac{s-1}{\mathsf{\text{log}}s}\right)+\frac{\mathcal{L}\left(\omega \right)}{\mathsf{\text{log}}s},\omega \in A\left(s\right)\hfill \end{array}$$

(2.22)

(2.22) and the inequality $1-\frac{1}{s}<\mathsf{\text{log}}s$ and log s < s - 1 (0 <s < 1) imply

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}{S}_{n,k_n}\left(\omega \right)\ge c^{\prime }\cdot s+\frac{\mathcal{L}\left(\omega \right)}{s-1},\omega \in A\left(s\right)\cap A\left(1\right)$$

(2.23)

Let D' be a set of countable real numbers dense in the interval (0, 1), and let $A_{*}={\cap }_{s\in D^{\prime }}A\left(s\right)$, h(s, x) = c's + x/(s - 1), then we have by (2.23)

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}{S}_{n,k_n}\left(\omega \right)\ge h\left(s,\mathcal{L}\left(\omega \right)\right),\omega \in A_{*},s\in D^{\prime }$$

(2.24)

Let c' > 0, it easy to see that if $0<\mathcal{L}\left(\omega \right)<c^{\prime }$, a.e., then, for fixed ω, $h\left(s,\mathcal{L}\left(\omega \right)\right)$ as a function of s attains its maximum value $h\left(1-\sqrt{\mathcal{L}\left(\omega \right)/c^{\prime }},\mathcal{L}\left(\omega \right)\right)=c^{\prime }-2\sqrt{c^{\prime }\mathcal{L}\left(\omega \right)}$, on the interval (0, 1), and h(s, 0) is increasing on the interval (0, 1) and lim_s→1+ h(s, 0) = c'. For each ω ∈ A_*∩A(1), if $\mathcal{L}\left(\omega \right)\ne \infty$, take l _n (ω) ∈ D', n = 1, 2,..., such that $l_n\left(\omega \right)\to 1-\sqrt{\mathcal{L}\left(\omega \right)/c^{\prime }}$. We have by the continuity of $h\left(s,\mathcal{L}\left(\omega \right)\right)$ with respect to s,

$$\underset{n\to +\infty }{\mathsf{\text{lim}}}h\left(l_n\left(\omega \right),\mathcal{L}\left(\omega \right)\right)=c^{\prime }-2\sqrt{c^{\prime }\mathcal{L}\left(\omega \right)},$$

(2.25)

By (2.24), we obtain

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}{S}_{n,k_n}\ge h\left(l_n\left(\omega \right),\mathcal{L}\left(\omega \right)\right),n=1,2,\dots$$

(2.26)

(2.25) and (2.26) imply

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{inf}}}{S}_{n,k_n}\left(\omega \right)\ge c^{\prime }-2\sqrt{c^{\prime }\mathcal{L}\left(\omega \right)},\omega \in A_{*}\cap A\left(1\right)$$

(2.27)

If $\mathcal{L}\left(\omega \right)=\infty$, (2.27) holds trivially. Since P (A_* ∩ A(1)) = 1, (2.19) holds by (2.27), when c' > 0. (2.19) also holds trivially when c' = 0. □

Remark 2. In case $\mathcal{L}\left(\omega \right)>c^{\prime }\ge 0$, a.e., we cannot get a better lower bound of $\mathsf{\text{lim}}\underset{n}{\mathsf{\text{inf}}}{S}_{n,k_n}\left(\omega \right)$. This motivates the following problem: under the conditions of Theorem 2, how to get a better lower bound of $\mathsf{\text{lim}}\underset{n}{\mathsf{\text{inf}}}{S}_{n,k_n}\left(\omega \right)$ in case of $\mathcal{L}\left(\omega \right)>c^{\prime }\ge 0$, a.e.?

Definition 2. (Generalized empirical distribution function) Let (X _n )_n∈Nbe identically distribution with common distribution function F , for each m, n ∈ N, let

$$F_{m,n}\left(x\right)=\frac{1}{n}{\displaystyle \sum _{k=m}^{m+n-1}}{1}_{\left(X_k\le x\right)}.$$

F _{m, n} = the observed frequency of values that are ≤ x from time m to m + n - 1. The F_1,nis the usual empirical distribution function, hence the name given above.

In particular, let B = (-∞, x], x ∈ R in Theorems 1 and 2, we can get a strong limit theorem for the generalized empirical distribution function.

Corollary 1. Let (X _n )_n∈Nbe i.i.d. random variables with common distribution function F, let B _n = (-∞, x], n = 1, 2,..., then

$$\underset{n}{\mathsf{\text{lim}}}{F}_{n,n+k_n-1}\left(x\right)=F\left(x\right),a.e.$$

Corollary 2. Let (X _n )_n∈Nbe independent random variables and (B _n )_n∈Nbe as Theorem 1, then

$$\underset{n}{\mathsf{\text{lim}}}\frac{1}{k_n}{\displaystyle \sum _{j=n}^{n+k_n-1}}\left[1_{B_j}\left(X_j\right)-P\left(X_j\in B_j\right)\right]=0,a.e.$$

(2.28)

Proof. Note that P (X _j ∈ B _j ) ≤ 1, j = 1, 2,... and in this case, 0 ≤ c, c' ≤ 1, $\mathcal{L}\left(\omega \right)=0$a.e., we have by (2.11)

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}{1}_{B_j}\left(X_j\right)-\sum _{j=n}^{n+k_n-1}\frac{\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}\right)\le 0,\omega \in A\left(s\right)$$

(2.29)

by (2.29) and the property of the superior above and the inequality 0 ≤ log(1+x) ≤ x(x > 0), we obtain

$$\begin{array}{c}\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}{\displaystyle \sum _{j=n}^{n+k_n-1}}\left[1_{B_j}\left(X_j\right)-P\left(X_j\in B_j\right)\right]\hfill \\ \le \underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left({\displaystyle \sum _{j=n}^{n+k_n-1}}\frac{\mathsf{\text{log}}\left[1+\left(s-1\right)P\left(X_j\in B_j\right)\right]}{\mathsf{\text{log}}s}-P\left(X_j\in B_j\right)\right)\hfill \\ \le \underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}\left(\sum _{j=n}^{n+k_n-1}\frac{\left(s-1\right)P\left(X_j\in B_j\right)}{\mathsf{\text{log}}s}-P\left(X_j\in B_j\right)\right)\hfill \\ \le \left(\frac{s-1}{\mathsf{\text{log}}s}-1\right),\omega \in A\left(s\right)\hfill \end{array}$$

(2.30)

Analogously as in the proof of Theorem 1, we obtain

$$\underset{n}{\mathsf{\text{lim}}\mathsf{\text{sup}}}\frac{1}{k_n}{\displaystyle \sum _{j=n}^{n+k_n-1}}\left[1_{B_j}\left(X_j\right)-P\left(X_j\in B_j\right)\right]\le 0,a.e.$$

(2.31)

Similarly, we have $\mathsf{\text{lim}}{\displaystyle {\mathsf{\text{inf}}}_n}\frac{1}{k_n}\sum _{j=n}^{n+k_n-1}\left[1_{B_j}\left(X_j\right)-P\left(X_j\in B_j\right)\right]\ge 0$, a.e. hence (2.28) follows immediately. □

Remark 3. Let B _n = B, Corollary 2 implies that $\frac{\mathsf{\text{lim}}{S}_{n,k_n}}{k_n}=P\left(X_1\in B\right)$ which gives the strong law of large numbers for the delayed arithmatic means.