The consistency for estimator of nonparametric regression model based on NOD errors

Abstract

By using some inequalities for NOD random variables, we give its application to investigate the nonparametric regression model based on these errors. Some consistency results for the estimator of g(x) are presented, including the mean convergence, uniform convergence, almost sure convergence and convergence rate. We generalize some related results and as an example of designed assumptions for weight functions, we give the nearest neighbor weights.

AMS Mathematical Subject Classification 2000: 62G05; 62G08.

1 Introduction

Consider a fixed design regression model

$$Y_{ni}=g\left(x_{ni}\right)+{\epsilon }_{ni},i=1,2,...,n,$$

(1.1)

where x _ni are design points on a set A in ${ℛ}^q$ for some q ≥ 1, g(·) is an unknown function on A and ε _ni are random errors. Assume that for each n, {ε _ni , 1 ≤ i ≤ n} has the same distribution as {ε _i , 1 ≤ i ≤ n}. As an estimator of g(·), the following weighted regression estimator is given:

$$g_n\left(x\right)=\sum _{i=1}^n{W}_{ni}\left(x\right)Y_{ni},$$

where W _ni (x) = W _ni (x,x_{n 1},...,x _nn ) are weighted functions.

The above estimator was first proposed by Georgiev [1] and subsequently has been studied by many authors. In the independent case, consistency and asymptotic normality have been investigated by Georgiev and Greblicki [2], Georgiev [3], Müller [4], and the references therein. Fan [5] extended the work of Georgiev [3] and Müller [4] in the estimation of the regression model to the case of L^q-mixingale for some 1 ≤ q ≤ 2. Roussas [6] discussed strong consistency and quadratic mean consistency of g _n (x), and Roussas et al. [7] established asymptotic normality of g _n (x) assuming that the errors are from a strictly stationary stochastic process and satisfying the strong mixing condition. Tran et al. [8] obtained the asymptotic normality of g _n (x) assuming that the errors form a linear time series, more precisely, a weakly stationary linear process based on a martingale difference sequence. Hu et al. [9] generalized the main results of Tran et al. [8]. Liang and Jing [10] established the consistency, uniform consistency, and asymptotic normality of g _n (x) under negatively associated (NA) samples. Meanwhile, for the semiparametric regression model, Ren and Chen [11] obtained the strong consistency for the least squares estimator of β and the nonparametric estimator of g(t) based on NA samples, Hu [12] obtained the consistency and complete consistency for these estimations based on the linear time series, Baek and Liang [13] established some asymptotic results for these estimations under NA samples, Liang et al. [14] also established some asymptotic results for a linear process based NA samples, etc. For more details of semiparametric regression model, one can refer to Hardle et al. [15] and the references therein.

In this article, we investigate the nonparametric regression model based on negatively orthant dependent (NOD) random variables, which is weaker than NA random variables. Some related definitions are given as follows:

Definition 1.1 Two random variables X and Y are said to be NQD if for ∀ x, y ∈ R,

$$P\left(X\le x,Y\le y\right)\le P\left(X\le x\right)P\left(Y\le y\right).$$

A sequence of random variables {X _n , n ≥ 1} is said to be pairwise NQD if for all i, j ∈ N, I ≠ j, and X _i and X _j are NQD.

The concept of NQD was intruduced by Lehmann [16] and he pointed out some useful properties of NQD, for example, let X and Y be NQD, then

(i) EXY ≤ EXEY,

(ii) P(X > x, Y > y) ≤ P(X > x)P(Y > y), for ∀ x, y ∈ R,

(iii) if f, g are both nondecreasing (or nonincreasing) functions, then f(X) and g(Y) are NQD.

Definition 1.2 A finite collection of random variables X₁, X₂,..., X _n is said to be NA if for every pair of disjoint subsets A₁, A₂ of {1, 2,..., n},

$$\mathsf{\text{Cov}}\left\{f\left(X_i:i\in A_1\right),g\left(X_j:j\in A_2\right)\right\}\le 0,$$

whenever f and g are coordinatewise nondecreasing such that this covariance exists.

An infinite sequence {X _n }_n≥1is NA if every finite subcollection is NA.

Definition 1.3 A finite collection of random variables X₁, X₂,...,X _n is said to be negatively upper orthant dependent (NUOD) if for all real numbers x₁,x₂,...,x _n ,

$$P\left(X_i>x_i,i=1,2,...,n\right)\le \prod _{i=1}^{n}P\left(X_i>x_i\right),$$

and negatively lower orthant dependent (NLOD) if for all real numbers x₁,x₂,...,x _n ,

$$P\left(X_i\le x_i,i=1,2,...,n\right)\le \prod _{i=1}^{n}P\left(X_i\le x_i\right).$$

A finite collection of random variables X₁, X₂,...,X _n is said to be NOD if they are both NUOD and NLOD.

An infinite sequence {X _n }_n≥1is said to be NOD (NUOD or NLOD) if every finite subcollection is NOD (NUOD or NLOD).

The concepts of NA and NOD sequences were introduced by Joag-Dev and Proschan [17]. They pointed out that NA random variables are NOD random variables, but neither NUOD nor NLOD implies NA. Various results and examples of NOD random variables can be found in Joag-Dev and Proschan [17], Bozorgnia et al. [18], Asadian et al. [19], Wang et al. [20], Wu [21, 22], Wang et al. [23, 24], Li et al. [25] and Sung [26], etc. Obviously, by the definitions of NOD and pairwise NQD, NOD random variables are pairwise NQD random variables. For more results and examples of pairwise NQD random variables, one can refer to Lehmann [16], Matula [27], Wu [28], Gan and Chen [29], Li and Yang [30], etc. But unlike NOD random variables, pairwise NQD random variables have not some nice inequalities such as Bernstein-type inequality as we know.

Inspired by Liang and Jing [10] and other articles referred above, we investigate the nonparametric regression model based on NOD random errors. By using the moment inequality, Bernstein-type inequality and truncating method for NOD random variables, we obtain some consistency results for estimator of g(x) such as the mean convergence, uniform convergence, almost sure convergence and convergence rate. We generalize some results of Liang and Jing [10] for NA random variables to the case of NOD random variables. Meanwhile, as an example of designed assumptions for weight functions, we give the nearest neighbor weights.

For any function g(x), we use c(g) to denote all continuity points of function g on the set A in ${ℛ}^q$ for some q ≥ 1. Let c, c₁, c₂, C, C₁, C₂,... denote the positive constants whose values may vary at each occurrence. ⌈x⌉ denotes the largest integer not exceeding x, I(B) is the indicator function of set B, x⁺ = xI(x ≥ 0), x^- = -xI(x < 0) and ∥x∥ denotes Euclidean norm of x. In this article, main results are presented in Section 2, some lemmas and the proofs of main results are presented in Sections 3 and 4, respectively.

2 The main results

Under the nonparametric regression model of (1.1), for any fixed point x ∈ A, some assumptions on weighted function W _ni (x) = W _ni (x, x_{n 1},..., x _nn ) are given as follows:

(H₁) ${\sum }_{i=1}^n{W}_{ni}\left(x\right)\to 1$ as n → ∞;

(H₂) ${\sum }_{i=1}^n\left|W_{ni}\left(x\right)\right|\le C$ for all n;

(H₃) ${\sum }_{i=1}^n{W}_{ni}^2\left(x\right)\to 0$ as n → ∞;

(H₄) ${\sum }_{i=1}^n\left|W_{ni}\left(x\right)\right|\cdot \left|g\left(x_{ni}\right)-g\left(x\right)\right|I\left(∥x_{ni}-x∥>a\right)\to 0$ as n → ∞ for all a > 0.

Theorem 2.1 Let {ε _n , n ≥ 1} be a mean zero NOD sequence. Assume that the conditions (H₁)-(H₄) hold true. If $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^2<\infty$, then for x ∈ c(g) and some p ∈ (0, 2],

$$E{\left|g\left(x\right)-g\left(x\right)\right|}^p\to 0,asn\to \infty .$$

(2.1)

If $\underset{n\ge 1}{\text{sup}}E{\left|{\epsilon }_n\right|}^p<\infty$ for some p > 2, then (2.1) also holds true.

In order to obtain uniform convergence for the estimator of g(x), for any fixed point x on a compact set A in ${ℛ}^q$ for some q ≥ 1, some uniform version of assumptions on W _ni (x) = W _ni (x,x_{n 1},...,x _nn ) are replaced by that as follows:

$$\left(H_1^{\prime }\right){\text{sup}}_{x\in A}\left|{\sum }_{i=1}^n{W}_{ni}\left(x\right)-1\right|\to 0\mathsf{\text{as}}n\to \infty ;$$

$$\left(H_2^{\prime }\right){\text{sup}}_{x\in A}\left|{\sum }_{i=1}^n{W}_{ni}\left(x\right)\right|\le C\mathsf{\text{for}}\mathsf{\text{all}}n;$$

$$\left(H_3^{\prime }\right){\text{sup}}_{x\in A}{\sum }_{i=1}^n{W}_{ni}^2\left(x\right)\to 0\mathsf{\text{as}}n\to \infty ;$$

$$\left(H_4^{\prime }\right){\text{sup}}_{x\in A}{\sum }_{i=1}^n\left|W_{ni}\left(x\right)\right|I\left(∥x_{ni}-x∥>a\right)\to 0\text{as}n\to \infty \mathsf{\text{for}}\mathsf{\text{all}}a>0.$$

Theorem 2.2 Let {ε _n , n > 1} be a mean zero NOD sequence. Assume that the conditions $\left(H_1^{\prime }\right)-\left(H_4^{\prime }\right)$ hold true and g is continuous on the compact set A. If ${\sup }_{n\ge 1}E{\epsilon }_n^2<\infty$, then for some p ∈ (0, 2],

$$\underset{x\in A}{\text{sup}}E{\left|g_n\left(x\right)-g\left(x\right)\right|}^p\to 0,asn\to \infty .$$

(2.2)

If $\underset{n\ge 1}{\text{sup}}E{\left|{\epsilon }_n\right|}^p<\infty$ for some p > 2, then (2.2) also holds true.

Next, we will study the almost sure convergence and convergence rate for the estimator of g(x). Similarly, for any fixed point x on the compact set A in ${ℛ}^q$ for some q ≥ 1, some assumptions on the W _ni (x) = W _ni (x, x_{n 1},...,x _nn ) are shown as follows:

(H₅) $\left|{\sum }_{i=1}^n{W}_{ni}\left(x\right)-1\right|=O\left(n^{-1/4}\right);$;

(H₆) ${\sum }_{i=1}^n\left|W_{ni}\left(x\right)\right|\le C$ for all n ≥ 1 and ${\max }_{1\le i\le n}\left|W_{ni}(x)\right|=O\left(n^{-1/2}{\log }^{-3/2}n\right)$;

(H₇) ${\sum }_{i=1}^n\left|W_{ni}\left(x\right)\right|\cdot \left|g\left(x_{ni}\right)-g\left(x\right)\right|I\left(∥x_{ni}-x∥>a{n}^{-1/4}\right)=O\left(n^{-1/4}\right)$ for some a > 0.

Theorem 2.3 Let {ε _n , n ≥ 1} be a mean zero NOD sequence such that ${\sup }_{n\ge 1}E{\epsilon }_n^2<\infty$. Suppose that the conditions (H₅)-(H₇) hold true and g(x) satisfies a local Lipschitz condition around the point x. Then for x ∈ A,

$$g_n\left(x\right)\to g\left(x\right),\mathsf{\text{as}}n\to \infty ,a.s.$$

(2.3)

Theorem 2.4 Let {ε _n , n ≥ 1} be a mean zero NOD sequence such that ${\sup }_{n\ge 1}E{\epsilon }_n^2<\infty$. Suppose that the conditions (H₅)-(H₇) hold true and g(x) satisfies a local Lipschitz condition around the point x. Then for x ∈ A,

$$g_n\left(x\right)-g\left(x\right)=O\left(n^{-1/4}\right),a.s.$$

(2.4)

Remark 2.1 The similar assumptions on weighted functions can be found in Ren and Chen [11], Hu et al. [31] and Liang and Jing [10], etc. Under the NA sequence and other assumptions, for some p > 1, Liang and Jing [10] obtained the result E|g _n (x)-g(x)|^p→ 0 as n → ∞ (see Liang and Jing [10, Theorem 2.1]). In our Theorem 2.1, we give the result E|g _n (x)-g(x)|^p→ 0 as n → ∞ for some p > 0. Liang and Jing [10] also studied the strong consistency of the estimator for g(x). In our Theorems 2.3 and 2.4, the strong consistency and convergence rate of the estimator for g(x) are presented. Since NA sequence is a NOD sequence, we generalize some results of Liang and Jing [10] to the case of NOD sequence.

Example 2.1 Here, we give an example that the designed assumptions (H₅)-(H₇) are satisfied for the nearest neighbor weights. Without loss of generality, let A = [0,1] and $x_{ni}=\frac{i}{n},1\le i\le n$. For x ∈ A, we rewrite |x_{n 1}- x|, |x_{n 2}- x|,..., |x _nn - x| as follows

$$\left|x_{R_1\left(x\right)}^{\left(n\right)}-x\right|\le \left|x_{R_2\left(x\right)}^{\left(n\right)}-x\right|\le \cdots \left|x_{R_n\left(x\right)}^{\left(n\right)}-x\right|,$$

(2.5)

if |x _ni - x| = |x _nj - x|, |x _ni - x| is in frond of |x _nj - x| for i < j. Let k _n = ⌈n^5/8⌉ and define the nearest neighbor weight functions as following

$$W_{ni}\left(x\right)=W_{ni}\left(x,x_{n1},x_{n2},\dots ,x_{nn}\right)=\left\{\begin{array}{cc}\frac{1}{k_n}\hfill & \mathsf{\text{if}}\left|x_{ni}-x\right|\le \left|x_{R_k\left(x\right)}^{\left(n\right)}-x\right|,\hfill \\ 0,\hfill & \mathsf{\text{otherwise}}.\hfill \end{array}\right.$$

(2.6)

Consequently, for every x ∈ [0,1], we can find by definition of R _i (x) and choice of x _ni that

$$\sum _{i=1}^n{W}_{ni}\left(x\right)=\sum _{i=1}^n{W}_{n{R}_i\left(x\right)}\left(x\right)=\sum _{i=1}^{k_n}\frac{1}{k_n}=1,$$

(2.7)

$$\underset{1\le i\le n}{\text{max}}{W}_{ni}\left(x\right)=\frac{1}{k_n}\le \frac{c_1}{n^{5/8}},$$

(2.8)

$$\begin{array}{ll}\hfill \sum _{i=1}^n{W}_{ni}\left(x\right)I\left(\left|x_{ni}-x\right|>a{n}^{-1/4}\right)& \le \sum _{i=1}^n{W}_{ni}\left(x\right)\frac{{\left(x_{ni}-x\right)}^2}{a^2{n}^{-1/2}}\\ =\sum _{i=1}^{k_n}\frac{{\left(x_{R_i\left(x\right)}^{\left(n\right)}-x\right)}^2}{k_n{a}^2}{n}^{1/2}\le \sum _{i=1}^{k_n}\frac{{\left(i/n\right)}^2}{k_n{a}^2}{n}^{1/2}\\ \le {\left(\frac{k_n}{na}\right)}^2{n}^{1/2}\le \frac{c_2}{n^{1/4}},\forall a>0.\end{array}$$

(2.9)

If g is continuous on [0,1], then by (2.6)-(2.9), it can find that the assumptions of (H₁)-(H₇) and $\left(H_1^{\prime }\right)-\left(H_4^{\prime }\right)$ are satisfied.

3 Some lemmas

Lemma 3.1 (cf. Bozorgnia et al. [18]). Let random variables X₁,X₂,...,X _n be NOD, f₁,f₂,...,f _n be all nondecreasing (or nonincreasing) functions, then random variables f₁(X₁), f₂(X₂),..., f _n (X _n ) are NOD.

Lemma 3.2 (cf. Asadian et al. [19]). Let {X _n , n ≥ 1} be a NOD sequence such that EX _n = 0 and E|X _n |^p< ∞ for all n ≥ 1 and p ≥ 2. Then

$$E{\left|\sum _{i=1}^n{X}_i\right|}^p\le c_p\left\{\sum _{i=1}^{n}E{\left|X_i\right|}^p+{\left(\sum _{i=1}^{n}E{X}_i^2\right)}^{p/2}\right\},$$

where c _p depends only on p.

Lemma 3.3 (cf. Wang et al. [20]). Let {X _n }_n≥1be a sequence of NOD random variables such that EX _n = 0 and |X _n | ≤ b for each n ≥ 1, where b is a positive constant.

Denote ${\Delta }_n^2={\sum }_{i=1}^{n}E{X}_i^2$ for each n ≥ 1. Then for every ε > 0,

$$P\left(\left|\sum _{i=1}^n{X}_i\right|\ge \epsilon \right)\le 2\text{exp}\left\{-\frac{{\epsilon }^2}{2\left(2{\Delta }_n^2+b\epsilon \right)}\right\}.$$

4 Proofs of the main results

Proof of Theorem 2.1: By C _r inequality, it has

$$E{\left|g_n\left(x\right)-g\left(x\right)\right|}^p\le c_p\left\{E{\left|g_n\left(x\right)-E{g}_n\left(x\right)\right|}^p+{\left|E{g}_n\left(x\right)-g\left(x\right)\right|}^p\right\}.$$

(4.1)

For x ∈ c(g) and a > 0,

$$\begin{array}{ll}\hfill \left|E{g}_n\left(x\right)-g\left(x\right)\right|& \le \sum _{i=1}^n\left|W_{ni}\left(x\right)\right|\cdot \left|g\left(x_{ni}\right)-g\left(x\right)\right|I\left(∥x_{ni}-x∥\le a\right)\\ +\sum _{i=1}^n\left|W_{ni}\left(x\right)\right|\cdot \left|g\left(x_{ni}\right)-g\left(x\right)\right|I\left(∥x_{ni}-x∥>a\right)\\ +\left|g\left(x\right)\right|\cdot \left|\sum _{i=1}^n{W}_{ni}\left(x\right)-1\right|.\end{array}$$

So, similar to the proof of (2.10) in Hu et al. [31], by conditions (H₁), (H₂) and (H₄), it is easy to have that

$$\left|E{g}_n\left(x\right)-g\left(x\right)\right|\to 0,x\in c\left(g\right).$$

(4.2)

On the other hand, by Lemma 3.1, for the fixed x, we can see that $\left\{W_{ni}^{+}\left(x\right){\epsilon }_i,1\le i\le n\right\}$ and $\left\{W_{ni}^{-}\left(x\right){\epsilon }_i,\le i\le n\right\}$ are also NOD sequences. Combining with $W_{ni}\left(x\right){\epsilon }_i=W_{ni}^{+}\left(x\right){\epsilon }_i-W_{ni}^{-}\left(x\right){\epsilon }_i$, without loss of generality, we assume W _ni (x) ≥ 0 in the proof. If 0 < p ≤ 2, by Jensen's inequality, Lemma 3.2, (H₃) and $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^2<\infty$, we have

$$\begin{array}{ll}\hfill E{\left|g_n\left(x\right)-E{g}_n\left(x\right)\right|}^p& =E{\left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_{ni}\right|}^p\\ =E{\left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_i\right|}^p\le {\left[E{\left(\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_i\right)}^2\right]}^{p/2}\\ \le C_1{\left[\sum _{i=1}^n{W}_{ni}^2\left(x\right)E{\epsilon }_i^2\right]}^{p/2}\le C_2{\left[\sum _{i=1}^n{W}_{ni}^2\left(x\right)\right]}^{p/2}\to 0,\end{array}$$

(4.3)

following from that {ε _ni , 1 ≤ i ≤ n} has the same distribution as {ε _i , 1 ≤ i ≤ n} for each n. Otherwise, for p > 2, by Lemma 3.2, $\underset{n\ge 1}{\text{sup}}E{\left|\epsilon \right|}^p<\infty$ and (H₃) again, we obtain

$$\begin{array}{ll}\hfill E{\left|g_n\left(x\right)-E{g}_n\left(x\right)\right|}^p& =E{\left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_{ni}\right|}^p=E{\left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_i\right|}^p\\ \le C_3\left\{\sum _{i=1}^n{W}_{ni}^p\left(x\right)E{\left|{\epsilon }_i\right|}^p+{\left[\sum _{i=1}^n{W}_{ni}^2\left(x\right)E{\epsilon }_i^2\right]}^{p/2}\right\}\\ \le C_4\left\{{\left[\sum _{i=1}^n{W}_{ni}^2\left(x\right)\right]}^{p/2}+{\left[\sum _{i=1}^n{W}_{ni}^2\left(x\right)\right]}^{p/2}\right\}\to 0,\end{array}$$

(4.4)

since ${\left({\sum }_{i=1}^n{a}_i^{\alpha }\right)}^{1/\alpha }\ge {\left({\sum }_{i=1}^n{a}_i^{\beta }\right)}^{1/\beta }$ for any positive number sequence {a _i ,1 ≤ i ≤ n} and 1 ≤ α ≤ β. Therefore, by (4.1)-(4.4), the desired result (2.1) has been proved completely.

Proof of Theorem 2.2: Since g is continuous in the compact set A, g is uniformly continuous in the compact set A. Consequently, similar to the proof of Theorem 2.1, we can get that

$$\underset{n\to \infty }{\text{lim}}\underset{x\in A}{\text{sup}}E{\left|g_n\left(x\right)-E{g}_n\left(x\right)\right|}^p=0,\underset{n\to \infty }{\text{lim}}\underset{x\in A}{\text{sup}}\left|E{g}_n\left(x\right)-g\left(x\right)\right|=0.$$

Therefore,

$$\underset{n\to \infty }{\text{lim}}\underset{x\in A}{\text{sup}}E{\left|g_n\left(x\right)-g\left(x\right)\right|}^p\le c_p\left\{\underset{n\to \infty }{\text{lim}}\underset{x\in A}{\text{sup}}E{\left|g_n\left(x\right)-E{g}_n\left(x\right)\right|}^p+\underset{n\to \infty }{\text{lim}}\underset{x\in A}{\text{sup}}{\left|E{g}_n\left(x\right)-g\left(x\right)\right|}^p\right\}=0,$$

which implies the desired result (2.2).

Proof of Theorem 2.3: Combining the proof of (4.2) with the assumptions of (H₅)-(H₇) and g(x) satisfying a local Lipschitz condition around the point x, we can get that

$$\left|E{g}_n\left(x\right)-g\left(x\right)\right|=O\left(n^{-1/4}\right).$$

(4.5)

Therefore, for x ∈ A, to prove (2.3), we only have to show that

$$g_n\left(x\right)-E{g}_n\left(x\right)\to 0,\mathsf{\text{as}}n\to \infty ,a.s.$$

(4.6)

Without loss of generality, we assume that W _ni (x) ≥ 0 in the proof. Let

$$\begin{array}{ll}\hfill {\epsilon }_{1,i}^{\left(n\right)}& =-i^{1/2}I\left({\epsilon }_{ni}<-i^{1/2}\right)+{\epsilon }_{ni}I\left(\left|{\epsilon }_{ni}\right|\le i^{1/2}\right)+i^{1/2}I\left({\epsilon }_{ni}>i^{1/2}\right),\\ \hfill {\epsilon }_{2,i}^{\left(n\right)}& =\left({\epsilon }_{ni}-i^{1/2}\right)I\left({\epsilon }_{ni}>i^{1/2}\right),{\epsilon }_{3,i}^{\left(n\right)}=\left({\epsilon }_{ni}+i^{1/2}\right)I\left({\epsilon }_{ni}<-i^{1/2}\right),\\ \hfill {\epsilon }_{1,i}& =-i^{1/2}I\left({\epsilon }_i<-i^{1/2}\right)+{\epsilon }_{i}I\left(\left|{\epsilon }_i\right|\le i^{1/2}\right)+i^{1/2}I\left({\epsilon }_i>i^{1/2}\right),\\ \hfill {\epsilon }_{2,i}& =\left({\epsilon }_i-i^{1/2}\right)I\left({\epsilon }_i>i^{1/2}\right),{\epsilon }_{3,i}=\left({\epsilon }_i+i^{1/2}\right)I\left({\epsilon }_i<-i^{1/2}\right).\end{array}$$

Since $E{\epsilon }_{ni}=E{\epsilon }_i=0$ for each n, it is easy to see that

$$\begin{array}{l}{g}_n\left(x\right)-E{g}_n\left(x\right)=\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_{ni}\\ =\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{1,i}^{\left(n\right)}-E{\epsilon }_{1,i}^{\left(n\right)}\right]+\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{2,i}^{\left(n\right)}-E{\epsilon }_{2,i}^{\left(n\right)}\right]+\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{3,i}^{\left(n\right)}-E{\epsilon }_{3,i}^{\left(n\right)}\right]\\ =:{T_n}_1+T_{n2}+T_{n3}.\end{array}$$

(4.7)

Obviously, for the fixed x and n, ${\left\{W_{ni}\left(x\right)\left({\epsilon }_{1,i}-E{\epsilon }_{1,i}\right)\right\}}_{1\le i\le n}$ is a NOD sequence with mean zero. Meanwhile, by the condition (H₆), it has

$$\begin{array}{ll}\hfill \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\left({\epsilon }_{1,i}-E{\epsilon }_{1,i}\right)\right|& \le 2n^{1/2}\underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\le c_1{\text{log}}^{-3/2}n,\\ \hfill \sum _{i=1}^n\mathsf{\text{Var}}\left[W_{ni}\left(x\right)\left({\epsilon }_{1,i}-E{\epsilon }_{1,i}\right)\right]& \le \sum _{i=1}^n{W}_{ni}^2\left(x\right)E{\epsilon }_i^2\\ \le c_2\underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^n\left|W_{ni}\left(x\right)\right|\le c_3{n}^{-1/2}{\text{log}}^{-3/2}n.\end{array}$$

Since {ε _ni , 1 ≤ i ≤ n} has the same distribution as {ε _i , 1 ≤ i ≤ n} for each n, we obtain by applying Lemma 3.3 that for every ϵ > 0,

$$\begin{array}{ll}\hfill P\left(\left|T_{n1}\right|\ge \epsilon \right)& =P\left(\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{1,i}^{\left(n\right)}-E{\epsilon }_{1,i}^{\left(n\right)}\right]\ge \epsilon \right)=P\left(\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{1,i}-E{\epsilon }_{1,i}\right]\ge \epsilon \right)\\ \le 2\text{exp}\left\{\frac{{\epsilon }^2}{2\left(2c_3{n}^{-1/2}{\text{log}}^{-3/2}n+c_1\epsilon {\text{log}}^{-3/2}n\right)}\right\}\\ \le 2\text{exp}\left\{-c_4{\text{log}}^{3/2}n\right\}\le c_5{n}^{-2},\mathsf{\text{for}}n\mathsf{\text{large}}\mathsf{\text{enough}},\end{array}$$

which implies

$$T_{n1}=\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{1,i}^{\left(n\right)}-E{\epsilon }_{1,i}^{\left(n\right)}\right]\to 0,\mathsf{\text{as}}n\to \infty ,a.s.,$$

(4.8)

following from Borel-Cantelli lemma.

Next, we turn to estimate T_{n 2}and T_{n 3}. It can be checked by $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^2<\infty$ that

$$\begin{array}{ll}\hfill \sum _{i=1}^{\infty }\frac{E{\epsilon }_{2,i}^{\left(n\right)}}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}& =\sum _{i=1}^{\infty }\frac{E{\epsilon }_{2,i}}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}\le \sum _{i=1}^{\infty }\frac{E\left[{\epsilon }_{i}I\left({\epsilon }_i>i^{1/2}\right)\right]}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}\\ \le \sum _{i=1}^{\infty }\frac{E{\epsilon }_i^2}{i{\text{log}}^{5/4}\left(2i\right)}<\infty ,\end{array}$$

(4.9)

which implies

$$\sum _{i=1}^{\infty }\frac{{\epsilon }_{2,i}^{\left(n\right)}}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}<\infty ,a.s.$$

Consequently, by Kronecker's lemma, we have that

$$\frac{1}{n^{1/2}{\text{log}}^{5/4}\left(2n\right)}\sum _{i=1}^n{\epsilon }_{2,i}^{\left(n\right)}\to 0,a.s.$$

Thus, by the condition (H₆), it is easy to see that

$$\begin{array}{ll}\hfill \left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_{2,i}^{\left(n\right)}\right|& \le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^n{\epsilon }_{2,i}^{\left(n\right)}\le c_7{n}^{-1/2}{\text{log}}^{-3/2}n\sum _{i=1}^n{\epsilon }_{2,i}^{\left(n\right)}\\ =o\left({\text{log}}^{-1/4}n\right),a.s.\end{array}$$

(4.10)

Obviously, by $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^2<\infty$ and (H₆) again,

$$\begin{array}{ll}\hfill \left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{2,i}^{\left(n\right)}\right|& =\left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{2,i}\right|\le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^{n}E\left[\left|{\epsilon }_i\right|I\left(\left|{\epsilon }_i\right|\ge i^{1/2}\right)\right]\\ \le c_8{n}^{-1/2}{\text{log}}^{-3/2}n\sum _{i=1}^n{i}^{-1/2}E\left[{\epsilon }_i^{2}I\left(\left|{\epsilon }_i\right|\ge i^{1/2}\right)\right]\\ =O\left({\text{log}}^{-3/2}n\right).\end{array}$$

(4.11)

Combining (4.10) with (4.11), it follows

$$\left|T_{n2}\right|=\left|\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{2,i}^{\left(n\right)}-E{\epsilon }_{2,i}^{\left(n\right)}\right]\right|=o\left({\text{log}}^{-1/4}n\right),a.s.$$

(4.12)

Likewise, by $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^2<\infty$, we will found that

$$\sum _{i=1}^{\infty }\frac{E\left|{\epsilon }_{3,i}^{\left(n\right)}\right|}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}=\sum _{i=1}^{\infty }\frac{E\left|{\epsilon }_{3,i}\right|}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}\le \sum _{i=1}^{\infty }\frac{-E\left[{\epsilon }_{i}I\left({\epsilon }_i<-i^{1/2}\right)\right]}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}\le \sum _{i=1}^{\infty }\frac{E{\epsilon }_i^2}{i{\text{log}}^{5/4}\left(2i\right)}<\infty$$

which implies

$$\sum _{i=1}^{\infty }\frac{\left|{\epsilon }_{3,i}^{\left(n\right)}\right|}{i^{1/2}{\text{log}}^{5/4}\left(2i\right)}<\infty ,a.s.$$

Then, by Kronecker's lemma

$$\frac{1}{n^{1/2}{\text{log}}^{5/4}\left(2n\right)}\sum _{i=1}^n\left|{\epsilon }_{3,i}^{\left(n\right)}\right|\to 0,a.s.$$

Consequently, by (H₆), it has that

$$\left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_{3,i}^{\left(n\right)}\right|\le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^n\left|{\epsilon }_{3,i}^{\left(n\right)}\right|=o\left({\text{log}}^{-1/4}n\right),a.s.$$

On the other hand, by (H₆) and $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^2<\infty$ again,

$$\begin{array}{ll}\hfill \left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{3,i}^{\left(n\right)}\right|& =\left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{3,i}\right|\le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^{n}E\left[\left|{\epsilon }_i\right|I\left(\left|{\epsilon }_i\right|>i^{1/2}\right)\right]\\ \le c{n}^{-1/2}{\text{log}}^{-3/2}n\sum _{i=1}^n{i}^{-1/2}E\left[{\epsilon }_i^{2}I\left(\left|{\epsilon }_i\right|>i^{1/2}\right)\right]\\ =O\left({\text{log}}^{-3/2}n\right).\end{array}$$

Finally,

$$\left|T_{n3}\right|=\left|\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{3,i}^{\left(n\right)}-E{\epsilon }_{3,i}^{\left(n\right)}\right]\right|=o\left({\text{log}}^{-1/4}n\right),a.s.$$

(4.13)

Therefore, by (4.7), (4.8), (4.12) and (4.13), (4.6) is completely proved. The desired result (2.3) follows from (4.5) and (4.6) immediately.

Proof of Theorem 2.4: By the estimation of (4.5), to prove (2.4), we only need to prove that |g _n (x) - Eg _n (x)| = O(n^-1/4), a.s. It is also to assume that W _ni (x) ≥ 0 in the proof. Similar to the proof of Theorem 2.3, we will use the same notation ${\epsilon }_{q,i}^{\left(n\right)},{\epsilon }_{q,i}$ and T _nq for q = 1, 2, 3, where i^1/2 is replaced by i^1/4. Obviously $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^4<\infty$ implies $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^2<\infty$, by (H₆), it has

$$\begin{array}{l}\underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\left({\epsilon }_{1,i}-E{\epsilon }_{1,i}\right)\right|\le 2n^{1/4}\underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\le c_1{n}^{-1/4}{\text{log}}^{-3/2}n,\\ \sum _{i=1}^n\mathsf{\text{Var}}\left[W_{ni}\left(x\right)\left({\epsilon }_{1,i}-E{\epsilon }_{1,i}\right)\right]\le \sum _{i=1}^n{W}_{ni}^2\left(x\right)E{\epsilon }_i^2\le c_2{n}^{-1/2}{\text{log}}^{-3/2}n.\end{array}$$

Since {ε _ni , 1 ≤ i ≤ n} has the same distribution as {ε _i , 1 ≤ i ≤ n} for each n, we obtain by applying Lemma 3.3 that for every ϵ > 0

$$\begin{array}{ll}\hfill P\left(\left|T_{n1}\right|\ge \epsilon n^{-1/4}\right)& =P\left(\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{1,i}^{\left(n\right)}-E{\epsilon }_{1,i}^{\left(n\right)}\right]\ge \epsilon n^{-1/4}\right)\\ =P\left(\sum _{i=1}^n{W}_{ni}\left(x\right)\left[{\epsilon }_{1,i}-E{\epsilon }_{1,i}\right]\ge \epsilon n^{-1/4}\right)\\ \le 2\text{exp}\left\{-\frac{{\epsilon }^2{n}^{-1/2}}{2\left(2c_2{n}^{-1/2}{\text{log}}^{-3/2}n+c_1\epsilon n^{-1/2}{\text{log}}^{-3/2}n\right)}\right\}\\ \le 2\text{exp}\left\{-c_3{\text{log}}^{3/2}n\right\}\le c_4{n}^{-2},\mathsf{\text{for}}n\mathsf{\text{large}}\mathsf{\text{enough}},\end{array}$$

which implies by Borel-Cantelli lemma that

$$n^{1/4}{T}_{n1}\to 0,a.s.$$

(4.14)

Meanwhile, it can be checked by $\underset{n\ge 1}{\text{sup}}E{\epsilon }_n^4<\infty$ that

$$\sum _{i=1}^{\infty }\frac{E{\epsilon }_{2,i}^{\left(n\right)}}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}=\sum _{i=1}^{\infty }\frac{E{\epsilon }_{2,i}}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}\le \sum _{i=1}^{\infty }\frac{E\left[{\epsilon }_{i}I\left({\epsilon }_i>i^{1/4}\right)\right]}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}\le \sum _{i=1}^{\infty }\frac{E{\epsilon }_i^4}{i{\text{log}}^{3/2}\left(2i\right)}<\infty ,$$

which implies

$$\sum _{i=1}^{\infty }\frac{{\epsilon }_{2,i}^{\left(n\right)}}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}<\infty ,a.s.$$

Then, we have by Kronecker's lemma that

$$\frac{1}{n^{1/4}{\text{log}}^{3/2}\left(2n\right)}\sum _{i=1}^n{\epsilon }_{2,i}^{\left(n\right)}\to 0,a.s.$$

Consequently, by (H.6), it follows

$$\left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_{2,i}^{\left(n\right)}\right|\le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^n{\epsilon }_{2,i}^{\left(n\right)}=o\left(n^{-1/4}\right),a.s.,$$

(4.15)

and

$$\begin{array}{ll}\hfill \left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{2,i}^{\left(n\right)}\right|& =\left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{2,i}\right|\\ \le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^{n}E\left[\left|{\epsilon }_i\right|I\left(\left|{\epsilon }_i\right|>i^{1/4}\right)\right]\\ \le c_5{n}^{-1/2}{\text{log}}^{-3/2}n\sum _{i=1}^n{i}^{-3/4}E\left[{\epsilon }_i^{4}I\left(\left|{\epsilon }_i\right|>i^{1/4}\right)\right]\\ =O\left(n^{-1/4}{\text{log}}^{-3/2}n\right).\end{array}$$

(4.16)

On the other hand, it can be checked that

$$\sum _{i=1}^{\infty }\frac{E\left|{\epsilon }_{3,i}^{\left(n\right)}\right|}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}=\sum _{i=1}^{\infty }\frac{E\left|{\epsilon }_{3,i}\right|}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}\le \sum _{i=1}^{\infty }\frac{-E\left[{\epsilon }_{i}I\left({\epsilon }_i<-i^{1/4}\right)\right]}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}\le \sum _{i=1}^{\infty }\frac{E{\epsilon }_i^4}{i{\text{log}}^{3/2}\left(2i\right)}<\infty ,$$

which implies

$$\sum _{i=1}^{\infty }\frac{\left|{\epsilon }_{3,i}^{\left(n\right)}\right|}{i^{1/4}{\text{log}}^{3/2}\left(2i\right)}<\infty ,a.s.$$

So, by Kronecker's lemma,

$$\frac{1}{n^{1/4}{\text{log}}^{3/2}\left(2n\right)}\sum _{i=1}^n\left|{\epsilon }_{3,i}^{\left(n\right)}\right|\to 0,a.s.$$

Consequently, by (H.6), we have

$$\left|\sum _{i=1}^n{W}_{ni}\left(x\right){\epsilon }_{3,i}^{\left(n\right)}\right|\le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^n\left|{\epsilon }_{3,i}^{\left(n\right)}\right|=o\left(n^{-1/4}\right),a.s.,$$

(4.17)

and

$$\begin{array}{ll}\hfill \left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{3,i}^{\left(n\right)}\right|& =\left|\sum _{i=1}^n{W}_{ni}\left(x\right)E{\epsilon }_{3,i}\right|\\ \le \underset{1\le i\le n}{\text{max}}\left|W_{ni}\left(x\right)\right|\sum _{i=1}^{n}E\left[\left|{\epsilon }_i\right|I\left(\left|{\epsilon }_i\right|>i^{1/4}\right)\right]\\ \le c{n}^{-1/2}{\text{log}}^{-3/2}n\sum _{i=1}^n{i}^{-3/4}E\left[{\epsilon }_i^{4}I\left(\left|{\epsilon }_i\right|>i^{1/4}\right)\right]\\ =O\left(n^{-1/4}{\text{log}}^{-3/2}n\right).\end{array}$$

(4.18)

Finally, similar to the proof of (2.3), by (4.14)-(4.18), it easily has that |g _n (x) -Eg _n (x)| = O(n^-1/4), a.s..