Asymptotic properties of wavelet-based estimator in nonparametric regression model with weakly dependent processes
© Zhou et al.; licensee Springer 2013
Received: 2 April 2013
Accepted: 8 May 2013
Published: 23 May 2013
In this paper, we consider a nonparametric regression model with replicated observations based on the φ-mixing and the ρ-mixing error’s structures respectively, for exhibiting dependence among the units. The wavelet procedures are developed to estimate the regression function. Under suitable conditions, we obtain expansions for the bias and the variance of wavelet estimator, prove the moment consistency, the strong consistency, the strong convergence rate of it, and establish the asymptotic normality of wavelet estimator.
from a discrete set of observations of the process at the points , is a zero mean stochastic process, defined on a probability space , and is an unknown function defined on a closed interval .
It is well known that regression model has a wide range of applications in filtering and prediction in communications and control systems, pattern recognition, classification and econometrics, and is an important tool of data analysis. Of much interest about the problem have been the weighted function estimates of g; see, for example, Priestley and Chao , Gasser and Müller [2, 3], Prakasa Rao , Clark  and the references therein for the independent case; Roussas , Fan , Roussas and Tran , Liang and Jing , Yang and Li , Yang  for the various dependent cases.
where the ’s are fixed with , and the ’s are zero mean random errors. The model can be applied to many fields. For instance, in hydrology many phenomena may be represented by a sequence of continuous responses , where represents the time elapsed from the beginning of a certain year, j indicates the corresponding year, is the measurement of the deviation from the annual mean ; in some biological phenomena as the growth of individual (or populations) will be the growth points of the j-individual, the measurement of the deviation from the mean growth of the response of the j-individual, and the points where measurements are taken . It is clear that the observations () made on the same experimental unit will in general be correlated. Hart and Wehrly  studied the asymptotic square mean error of a kernel estimator in the model with zero mean errors ’s be correlated, that is, for and zero elsewhere where is a correlation function. The uncorrelated assumption between units is unrealistic. In practice, sometimes the observed responses in different units are also correlated, more precisely a sequence of responses curve has an intrinsic dependence structure, such as mixing conditions. Under a weak error’s structure among units, Fraiman and Iribarren  proposed nonparametric estimates of in model (1.2) based on locally weighted averages, and gave their local and global asymptotic behaviors.
In the paper, we develop wavelet methods to estimate a regression function in the model (1.2) with the φ-mixing and ρ-mixing error’s structures respectively, that is, is a φ-mixing or ρ-mixing process. One motivation for using wavelet for the model (1.2) is that most of the nonparametric analyses of regression models are based on an important assumption that the regression function is smooth; but in reality, the assumption may not be satisfied. The major advantage of the wavelet is that the hypotheses of degrees of smoothness of the underlying function are less restrictive. Up to now, there have been no results on wavelet estimation for model (1.2). Another motivation for considering the model (1.2) with weakly dependent processes is that our interest is to avoid as far as possible any assumptions on the error’s structure within for each j (), and in the meantime to exhibit weakly dependence among the units.
For a systematic discussion of wavelets and their applications in statistics, see the recent monographs by Härdle et al.  and Vidakovic . Due to their ability to adapt to local features of unknown curves, many authors have applied wavelet procedures to estimate the general nonparametric model. See recent works, for example, Antoniadis et al.  and Xue  on independent errors; Johnstone and Silverman  for correlated noise; Liang et al.  on martingale difference errors; Li and Xiao  for long memory data; Li et al.  on associated samples; Xue , Sun and Chai , Li and Guo  and Liang  on mixing error assumptions.
For dealing with weakly dependent data, bootstrap and blockwise are well known. They are useful techniques of resampling, which can preserve the dependent properties of the data by appropriately choosing blocks of data. They have been sufficiently investigated by many papers, for example, Bühlman and Künsch , Yuichi , Lahiri , Lin and Zhang [29, 30] and Lin et al. . For the nonparametric regression model without repeated observations under weakly dependent processes, Lin and Zhang  respectively adopted bootstrap wavelet and blockwise bootstrap wavelet to generate an independent blockwise sample from the original dependent data, defined the wavelet estimators of , and then took advantage of the independence of the blockwise sample to prove some asymptotic properties of the wavelet estimators. In addition, the weakly dependent conditions and in Lin and Zhang  are weaker than the conditions for the consistency and asymptotic normality of ρ-mixing and φ-mixing dependent sequences. In our paper, we consider the nonparametric regression model with repeated observations under the specific ρ-mixing and φ-mixing dependent processes. We do not use bootstrap and/or blockwise to define wavelet estimator of , but construct it by the following simple formula (2.2), and can show directly some asymptotic behaviors of the wavelet estimator by applying the fundamental properties of ρ-mixing and φ-mixing sequences and some proof techniques.
Recall the definitions of the sequences of the φ-mixing and ρ-mixing random variables. Let be a sequence of random variables defined on a probability space , be σ-algebra generated by , and denote to be the set of all -measurable random variables with second moments.
The concept of a mixing sequence is central in many areas of economics, finance and other sciences. A mixing time series can be viewed as a sequence of random variables for which the past and distant future are asymptotically independent. A number of limit theorems for φ-mixing and ρ-mixing random variables have been studied by many authors. For example, see Shao , Peligrad , Utev , Kiesel , Chen et al.  and Zhou  for φ-mixing; Peligrad , Peligrad and Shao [39, 40], Shao  and Bradley  for ρ-mixing. Some limit theories can be found in the monograph of Lin and Lu .
The article is structured as follows. In Section 2, we introduce the wavelet estimation procedures and establish main results. The proofs of the main results are provided in Section 3.
2 Estimators and main results
where . Expressing the model in this way is useful since the problem of estimating may now be regarded as that of fitting a curve through the sample means . The wavelet technique is applied to estimate the regression function in model (1.2). The detailed procedure is summarized below.
In the sequel, let denote generic finite positive constants, whose values are unimportant and may change from line to line. Set and suppose that for and .
Before formulating the main results, we first give some assumptions, which are quite mild and can be easily satisfied.
is a sequence of φ-mixing (ρ-mixing);
is a sequence of identically distributed φ-mixing (ρ-mixing);
is a sequence of strictly stationary φ-mixing (ρ-mixing).
for some ;
() for some .
A4. , , where is the Sobolev space of order υ, i.e., if , then , where denotes the Fourier transform of h.
A5. satisfies the Lipschitz condition of order .
A6. is q regular with , satisfies the Lipschitz condition with order 1 and has a compact support, and satisfies as , where is the Fourier transform of .
A8. , as , and , as , such that , and , where υ and γ are defined in (A4) and (A5), respectively.
Remark 2.1 We refer to the monograph of Doukhan  for properties of φ-mixing and ρ-mixing, and more mixing conditions.
Remark 2.2 It is well known that (A4)-(A7) are the mild regularity conditions for wavelet smoothing; see Antoniadis et al. , Chai and Xu , Xue , Sun and Chai , Zhou and You  and Li and Guo .
Remark 2.3 (A8) is satisfied easily. For example, for any , and for and .
Our results are listed as follows.
, where is defined in Lemma 3.2;
where denotes convergence in distribution.
3 Proofs of the main results
In order to prove the main results, we first present several lemmas.
uniformly in , as .
It follows easily from Theorem 3.2 of Antoniadis et al. .
- (b)Let be a ρ-mixing sequence, , with and . Then
Lemmas 3.3(a) and (b) respectively come from Lemmas 10.1.d and 10.1.c of Lin and Lu .
- (i)If , then
- (ii)Suppose that there exists an array of positive numbers such that for every , . Then, for any , there exists a positive constant such that
- (b)Let be a ρ-mixing sequence with . Then, for any , there exists a positive constant such that
We are now in a position to give the proofs of the main results.
By Lemma 3.2, (a) holds.
Therefore, (b) holds for φ-mixing. Similar to the arguments, we obtain (b) for ρ-mixing by Lemma 3.3(b). □
Proof of Theorem 2.2 We know that . It follows easily from Theorem 2.1. □
φ-mixing. Here, we consider under two different assumptions: [(A1)(ii) and (A2)(i)(ii)] and [(A1)(i) and (A2)(iii)], respectively.
From (3.6), (3.7), (3.9) and (3.10), we obtain (3.5).
ρ-mixing. We also consider under two different assumptions: [(A1)(ii) and (A2)(i)(ii)] and [(A1)(i) and (A2)(iii)], respectively.
Note that as , hence for . Further, for any and .
Therefore, (3.7) holds.
Thus, we obtain (3.5).
So, we complete the proof of Theorem 2.3. □
by (3.11)-(3.13) and (3.15)-(3.16).
- (1)φ-mixing. Taking , we have , , and . By Lemma 3.4(a), we have
- (2)ρ-mixing. Similar to the arguments in the proof of Theorem 2.3, for any and . Taking , we have and . Next, take small enough such that . By Lemma 3.4(b), we have
Therefore, we also obtain (3.17).
So, we complete the proof of Theorem 2.4. □
which can be made arbitrarily small if we first choose ς such that is small, and for this ς we choose m large enough so that makes arbitrarily small. Thus, we obtain (3.18).
since by Lemma 3.6, by (3.21), and by Lemma 3.1.
Thus, we complete the proof of Theorem 2.5. □
4 Conclusion and discussion
The paper studies a nonparametric regression model with replicated observations under weakly dependent processes by wavelet procedures. For exhibiting dependence among the units, we assume that is a φ-mixing or ρ-mixing process, and avoid as far as possible any assumptions on the error’s structure among for each j (). Under suitable conditions, we obtain expansions for the bias and the variance of wavelet estimator, prove the moment consistency, the strong consistency, the strong convergence rate of it, and establish the asymptotic normality of wavelet estimator. For the general nonparametric model, consistency results of linear wavelet estimator can be derived from general results on regression estimators in the case of dependent errors. But our results cannot be derived directly from general results on regression estimators because the nonparametric regression model with repeated measurements we considered has complex dependent error’s structure. Bootstrap and blockwise are useful techniques of resampling, which can preserve the dependent properties of the data by appropriately choosing blocks of data. They have been sufficiently investigated for weakly dependent data by many papers, for example, Bühlman and Künsch , Yuichi , Lahiri , Lin and Zhang [29, 30] and Lin et al. . In the future, we may try bootstrap and blockwise methods into our model. Since linear wavelet is not adaptive, nonlinear wavelet and design-adapted wavelet have also received considerable attention recently; see, for example, Li , Liang and Uña-Álvarez  and Chesneau  for (conditional) density estimator; Li and Xiao  and Uña-Álvarez et al.  for nonparametric models; and Delouille and Sachs  for nonlinear autoregressive models, and so on. At present, our paper mainly concentrates on linear wavelet estimator in nonparametric regression model with repeated measurements under weakly dependent processes. Although it is easy to construct nonlinear wavelet estimator in our model, it is very difficult to establish asymptotic theory of nonlinear wavelet estimator and to prove it since the structure of our model is complex. It will be a challenging work. Interesting work for further research includes nonlinear wavelet and design-adapted wavelet estimations for our model.
The authors thank the associate editor and two anonymous referees, whose valuable comments greatly improved the paper. This work is partially supported by Key Natural Science Foundation of Higher Education Institutions of Anhui Province (KJ2012A270), NSFC (11171065, 11061002), Youth Foundation for Humanities and Social Sciences Project from Ministry of Education of China (11YJC790311), NSFJS (BK2011058), National Natural Science Foundation of Guangxi (2011GXNSFA018126) and Postdoctoral Research Program of Jiangsu Province of China (1202013C).
- Priestley MB, Chao MT: Nonparametric function fitting. J. R. Stat. Soc. B 1972, 4: 385–392.MathSciNetMATHGoogle Scholar
- Gasser T, Müller HG: Kernel estimation of regression function. Lecture Notes in Mathematics 757. In Smooth Techniques for Curve Estimation. Edited by: Gasser T, Rosenblatt M. Springer, Berlin; 1979:23–68.View ArticleGoogle Scholar
- Gasser T, Müller HG: Estimating regression functions and their derivatives by the kernel method. Scand. J. Stat. 1984, 11: 171–185.MathSciNetMATHGoogle Scholar
- Prakasa Rao BLS: Nonparametric Functional Estimation. Academic Press, Orlando; 1983.MATHGoogle Scholar
- Clark RM: Nonparametric estimation of a smooth regression function. J. R. Stat. Soc. B 1997, 39: 107–113.Google Scholar
- Roussas GG: Consistent regression with fixed design points under dependence condition. Stat. Probab. Lett. 1989, 8: 41–50. 10.1016/0167-7152(89)90081-3MathSciNetView ArticleMATHGoogle Scholar
- Fan Y: Consistent nonparametric multiple regression for dependent heterogeneous processes: the fixed design case. J. Multivar. Anal. 1990, 33: 72–88. 10.1016/0047-259X(90)90006-4View ArticleMathSciNetMATHGoogle Scholar
- Roussas GG, Tran LT: Fixed design regression for time series: asymptotic normality. J. Multivar. Anal. 1992, 40: 262–291. 10.1016/0047-259X(92)90026-CMathSciNetView ArticleMATHGoogle Scholar
- Liang HY, Jing BY: Asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences. J. Multivar. Anal. 2005, 95: 227–245. 10.1016/j.jmva.2004.06.004MathSciNetView ArticleMATHGoogle Scholar
- Yang SC, Li YM: Uniform asymptotic normality of the regression weighted estimator for strong mixing samples. Acta Math. Sin. 2006, 49A: 1163–1170.MATHGoogle Scholar
- Yang SC: Maximal moment inequality for partial sums of strong mixing sequences and application. Acta Math. Sin. Engl. Ser. 2007, 23: 1013–1024. 10.1007/s10114-005-0841-9MathSciNetView ArticleMATHGoogle Scholar
- Fraiman R, Iribarren GP: Nonparametric regression estimation in models with weak error’s structure. J. Multivar. Anal. 1991, 37: 180–196. 10.1016/0047-259X(91)90079-HView ArticleMATHGoogle Scholar
- Hart JD, Wehrly TE: Kernel regression estimation using repeated measurements data. J. Am. Stat. Assoc. 1986, 81: 1080–1088. 10.1080/01621459.1986.10478377MathSciNetView ArticleMATHGoogle Scholar
- Härdle W, Kerkyacharian G, Picard D, Tsybabov A: Wavelets: Approximation and Statistical Applications. Springer, New York; 1998.View ArticleMATHGoogle Scholar
- Vidakovic B: Statistical Modeling by Wavelet. Wiley, New York; 1999.View ArticleMATHGoogle Scholar
- Antoniadis A, Gregoire G, Mckeague IW: Wavelet methods for curve estimation. J. Am. Stat. Assoc. 1994, 89: 1340–1352. 10.1080/01621459.1994.10476873MathSciNetView ArticleMATHGoogle Scholar
- Xue LG: Strong uniform convergence rates of the wavelet estimator of regression function under completed and censored data. Acta Math. Appl. Sin. 2002, 25: 430–438.MathSciNetMATHGoogle Scholar
- Johnstone IM, Silverman BW: Wavelet threshold estimators for data with correlated noise. J. R. Stat. Soc. B 1997, 59: 319–351. 10.1111/1467-9868.00071MathSciNetView ArticleMATHGoogle Scholar
- Liang HY, Zhang DX, Lu BX: Wavelet estimation in nonparametric model under martingale difference errors. Appl. Math. J. Chin. Univ. Ser. B 2004, 19: 302–310. 10.1007/s11766-004-0039-4MathSciNetView ArticleMATHGoogle Scholar
- Li LY, Xiao YM: Wavelet-based estimators of mean regression function with long memory data. Appl. Math. Mech. 2006, 27: 901–910. 10.1007/s10483-006-0705-1MathSciNetView ArticleMATHGoogle Scholar
- Li YM, Yang SC, Zhou Y: Consistency and uniformly asymptotic normality of wavelet estimator in regression model with associated samples. Stat. Probab. Lett. 2008, 78: 2947–2956. 10.1016/j.spl.2008.05.004MathSciNetView ArticleMATHGoogle Scholar
- Xue LG: Uniform convergence rates of the wavelet estimator of regression function under mixing error. Acta Math. Sci. 2002, 22A: 528–535.MathSciNetMATHGoogle Scholar
- Sun Y, Chai GX: Nonparametric wavelet estimation of a fixed designed regression function. Acta Math. Sci. 2004, 24A: 597–606.MathSciNetMATHGoogle Scholar
- Li YM, Guo JH: Asymptotic normality of wavelet estimator for strong mixing errors. J. Korean Stat. Soc. 2009, 38: 383–390. 10.1016/j.jkss.2009.03.002View ArticleMathSciNetMATHGoogle Scholar
- Liang HY: Asymptotic normality of wavelet estimator in heteroscedastic model with α -mixing errors. J. Syst. Sci. Complex. 2011, 24: 725–737. 10.1007/s11424-010-8354-8MathSciNetView ArticleMATHGoogle Scholar
- Bühlman, P, Künsch, HR: The blockwise bootstrap for general parameters of a stationary time series. Research report 70, ETH Zurich (1993)Google Scholar
- Yuichi K: Empirical likelihood methods with weakly dependent processes. Ann. Stat. 1997, 25: 2084–2102. 10.1214/aos/1069362388View ArticleMathSciNetMATHGoogle Scholar
- Lahiri SH: Theoretical comparison of block bootstrap methods. Ann. Stat. 1999, 27: 386–404. 10.1214/aos/1018031117View ArticleMATHGoogle Scholar
- Lin L, Zhang RC: Blockwise empirical Euclidean likelihood for weakly dependent processes. Stat. Probab. Lett. 2001, 53: 143–152. 10.1016/S0167-7152(01)00066-9View ArticleMathSciNetMATHGoogle Scholar
- Lin L, Zhang RC: Bootstrap wavelet in the nonparametric regression model with weakly dependent processes. Acta Math. Sci. 2004, 24B: 61–70.MathSciNetMATHGoogle Scholar
- Lin L, Fan Y, Tan L: Blockwise bootstrap wavelet in nonparametric regression model with weakly dependent processes. Metrika 2008, 67: 31–48.MathSciNetView ArticleMATHGoogle Scholar
- Shao QM: A moment inequality and its application. Acta Math. Sin. 1988, 31: 736–747.MATHGoogle Scholar
- Peligrad M: The r -quick version of the strong law for stationary φ -mixing sequences. In Proceedings of the International Conference on Almost Everywhere Convergence in Probability and Statistics. Academic Press, New York; 1989:335–348.Google Scholar
- Utev SA: Sums of random variables with φ -mixing. Sib. Adv. Math. 1991, 1: 124–155.MathSciNetMATHGoogle Scholar
- Kiesel R: Summability and strong laws for φ -mixing random variables. J. Theor. Probab. 1998, 11: 209–224. 10.1023/A:1021655227120MathSciNetView ArticleMATHGoogle Scholar
- Chen PY, Hu TC, Volodin A: Limiting behaviour of moving average processes under φ -mixing assumption. Stat. Probab. Lett. 2009, 79: 105–111. 10.1016/j.spl.2008.07.026MathSciNetView ArticleMATHGoogle Scholar
- Zhou XC: Complete moment convergence of moving average processes under φ -mixing assumptions. Stat. Probab. Lett. 2010, 80: 285–292. 10.1016/j.spl.2009.10.018View ArticleMathSciNetMATHGoogle Scholar
- Peligrad M: On the central limit theorem for ρ -mixing sequences of random variables. Ann. Probab. 1987, 15: 1387–1394. 10.1214/aop/1176991983MathSciNetView ArticleMATHGoogle Scholar
- Peligrad M, Shao QM: Estimation of variance for ρ -mixing sequences. J. Multivar. Anal. 1995, 52: 140–157. 10.1006/jmva.1995.1008MathSciNetView ArticleMATHGoogle Scholar
- Peligrad M, Shao QM: A note on estimation of the variance of partial sums for ρ -mixing random variables. Stat. Probab. Lett. 1996, 28: 141–145.MathSciNetView ArticleMATHGoogle Scholar
- Shao QM: Maximal inequalities for partial sums of ρ -mixing sequences. Ann. Probab. 1995, 23: 948–965. 10.1214/aop/1176988297MathSciNetView ArticleMATHGoogle Scholar
- Bradley R: A stationary rho-mixing Markov chain which is not “interlaced” rho-mixing. J. Theor. Probab. 2001, 14: 717–727. 10.1023/A:1017545123473View ArticleMathSciNetMATHGoogle Scholar
- Lin ZY, Lu CR: Limit Theory for Mixing Dependent Random Variables. Science Press and K.A.P., Beijing; 1996.MATHGoogle Scholar
- Doukhan P Lecture Notes in Statistics. In Mixing: Properties and Examples. Springer, Berlin; 1994.View ArticleGoogle Scholar
- Chai GX, Xu KJ: Wavelet smoothing in semiparametric regression model. Chinese J. Appl. Probab. Statist. 1999, 15: 97–105.MathSciNetMATHGoogle Scholar
- Zhou X, You JH: Wavelet estimation in varying-coefficient partially linear regression models. Stat. Probab. Lett. 2004, 68: 91–104. 10.1016/j.spl.2004.01.018MathSciNetView ArticleMATHGoogle Scholar
- Walter GG: Wavelets and Orthogonal Systems with Applications. CRC Press, Boca Raton; 1994.MATHGoogle Scholar
- Ibragimov IA: Some limit theorems for stationary processes. Theory Probab. Appl. 1962, 7: 349–382. 10.1137/1107036View ArticleMathSciNetMATHGoogle Scholar
- Ibragimov IA: A note on the central limit theorem for dependent random variables. Theory Probab. Appl. 1975, 20: 134–139.Google Scholar
- Li LY: Non-linear wavelet-based density estimators under random censorship. J. Stat. Plan. Inference 2003, 117: 35–58. 10.1016/S0378-3758(02)00366-XView ArticleMathSciNetMATHGoogle Scholar
- Liang HY, Uña-Álvarez J: Wavelet estimation of conditional density with truncated, censored and dependent data. J. Multivar. Anal. 2011, 102: 448–467. 10.1016/j.jmva.2010.10.004View ArticleMathSciNetMATHGoogle Scholar
- Chesneau C: On the adaptive wavelet deconvolution of a density for strong mixing sequences. J. Korean Stat. Soc. 2012, 41: 423–436. 10.1016/j.jkss.2012.01.005MathSciNetView ArticleMATHGoogle Scholar
- Uña-Álvarez J, Liang HY, Rodríguez-Casal A: Nonlinear wavelet estimator of the regression function under left-truncated dependent data. J. Nonparametr. Stat. 2010, 22: 319–344. 10.1080/10485250903469736MathSciNetView ArticleMATHGoogle Scholar
- Delouille V, Sachs RV: Estimation of nonlinear autoregressive models using design-adapted wavelets. Ann. Inst. Stat. Math. 2005, 57: 235–253. 10.1007/BF02507024View ArticleMathSciNetMATHGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.