- Open Access
Maximum likelihood estimators in linear regression models with Ornstein-Uhlenbeck process
Journal of Inequalities and Applications volume 2014, Article number: 301 (2014)
The paper studies the linear regression model
with parameters , and the standard Brownian motion. Firstly, the maximum likelihood (ML) estimators of β, λ and are given. Secondly, under general conditions, the asymptotic properties of the ML estimators are investigated. And then, limiting distributions for likelihood ratio test statistics of the hypothesis are also given. Lastly, the validity of the method are illuminated by two real examples.
MSC:62J05, 62M10, 60J60.
Consider the following linear regression model
where ’s are scalar response variables, ’s are explanatory variables, β is an m-dimensional unknown parameter, and is an Ornstein-Uhlenbeck process, which satisfies the linear stochastic differential equation (SDE)
with parameters , and the standard Brownian motion.
It is well known that a linear regression model is the most important and popular model in the statistical literature, which attracts many people to investigate the model. For an ordinary linear regression model (when the errors are independent and identically distributed (i.i.d.) random variables), Wang and Zhou , Anatolyev , Bai and Guo , Chen , Gil et al. , Hampel et al. , Cui , Durbin  and Li and Yang  used various estimation methods to obtain estimators of the unknown parameters in (1.1) and discussed some large or small sample properties of these estimators. Recently, linear regression with serially correlated errors has attracted increasing attention from statisticians and economists. One case of considerable interest is that the errors are autoregressive processes; Hu , Wu , and Fox and Taqqu  established its asymptotic normality with the usual -normalization in the case of long memory stationary Gaussian observations errors. Giraitis and Surgailis  extended this result to non-Gaussian linear sequences. Koul and Surgailis  established the asymptotic normality of the Whittle estimator in linear regression models with non-Gaussian long memory moving average errors. Shiohama and Taniguchi  estimated the regression parameters in a linear regression model with autoregressive process. Fan  investigated moderate deviations for M-estimators in linear models with ϕ-mixing errors.
The Ornstein-Uhlenbeck process was originally introduced by Ornstein and Uhlenbeck  as a model for particle motion in a fluid. In physical sciences, the Ornstein-Uhlenbeck process is a prototype of a noisy relaxation process, whose probability density function can be described by the Fokker-Planck equation (see Janczura et al. , Debbasch et al. , Gillespie , Ditlevsen and Lansky , Garbaczewski and Olkiewicz , Plastino and Plastino ):
This process is now widely used in many areas of application. The main characteristic of the Ornstein-Uhlenbeck process is the tendency to return towards the long-term equilibrium μ. This property, known as mean-reversion, is found in many real life processes, e.g., in commodity and energy price processes (see Fasen , Yu , Geman ). There are a number of papers concerned with the Ornstein-Uhlenbeck process, for example, Janczura et al. , Zhang et al. , Rieder , Iacus , Bishwal , Shimizu , Zhang and Zhang , Chronopoulou and Viens , Lin and Wang  and Xiao et al. . It is well known that the solution of model (1.2) is an autoregressive process. For a constant or functional or random coefficient autoregressive model, many people (for example, Magdalinos , Andrews and Guggenberger , Fan and Yao , Berk , Goldenshluger and Zeevi , Liebscher , Baran et al. , Distaso  and Harvill and Ray ) used various estimation methods to obtain estimators and discussed some asymptotic properties of these estimators, or investigated hypotheses testing.
By (1.1) and (1.2), we can obtain that the more general process satisfies the SDE
where is a time-dependent mean reversion level with three parameters. Thus, model (1.3) is a general Ornstein-Uhlenbeck process. Its special cases have gained much attention and have been applied to many fields such as economics, physics, geography, geology, biology and agriculture. Dehling et al.  considered the model with maximum likelihood estimate, and proved strong consistency and asymptotic normality. Lin and Wang  established the existence of a successful coupling for a class of stochastic differential equations given by (1.3). Bishwal  investigated the uniform rate of weak convergence of the minimum contrast estimator in the Ornstein-Uhlenbeck process (1.3).
The solution of model (1.2) is given by
The process observed in discrete time is more relevant in statistics and economics. Therefore, by (1.4), the Ornstein-Uhlenbeck time series for is given by
where i.i.d. random errors and with equidistant time lag d, fixed in advance. Models (1.1) and (1.5) include many special cases such as a linear regression model with constant coefficient autoregressive processes (when ; see Hu , Wu , Maller , Pere  and Fuller ), Ornstein-Uhlenbeck time series or processes (when ; see Rieder , Iacus , Bishwal , Shimizu  and Zhang and Zhang ), constant coefficient autoregressive processes (when , ; see Chambers , Hamilton , Brockwell and Davis  and Abadir and Lucas , etc.).
The paper discusses models (1.1) and (1.5). The organization of the paper is as follows. In Section 2 some estimators of β, θ and are given by the quasi-maximum likelihood method. Under general conditions, the existence and consistency of the quasi-maximum likelihood estimators as well as asymptotic normality are investigated in Section 3. The hypothesis testing is given in Section 4. Some preliminary lemmas are presented in Section 5. The main proofs of theorems are presented in Section 6, with two real examples in Section 7.
2 Estimation method
Without of loss generality, we assume that , in the sequel. Write the ‘true’ model as
By (2.2), we have
Thus is measurable with respect to the σ-field H generated by , and
We maximize (2.5) to obtain QML estimators denoted by , , (when they exist). Then the first derivatives of may be written as
Thus , , satisfy the following estimation equations:
To obtain our results, the following conditions are sufficient (see Maller ).
(A1) is positive definite for sufficiently large n and
where , denotes the maximum in absolute value of the eigenvalues of a symmetric matrix.
For ease of exposition, we shall introduce the following notations which will be used later in the paper.
Let -vector . Define
By (2.7) and (2.8), we get the components of
Hence we have
where the ∗ indicates that the elements are filled in by symmetry. By (2.18), we have
3 Large sample properties of the estimators
Theorem 3.1 Suppose that conditions (A1)-(A2) hold. Then there is a sequence such that, for each , as , the probability
where, for each , and , define neighborhoods
Theorem 3.2 Suppose that conditions (A1)-(A2) hold. Then
In the following, we will investigate some special cases in models (1.1) and (1.5). From Theorem 3.1 and Theorem 3.2, we obtain the following results. Here we omit their proofs.
Corollary 3.1 If , then
Corollary 3.2 If , then
4 Hypothesis testing
In order to fit a data set , we may use model (1.3) or an Ornstein-Uhlenbeck process with a constant mean level model
If , then we use model (1.3), namely models (1.1) and (1.2). If , then we use model (1.4). How to know or ? In the section, we shall consider the question about hypothesis testing and obtain limiting distributions for likelihood ratio (LR) test statistics (see Fan and Jiang ).
Under the null hypothesis
let , , be the corresponding ML estimators of β, λ, . Also let
By (2.9) and (2.5), we have that
By (4.5) and (4.6), we have
Large values of suggest rejection of the null hypothesis.
Theorem 4.1 Suppose that conditions (A1)-(A2) hold. If holds, then
5 Some lemmas
Throughout this paper, let C denote a generic positive constant which could take different value at each occurrence. To prove our main results, we first introduce the following lemmas.
Lemma 5.1 If condition (A1) holds, then for any the matrix is positive definite for large enough n, and
Proof Let and be the smallest and largest roots of . Then from Ex. 22.1 of Rao ,
for unit vectors u. Thus by (2.18) there are some and such that implies
By (2.16) and (5.1), we have
By Rao [, p.60] and (2.17), we have
From (5.3) and ,
Lemma 5.2 The matrix is positive definite for large enough n, and .
Proof Note that is positive definite and . It is easy to show that the matrix is positive definite for large enough n. By (2.8), we have
Note that and are independent, so we have . Thus, by (2.7) and , we have
Hence, from (5.5) and (5.6),
By (2.8) and (2.20), we have
Note that is a martingale difference sequence with
By (2.7), (2.8), and noting that and are independent, we have
From (5.8)-(5.10), it follows that . The proof is completed. □
Lemma 5.3 (Maller )
Let be a symmetric random matrix with eigenvalues , . Then
Lemma 5.4 For each ,
Proof Let be a square root decomposition of . Then
Let . Then
From (2.20), (2.21) and (5.14),
As the first step, we will show that, for each ,
In fact, note that
Let , , and let , . By the Cauchy-Schwarz inequality, Lemma 5.1 and noting , we have
Similar to the proof of , we easily obtain
By the Cauchy-Schwarz inequality, Lemma 5.1 and noting , we have
Hence, (5.23) follows from (5.24)-(5.27).
For the second step, we will show that
For and each , we have
By (5.32) and Lemma 5.1, we have
Using the Cauchy-Schwarz inequality and (5.33), we obtain
Using a similar argument as , we obtain that
By the Cauchy-Schwarz inequality and (5.33), (5.25), we get
By (5.25), we have
Thus, by the Chebychev inequality and (5.37),
By Lemma 5.1 and (2.3), we have
Thus, by the Chebychev inequality and (5.39),
Using a similar argument as , we obtain
Thus (5.28) follows immediately from (5.31), (5.34)-(5.36), (5.38), (5.40) and (5.41).
For the third step, we will show that
By (3.3) and (3.4), we obtain that
By (5.29), we have
By (5.32), it is easy to show that
By Lemma 5.1, (2.3) and (5.32), we have
Thus by the Chebychev inequality and (5.48),
so we have
Thus, by (5.46), (5.47), (5.49) and (5.52), we have
By (5.29), we have
It is easy to show that
Note that is a martingale difference sequence, so we have
By (5.54)-(5.56), we have
It is easily proved that
Hence, (5.42) follows immediately from (5.43)-(5.45), (5.53), (5.57) and (5.58). This completes the proof of (5.11) from (5.17), (5.23), (5.28) and (5.42).
It is well known that as . To prove (5.12), we need to show that
This follows immediately from (2.20) and the Markov inequality.
Finally, we will prove (5.13). By (5.11) and (5.12), we have
uniformly in for each . Thus, by Lemma 5.3,
This implies (5.13). □
Lemma 5.5 (Hall and Heyde )
Let be a zero-mean, square-integrable martingale array with differences , and let be an a.s. finite random variable. Suppose that for all , and . Then
where the r.v. Z has the characteristic function .
6 Proof of theorems
Proof of Theorem 3.1 Take , let
be the boundary of , and let . Using (2.19) and the Taylor expansion, for each , we have
where for some .
Let and . Take and , and by (6.2), we obtain that
By Lemma 5.2 and the Chebychev inequality, we obtain
Let , then , and using (5.13), we have
By (6.3)-(6.5), we have
By Lemma 5.3, as . Hence . Moreover, from (5.13), we have
This implies that is concave on . Noting this fact and (6.6), we get
On the event in the brackets, the continuous function has a unique maximum in θ over the compact neighborhood . Hence
Moreover, there is a sequence such that satisfies
This is a QML estimator for . It is clearly consistent, and
Since are ML estimators for , is an ML estimator for from (2.9).
To complete the proof, we will show that as . If , then and .
By (2.12) and (2.1), we have
By (2.9), (2.11) and (6.8), we have
From (6.8), it follows that
From (2.2), we get
By (6.9)-(6.11), we have