 Research
 Open Access
 Published:
The efficiency comparisons between OLSE and BLUE in a singular linear model
Journal of Inequalities and Applications volume 2013, Article number: 17 (2013)
Abstract
This paper is mainly concerned with the efficiency comparison between OLSE and BLUE in a singular linear model. We define the efficiencies between OLSE and BLUE by means of the matrix Euclidean norm and prove a matrix Euclidean norm version of the Kantorovich inequality to limit upper or lower bounds of these efficiencies. It relaxes the assumptions that the covariance matrix is positive definite and the design matrix has full column rank.
MSC:62J05, 62H05, 62H20.
1 Introduction
Inequalities are studied and utilized widely in many fields such as in matrix theory, statistics and so on. In statistics, they are often used to make efficiency comparisons between two estimators. For example, Wang and Shao [1] have discussed the efficiency comparisons between the ordinary least squares estimator (OLSE) and the best linear unbiased estimator (BLUE) in linear models. In this paper, our goal is to make the comparison of efficiencies between OLSE and BLUE in a singular linear model by using matrix norm versions of the Kantorovich inequality involving a nonnegative definite matrix.
Consider the following linear regression model:
where y\in {R}^{n} is the vector of n observations, X\in {R}^{n\times p} is the known design matrix, \beta \in {R}^{P} is the unknown vector of regression coefficients and \epsilon \in {R}^{n} is the error vector with mean vector zero and the covariance matrix Σ.
When X has full column rank and Σ is assumed to be positive definite, it is well known that the best linear unbiased estimator (BLUE) of β can be expressed as
and the ordinary least squares estimator (OLSE) of β is given by
According to the Löwner ordering, we can easily compute from (1.2) and (1.3) that cov(\stackrel{\u02c6}{\beta})cov(\tilde{\beta})\ge 0, which is nonnegative definite. Since there is no unique way to measure how ‘bad’ the OLSE can be with respect to the BLUE, various criteria have been considered in the literature; see, e.g., [2–6]. Among these criteria, the frequently used measure is the Watson efficiency [2] defined as follows:
where \cdot  indicates the determinant of the matrix concerned. The lower bound is provided by the BloomfildWatsonKnott inequality; see, e.g., [7, 8]. However, Yang and Wang [9] have shown that such a criterion is not always so satisfactory and provided an alternative form defined as the ratio of the Euclidean norms (or Frobenius norms) of the corresponding covariance matrices:
Many authors assume that the covariance matrix is nonsingular in their analysis of this classic linear model. But the number of characteristics that could be included in the model may be clearly limited by this assumption of nonsingularity. A few authors relax the condition of nonsingularity and consider a singular linear model. For example, Liski et al. [10] and Liu [4] make efficiency comparisons between the OLSE and BLUE in a singular linear model. In the present paper, the singular linear model is further studied.
The Watson efficiency {\varphi}_{1} has been generalized to a weakly singular model; see, e.g., [11]. For a general case of the underlying singular linear model, it is not interesting because the denominator reduces to zero. In order to relax assumptions on the rank of X and Σ, we mainly discuss its alternative form based on the Euclidean norm [9, 12].
We hereinafter introduce some useful notations. Let the symbols {A}^{\prime}, {A}^{}, {A}^{+}, \mathcal{R}(A), \mathcal{R}{(A)}^{\perp} and rk(A) stand for the transpose, a generalized inverse, the MoorePenrose inverse, the column space, the orthogonal complement of the column space and the rank of the matrix A, respectively. Moreover, write {P}_{A}=A{A}^{+}=A{({A}^{\prime}A)}^{+}{A}^{\prime} and {M}_{A}=I{P}_{A}, in particular, H={P}_{X}, M=IH. {\lambda}_{i}(A) denotes the i th largest eigenvalue of the matrix A.
2 A new Kantorovichtype inequality
We start with some lemmas which are very useful in the following.
Lemma 2.1 Let A be an n\times n complex matrix and {\lambda}_{1},\dots ,{\lambda}_{n} be eigenvalues of A. Then we have
and the equality holds if and only if A is a regular matrix.
Proof The proof is very easy, we therefore omit it here. □
Lemma 2.2 Let A be an n\times n positive semidefinite Hermitian matrix and U be an orthogonal projection matrix with rk(U)=k. Then we have
The proof can be found in [13]. See also [14].
Lemma 2.3 The Pólya and Szegö inequality
where 0<{m}_{1}\le {a}_{i}\le {M}_{1}, 0<{m}_{2}\le {b}_{i}\le {M}_{2}, i=1,\dots ,n.
Lemma 2.4 Let A and B be two n\times n positive semidefinite Hermitian matrices, and U be an n\times k matrix, \mathcal{R}(A)\subset \mathcal{R}(B), rk(B)=q, rk(BU)=t, then
The proof can be found in [15]. See also [14].
Theorem 2.5 Let A be an n\times n positive semidefinite Hermitian matrix and {\lambda}_{1}\ge \cdots \ge {\lambda}_{s}>0 (s\le n) be the ordered eigenvalues of A, and let U be an n\times p complex matrix such that {U}^{\ast}U={I}_{p}. If p\le s, we then have
Proof The proof is similar to Theorem 1 in [9], we therefore omit it here. □
3 The comparison of efficiencies
The Watson efficiency [2, 16] and its decompositions [11] are usually used to measure the efficiency of the ordinary least squares. However, Yang and Wang [9] show that such a criterion does not always work well in some cases and propose an alternative form
The above formula and its lower bound both require the covariance matrix Σ to be positive definite and the design matrix to have full column rank. This assumption limits clearly the number of characters which may be included in the model. We here generalize this formula to the situation where the matrices X and Σ can be of arbitrary rank.
In the following, we divide singular linear models into three categories in accordance with the assumptions on X and Σ. These categories are as follows:

(1)
\mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma}), rk(X)=p, Σ is possibly singular;

(2)
\mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma}), rk(X)<p, Σ is possibly singular;

(3)
Σ is possibly singular.
From now on, we always assume rk(\mathrm{\Sigma})=s (s<n). Then any given singular linear model can be uniquely classified into i (i=1,2,3). Many authors have contributed to the theory in the literature; see, e.g., [17, 18]. The general representations for the BLUE of Xβ and their covariance matrices can be given respectively by
where W=\mathrm{\Sigma}+XU{X}^{\prime} and here U\ge 0 is an arbitrary matrix such that \mathcal{R}(W)=\mathcal{R}(X:\mathrm{\Sigma}). In particular, {\mathrm{\Sigma}}^{} can play the same role as {\mathrm{\Sigma}}^{1} does when Σ is nonsingular as long as \mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma}). That is to say, {W}^{} can be replaced by {\mathrm{\Sigma}}^{} in this case. We then have
The covariance matrix of the wellknown OLSE of Xβ is given by
In the following, we make efficiency comparisons between the OLSE and BLUE in a singular model according to the above category.
Firstly, we will discuss the category (1). The matrix product {X}^{\prime}{\mathrm{\Sigma}}^{}X is invariant for all the choices of generalized inverse {\mathrm{\Sigma}}^{} because of the column space inclusion \mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma}). Applying the rank rule of the matrix product [19], we can get that
Note that \mathcal{R}(\mathrm{\Sigma})=\mathcal{R}({\mathrm{\Sigma}}^{+}), and then we can conclude that {X}^{\prime}\mathrm{\Sigma}X and {X}^{\prime}{\mathrm{\Sigma}}^{+}X are both nonsingular. In the literature, such a model is often regarded as a weakly singular model or the ZyskindMartin model [20]. In this model, the relative efficiency ρ becomes
It is easy to prove that {\rho}_{1}\le 1. The following theorem gives its lower bound.
Theorem 3.1 In the linear regression model (1.1), let {\lambda}_{1}\ge \cdots \ge {\lambda}_{s} (s<n) be the ordered eigenvalues of Σ and X be an n\times p design matrix with rk(X)=p, \mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma}). Then we have
Proof We can firstly compute that
There exists some orthogonal matrix P such that \mathrm{\Sigma}=P\mathrm{\Lambda}{P}^{\prime}, so , where \mathrm{\Lambda}=diag({\lambda}_{1},\dots ,{\lambda}_{s},0,\dots ,0). Let , and then we have that and
Using Theorem 2.5, the result in Theorem 3.1 can be established. □
Secondly, we will consider the category (2). Let rk(X)=r (r<p). Using equation (3.6), we can get that rk({X}^{\prime}\mathrm{\Sigma}X)=rk({X}^{\prime}{\mathrm{\Sigma}}^{+}X)=r. Analogically, the relative efficiency ρ becomes
It is easy to prove that {\rho}_{2}\le 1. The following theorem gives its lower bound.
Theorem 3.2 In the linear regression model (1.1), let {\lambda}_{1}\ge \cdots \ge {\lambda}_{s} (s<n) be the ordered eigenvalues of Σ and X be an n\times p design matrix with rk(X)=r (r<p), \mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma}). We then have
Proof It is easy to prove that
Then the proof is similar to Theorem 3.1, therefore we omit it here. □
Finally, we take into account the category (3). Owing to (3.3), we may write
Therefore, we define
Let dim\mathcal{R}(X)\cap \mathcal{R}(\mathrm{\Sigma})=g (g\ge 0) and rk(X)=r (r\le p). Note that
Due to equation (3.6), we have
As a result from
we then have
Similarly, we can obtain that
In view of \mathcal{R}(M)=\mathcal{R}{(X)}^{\perp} and
we can get that
Theorem 3.3 In the linear regression model (1.1), let {\lambda}_{1}\ge \cdots \ge {\lambda}_{s} (s<n) be the ordered eigenvalues of Σ and X be an n\times p design matrix with rk(X)=r (r\le p), dim\mathcal{R}(X)\cap \mathcal{R}(\mathrm{\Sigma})=g, rk(H\mathrm{\Sigma}M{(M\mathrm{\Sigma}M)}^{}M\mathrm{\Sigma}H)=h, we then have
Proof For convenience, let a=\parallel H\mathrm{\Sigma}M{(M\mathrm{\Sigma}M)}^{}M\mathrm{\Sigma}H\parallel and b=\parallel H\mathrm{\Sigma}H\parallel. Then H\mathrm{\Sigma}M{(M\mathrm{\Sigma}M)}^{}M\mathrm{\Sigma}H is invariant for all the choices of generalized inverses {(M\mathrm{\Sigma}M)}^{}. From Lemma 2.1, we can easily get that
Obviously, h=rk(H\mathrm{\Sigma}M{(M\mathrm{\Sigma}M)}^{}M\mathrm{\Sigma}H)\le rk(\mathrm{\Sigma}H)=g and h\le rk(\mathrm{\Sigma}M)=sg. Since Σ and \mathrm{\Sigma}H\mathrm{\Sigma} are positive semidefinite matrices and \mathcal{R}(\mathrm{\Sigma}H\mathrm{\Sigma})\subset \mathcal{R}(\mathrm{\Sigma}), we can derive from Lemma 2.4 that
Here H is an orthogonal projection matrix, and then we obtain from Lemma 2.2 that
Furthermore, since H\mathrm{\Sigma}H is a Hermitian matrix, by Lemma 2.1, we can get that
The Sylvester theorem (see, e.g., [14]) shows that nr+g>s. Analogically, we can get that
Applying the wellknown arithmeticharmonic mean inequality, we have
Firstly, we suppose that h\le s+rn. We can then compute that
By the Pólya and Szegö inequality and a nontrivial but elementary combinational argument, we can establish the first inequality. In fact, the second inequality is similar. □
4 Conclusions
In this article, we use several new matrix norm versions of the Kantorovich inequality involving a nonnegative definite matrix to make the comparison of efficiencies between OLSE and BLUE in a singular linear model. The singular linear model is divided into three categories in accordance with the assumptions on the ranks of X and Σ. We introduce some new relative efficiency criteria and their lower or upper bounds are given based on matrix norm inequalities in Theorem 3.1, Theorem 3.2 and Theorem 3.3.
References
Wang SG, Shao J: Constrained Kantorovich inequality and relative efficiency of least squares. J. Multivar. Anal. 1992, 42: 284–298. 10.1016/0047259X(92)90048K
Watson, GS: Serial correlation in regression analysis. PhD thesis, Department of Experimental Statistics, North Carolina State College, Raleigh (1951)
Khatri CG, Rao CR: Some generalizations of the Kantorovich inequality. Sankhya 1982, 44: 91–102.
Liu SZ: Efficiency comparisons between the OLSE and the BLUE in a singular linear model. J. Stat. Plan. Inference 2000, 84: 191–200. 10.1016/S03783758(99)001494
Rao CR: The inefficiency of least squares: extension of the Kantorovich inequality. Linear Algebra Appl. 1985, 70: 249–255.
Yang H: Extensions of the Kantorovich inequality and the error ratio efficiency of the mean square. Math. Appl. 1988, 4: 85–90.
Bloomfield P, Watson GS: The inefficiency of least squares. Biometrika 1975, 62: 121–128. 10.1093/biomet/62.1.121
Knott M: On the minimum efficiency of the least square. Biometrika 1975, 62: 129–132. 10.1093/biomet/62.1.129
Yang H, Wang LT: An alternative form of the Watson efficiency. J. Stat. Plan. Inference 2009, 139: 2767–2774. 10.1016/j.jspi.2009.01.002
Liski EP, Puntanen S, Wang SG: Bounds for the trace of the difference of the covariance matrices of the OLSE and BLUE. Linear Algebra Appl. 1992, 176: 121–130.
Chu K, Isotalo J, Puntanen S, Styan GPH: On decomposing the Watson efficiency of ordinary least squares in a partitioned weakly singular linear model. Sankhya 2004, 66: 634–651.
Wang LT, Yang H: Several matrix Euclidean norm inequalities involving Kantorovich inequality. J. Inequal. Appl. 2009., 2009: Article ID 291984
Poincare H: Sur les equation aux derives partielles de la physique mathematique. Am. J. Math. 1890, 12: 211–294. 10.2307/2369620
Wang SG, Jia ZZ: Matrix Inequality. Anhui Education Press, Hefei; 1994.
Scott AJ, Styan GPH: On a separation theorem for generalized eigenvalues and a problem in the analysis of sample surveys. Linear Algebra Appl. 1985, 70: 209–224.
Chu K, Isotalo J, Puntanen S, Styan GPH: The efficiency factorization multiplier for the Watson efficiency in partitioned linear models: some examples and a literature review. J. Stat. Plan. Inference 2007, 137: 3336–3351. 10.1016/j.jspi.2007.03.015
Christensen R: Plane Answers to Complex Questions: The Theory of Linear Models. Springer, New York; 1987.
Groß J: The general GaussMarkov model with possibly singular dispersion matrix. Stat. Pap. 2004, 45: 311–336. 10.1007/BF02777575
Meyer CD: Matrix Analysis and Applied Linear Algebra. Cambridge University Press, Cambridge; 2001.
Zyskind G, Martin FB: On best linear estimation and a general GaussMarkov theorem in linear models with arbitrary nonnegative covariance structure. SIAM J. Appl. Math. 1969, 17: 1190–1202. 10.1137/0117110
Acknowledgements
The authors thank very much associate editors and referees for their insightful comments that led to improving the presentation. This work is partly supported by the National Natural Science Foundation of China (No. 11126211; No. 61201398) and the Natural Science Foundation of Zhejiang Province (No. LQ12A01021).
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
LW carried out the singular model study and proved several inequalities. GP participated in the proof of the inequalities. All authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Wang, L., Pan, G. The efficiency comparisons between OLSE and BLUE in a singular linear model. J Inequal Appl 2013, 17 (2013). https://doi.org/10.1186/1029242X201317
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1029242X201317
Keywords
 Euclidean norm
 least squares
 singular linear model
 Watson efficiency