- Research
- Open Access

# The efficiency comparisons between OLSE and BLUE in a singular linear model

- Litong Wang
^{1}Email author and - Guobing Pan
^{2}

**2013**:17

https://doi.org/10.1186/1029-242X-2013-17

© Wang and Pan; licensee Springer 2013

**Received:**12 August 2012**Accepted:**21 December 2012**Published:**14 January 2013

## Abstract

This paper is mainly concerned with the efficiency comparison between OLSE and BLUE in a singular linear model. We define the efficiencies between OLSE and BLUE by means of the matrix Euclidean norm and prove a matrix Euclidean norm version of the Kantorovich inequality to limit upper or lower bounds of these efficiencies. It relaxes the assumptions that the covariance matrix is positive definite and the design matrix has full column rank.

**MSC:**62J05, 62H05, 62H20.

## Keywords

- Euclidean norm
- least squares
- singular linear model
- Watson efficiency

## 1 Introduction

Inequalities are studied and utilized widely in many fields such as in matrix theory, statistics and so on. In statistics, they are often used to make efficiency comparisons between two estimators. For example, Wang and Shao [1] have discussed the efficiency comparisons between the ordinary least squares estimator (OLSE) and the best linear unbiased estimator (BLUE) in linear models. In this paper, our goal is to make the comparison of efficiencies between OLSE and BLUE in a singular linear model by using matrix norm versions of the Kantorovich inequality involving a nonnegative definite matrix.

where $y\in {R}^{n}$ is the vector of *n* observations, $X\in {R}^{n\times p}$ is the known design matrix, $\beta \in {R}^{P}$ is the unknown vector of regression coefficients and $\epsilon \in {R}^{n}$ is the error vector with mean vector zero and the covariance matrix Σ.

*X*has full column rank and Σ is assumed to be positive definite, it is well known that the best linear unbiased estimator (BLUE) of

*β*can be expressed as

*β*is given by

*e.g.*, [2–6]. Among these criteria, the frequently used measure is the Watson efficiency [2] defined as follows:

*e.g.*, [7, 8]. However, Yang and Wang [9] have shown that such a criterion is not always so satisfactory and provided an alternative form defined as the ratio of the Euclidean norms (or Frobenius norms) of the corresponding covariance matrices:

Many authors assume that the covariance matrix is nonsingular in their analysis of this classic linear model. But the number of characteristics that could be included in the model may be clearly limited by this assumption of nonsingularity. A few authors relax the condition of nonsingularity and consider a singular linear model. For example, Liski *et al.* [10] and Liu [4] make efficiency comparisons between the OLSE and BLUE in a singular linear model. In the present paper, the singular linear model is further studied.

The Watson efficiency ${\varphi}_{1}$ has been generalized to a weakly singular model; see, *e.g.*, [11]. For a general case of the underlying singular linear model, it is not interesting because the denominator reduces to zero. In order to relax assumptions on the rank of *X* and Σ, we mainly discuss its alternative form based on the Euclidean norm [9, 12].

We hereinafter introduce some useful notations. Let the symbols ${A}^{\prime}$, ${A}^{-}$, ${A}^{+}$, $\mathcal{R}(A)$, $\mathcal{R}{(A)}^{\perp}$ and $rk(A)$ stand for the transpose, a generalized inverse, the Moore-Penrose inverse, the column space, the orthogonal complement of the column space and the rank of the matrix *A*, respectively. Moreover, write ${P}_{A}=A{A}^{+}=A{({A}^{\prime}A)}^{+}{A}^{\prime}$ and ${M}_{A}=I-{P}_{A}$, in particular, $H={P}_{X}$, $M=I-H$. ${\lambda}_{i}(A)$ denotes the *i* th largest eigenvalue of the matrix *A*.

## 2 A new Kantorovich-type inequality

We start with some lemmas which are very useful in the following.

**Lemma 2.1**

*Let*

*A*

*be an*$n\times n$

*complex matrix and*${\lambda}_{1},\dots ,{\lambda}_{n}$

*be eigenvalues of*

*A*.

*Then we have*

*and the equality holds if and only if* *A* *is a regular matrix*.

*Proof* The proof is very easy, we therefore omit it here. □

**Lemma 2.2**

*Let*

*A*

*be an*$n\times n$

*positive semidefinite Hermitian matrix and*

*U*

*be an orthogonal projection matrix with*$rk(U)=k$.

*Then we have*

The proof can be found in [13]. See also [14].

**Lemma 2.3**

*The Pólya and Szegö inequality*

*where* $0<{m}_{1}\le {a}_{i}\le {M}_{1}$, $0<{m}_{2}\le {b}_{i}\le {M}_{2}$, $i=1,\dots ,n$.

**Lemma 2.4**

*Let*

*A*

*and*

*B*

*be two*$n\times n$

*positive semidefinite Hermitian matrices*,

*and*

*U*

*be an*$n\times k$

*matrix*, $\mathcal{R}(A)\subset \mathcal{R}(B)$, $rk(B)=q$, $rk(BU)=t$,

*then*

The proof can be found in [15]. See also [14].

**Theorem 2.5**

*Let*

*A*

*be an*$n\times n$

*positive semidefinite Hermitian matrix and*${\lambda}_{1}\ge \cdots \ge {\lambda}_{s}>0$ ($s\le n$)

*be the ordered eigenvalues of*

*A*,

*and let*

*U*

*be an*$n\times p$

*complex matrix such that*${U}^{\ast}U={I}_{p}$.

*If*$p\le s$,

*we then have*

*Proof* The proof is similar to Theorem 1 in [9], we therefore omit it here. □

## 3 The comparison of efficiencies

The above formula and its lower bound both require the covariance matrix Σ to be positive definite and the design matrix to have full column rank. This assumption limits clearly the number of characters which may be included in the model. We here generalize this formula to the situation where the matrices *X* and Σ can be of arbitrary rank.

*X*and Σ. These categories are as follows:

- (1)
$\mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma})$, $rk(X)=p$, Σ is possibly singular;

- (2)
$\mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma})$, $rk(X)<p$, Σ is possibly singular;

- (3)
Σ is possibly singular.

*i*($i=1,2,3$). Many authors have contributed to the theory in the literature; see,

*e.g.*, [17, 18]. The general representations for the BLUE of

*Xβ*and their covariance matrices can be given respectively by

*Xβ*is given by

In the following, we make efficiency comparisons between the OLSE and BLUE in a singular model according to the above category.

*ρ*becomes

It is easy to prove that ${\rho}_{1}\le 1$. The following theorem gives its lower bound.

**Theorem 3.1**

*In the linear regression model*(1.1),

*let*${\lambda}_{1}\ge \cdots \ge {\lambda}_{s}$ ($s<n$)

*be the ordered eigenvalues of*Σ

*and*

*X*

*be an*$n\times p$

*design matrix with*$rk(X)=p$, $\mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma})$.

*Then we have*

*P*such that $\mathrm{\Sigma}=P\mathrm{\Lambda}{P}^{\prime}$, so , where $\mathrm{\Lambda}=diag({\lambda}_{1},\dots ,{\lambda}_{s},0,\dots ,0)$. Let , and then we have that and

Using Theorem 2.5, the result in Theorem 3.1 can be established. □

*ρ*becomes

It is easy to prove that ${\rho}_{2}\le 1$. The following theorem gives its lower bound.

**Theorem 3.2**

*In the linear regression model*(1.1),

*let*${\lambda}_{1}\ge \cdots \ge {\lambda}_{s}$ ($s<n$)

*be the ordered eigenvalues of*Σ

*and*

*X*

*be an*$n\times p$

*design matrix with*$rk(X)=r$ ($r<p$), $\mathcal{R}(X)\subset \mathcal{R}(\mathrm{\Sigma})$.

*We then have*

*Proof*It is easy to prove that

Then the proof is similar to Theorem 3.1, therefore we omit it here. □

**Theorem 3.3**

*In the linear regression model*(1.1),

*let*${\lambda}_{1}\ge \cdots \ge {\lambda}_{s}$ ($s<n$)

*be the ordered eigenvalues of*Σ

*and*

*X*

*be an*$n\times p$

*design matrix with*$rk(X)=r$ ($r\le p$), $dim\mathcal{R}(X)\cap \mathcal{R}(\mathrm{\Sigma})=g$, $rk(H\mathrm{\Sigma}M{(M\mathrm{\Sigma}M)}^{-}M\mathrm{\Sigma}H)=h$,

*we then have*

*Proof*For convenience, let $a=\parallel H\mathrm{\Sigma}M{(M\mathrm{\Sigma}M)}^{-}M\mathrm{\Sigma}H\parallel $ and $b=\parallel H\mathrm{\Sigma}H\parallel $. Then $H\mathrm{\Sigma}M{(M\mathrm{\Sigma}M)}^{-}M\mathrm{\Sigma}H$ is invariant for all the choices of generalized inverses ${(M\mathrm{\Sigma}M)}^{-}$. From Lemma 2.1, we can easily get that

*H*is an orthogonal projection matrix, and then we obtain from Lemma 2.2 that

*e.g.*, [14]) shows that $n-r+g>s$. Analogically, we can get that

By the Pólya and Szegö inequality and a nontrivial but elementary combinational argument, we can establish the first inequality. In fact, the second inequality is similar. □

## 4 Conclusions

In this article, we use several new matrix norm versions of the Kantorovich inequality involving a nonnegative definite matrix to make the comparison of efficiencies between OLSE and BLUE in a singular linear model. The singular linear model is divided into three categories in accordance with the assumptions on the ranks of *X* and Σ. We introduce some new relative efficiency criteria and their lower or upper bounds are given based on matrix norm inequalities in Theorem 3.1, Theorem 3.2 and Theorem 3.3.

## Declarations

### Acknowledgements

The authors thank very much associate editors and referees for their insightful comments that led to improving the presentation. This work is partly supported by the National Natural Science Foundation of China (No. 11126211; No. 61201398) and the Natural Science Foundation of Zhejiang Province (No. LQ12A01021).

## Authors’ Affiliations

## References

- Wang SG, Shao J: Constrained Kantorovich inequality and relative efficiency of least squares.
*J. Multivar. Anal.*1992, 42: 284–298. 10.1016/0047-259X(92)90048-KMATHMathSciNetView ArticleGoogle Scholar - Watson, GS: Serial correlation in regression analysis. PhD thesis, Department of Experimental Statistics, North Carolina State College, Raleigh (1951)Google Scholar
- Khatri CG, Rao CR: Some generalizations of the Kantorovich inequality.
*Sankhya*1982, 44: 91–102.MATHMathSciNetGoogle Scholar - Liu SZ: Efficiency comparisons between the OLSE and the BLUE in a singular linear model.
*J. Stat. Plan. Inference*2000, 84: 191–200. 10.1016/S0378-3758(99)00149-4MATHView ArticleGoogle Scholar - Rao CR: The inefficiency of least squares: extension of the Kantorovich inequality.
*Linear Algebra Appl.*1985, 70: 249–255.MATHMathSciNetView ArticleGoogle Scholar - Yang H: Extensions of the Kantorovich inequality and the error ratio efficiency of the mean square.
*Math. Appl.*1988, 4: 85–90.Google Scholar - Bloomfield P, Watson GS: The inefficiency of least squares.
*Biometrika*1975, 62: 121–128. 10.1093/biomet/62.1.121MATHMathSciNetView ArticleGoogle Scholar - Knott M: On the minimum efficiency of the least square.
*Biometrika*1975, 62: 129–132. 10.1093/biomet/62.1.129MATHMathSciNetView ArticleGoogle Scholar - Yang H, Wang LT: An alternative form of the Watson efficiency.
*J. Stat. Plan. Inference*2009, 139: 2767–2774. 10.1016/j.jspi.2009.01.002MATHView ArticleGoogle Scholar - Liski EP, Puntanen S, Wang SG: Bounds for the trace of the difference of the covariance matrices of the OLSE and BLUE.
*Linear Algebra Appl.*1992, 176: 121–130.MATHMathSciNetView ArticleGoogle Scholar - Chu K, Isotalo J, Puntanen S, Styan GPH: On decomposing the Watson efficiency of ordinary least squares in a partitioned weakly singular linear model.
*Sankhya*2004, 66: 634–651.MATHMathSciNetGoogle Scholar - Wang LT, Yang H: Several matrix Euclidean norm inequalities involving Kantorovich inequality.
*J. Inequal. Appl.*2009., 2009: Article ID 291984Google Scholar - Poincare H: Sur les equation aux derives partielles de la physique mathematique.
*Am. J. Math.*1890, 12: 211–294. 10.2307/2369620MATHMathSciNetView ArticleGoogle Scholar - Wang SG, Jia ZZ:
*Matrix Inequality*. Anhui Education Press, Hefei; 1994.Google Scholar - Scott AJ, Styan GPH: On a separation theorem for generalized eigenvalues and a problem in the analysis of sample surveys.
*Linear Algebra Appl.*1985, 70: 209–224.MATHMathSciNetView ArticleGoogle Scholar - Chu K, Isotalo J, Puntanen S, Styan GPH: The efficiency factorization multiplier for the Watson efficiency in partitioned linear models: some examples and a literature review.
*J. Stat. Plan. Inference*2007, 137: 3336–3351. 10.1016/j.jspi.2007.03.015MATHMathSciNetView ArticleGoogle Scholar - Christensen R:
*Plane Answers to Complex Questions: The Theory of Linear Models*. Springer, New York; 1987.MATHView ArticleGoogle Scholar - Groß J: The general Gauss-Markov model with possibly singular dispersion matrix.
*Stat. Pap.*2004, 45: 311–336. 10.1007/BF02777575MATHView ArticleGoogle Scholar - Meyer CD:
*Matrix Analysis and Applied Linear Algebra*. Cambridge University Press, Cambridge; 2001.Google Scholar - Zyskind G, Martin FB: On best linear estimation and a general Gauss-Markov theorem in linear models with arbitrary nonnegative covariance structure.
*SIAM J. Appl. Math.*1969, 17: 1190–1202. 10.1137/0117110MATHMathSciNetView ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.