From Equivalent Linear Equations to Gauss-Markov Theorem
© Czesław Stępniak. 2010
Received: 17 December 2009
Accepted: 27 June 2010
Published: 13 July 2010
Gauss-Markov theorem reduces linear unbiased estimation to the Least Squares Solution of inconsistent linear equations while the normal equations reduce the second one to the usual solution of consistent linear equations. It is rather surprising that the second algebraic result is usually derived in a differential way. To avoid this dissonance, we state and use an auxiliary result on equivalence of two systems of linear equations. This places us in a convenient position to attack on the main problems in the Gauss-Markov model in an easy way.
The Gauss-Markov theorem is the most classical achievement in statistics. Its role in statistics is comparable with that of the Pythagorean theorem in geometry. In fact, there are close relations between both of them.
The Gauss-Markov theorem is presented in many books and derived in many ways. The most popular approaches involve
(iv)projection operators (see Seber ).
We presume that such a big market has many clients. This paper is intended for some of them. Our consideration is straightforward and self-contained. Moreover, it needs only moderate prerequisites.
The main tool used in this paper is equivalence of two systems of linear equations.
(i.e., the orthogonal projector onto ) It follows from definition (2.4) that The following lemma (see ) will be a key tool in the further consideration.
3. Least Squares Solution
The following theorem shows that this definition is not empty and reduces the LSS of an inconsistent equation (3.1) to the ordinary solution of a consistent one.
Equation (3.1) has at least one LSS.
In the statistical literature, (3.3) is said to be normal.
Statement (d) follows directly from definition of kernel.
4. Gauss-Markov Model and Gauss-Markov Theorem
Hence, any unbiased estimator of may be presented in the form , where the first component is the LSE of while . In particular the components are not correlated. Therefore, the variance of the sum is greater than the variance of the LSE , unless . Moreover, by Theorem 3.2(d) this estimator is invariant with respect to the choice of the LSS . In consequence, the LSE of the function is unique.
Thanks are due to a reviewer for his comments leading to the improvement in the presentation of this paper.
- Kruskal W: The coordinate-free approach to Gauss-Markov estimation, and its application to missing and extra observations. In Proceedings of 4th Berkeley Symposium on Mathematical Statistics and Probability. Volume 1. University of California Press, Berkeley, Calif, USA; 1961:435–451.Google Scholar
- Kruskal W: When are Gauss-Markov and least squares estimators identical? A coordinate-free approach. Annals of Mathematical Statistics 1968, 39: 70–75. 10.1214/aoms/1177698505MathSciNetView ArticleMATHGoogle Scholar
- Scheffé H: The Analysis of Variance. John Wiley & Sons, New York, NY, USA; 1959:xvi+477.MATHGoogle Scholar
- Rao CR: Linear Statistical Inference and Its Applications. 2nd edition. John Wiley & Sons, New York, NY, USA; 1973:xx+625.View ArticleMATHGoogle Scholar
- Rao CR, Mitra SK: Generalized Inverse of Matrices and Its Applications. John Wiley & Sons, New York, NY, USA; 1971:xiv+240.MATHGoogle Scholar
- Bapat RB: Linear Algebra and Linear Models, Universitext. 2nd edition. Springer, New York, NY, USA; 2000:x+138.Google Scholar
- Seber GAF: Linear Regression Analysis. John Wiley & Sons, New York, NY, USA; 1977:xvii+465.MATHGoogle Scholar
- Stępniak C: Through a generalized inverse. Demonstratio Mathematica 2008, 41(2):291–296.MathSciNetMATHGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.