Convergence on successive over-relaxed iterative methods for non-Hermitian positive definite linear systems
- Cheng-yi Zhang^{1}Email author,
- Guangyan Miao^{2} and
- Yan Zhu^{3}
DOI: 10.1186/s13660-016-1100-6
© Zhang et al. 2016
Received: 25 January 2016
Accepted: 5 June 2016
Published: 17 June 2016
Abstract
Some convergence conditions on successive over-relaxed (SOR) iterative method and symmetric SOR (SSOR) iterative method are proposed for non-Hermitian positive definite linear systems. Some examples are given to demonstrate the results obtained.
Keywords
SOR iterative method SSOR iterative method non-Hermitian positive definite linear system convergenceMSC
15A06 15A15 15A481 Introduction
2 Main results
The main theoretical results in this paper are given to propose some convergence conditions on the SOR and SSOR iterative methods. For convenience of presentation, in Section 2.1, we give some classic iterative methods for linear systems.
2.1 SOR iterative methods
2.2 Convergence results for SOR iterative method
Throughout the paper, we denote the conjugate transpose of a vector \(x\in\mathbb{C}\) and a matrix \(A\in\mathbb{C}^{n\times n}\), the spectrum of A, and the set of all eigenvectors of A by \(x^{*}\) and \(A^{*}\), \(\sigma(A)=\{\lambda\in\mathbb{C}: \operatorname{det}(\lambda I-A)=0\}\), and \(\vartheta(A)=\{x\in\mathbb{C}^{n}: Ax=\lambda x, \lambda\in\sigma(A)\} \), respectively. Let \(\rho(A)=\max_{\lambda\in\sigma(A)\vert \lambda \vert }\) be the spectral radius of A, and \(\vartheta_{\max}(A)=\{x\in\vartheta (A): Ax=\lambda x, \vert \lambda \vert =\rho(A)\}\). The following lemmas will be used in this paper.
Lemma 1
Let \(A=M-N\in\mathbb{C}^{n\times n}\) with M nonsingular and \(F=M^{-1}N\). Then \(\rho(F)<1\) if and only if \(H-F^{*}HF\) is Hermitian positive definite on \(\vartheta(F)\) for any \(n\times n\) Hermitian positive definite matrix H.
Proof
From Lemma 1 we have the following conclusion.
Lemma 2
Let \(A=M-N\in\mathbb{C}^{n\times n}\) with M nonsingular and \(F=M^{-1}N\). Then \(\rho(F)<1\) if and only if \(M^{*}M-N^{*}N\) is Hermitian positive definite on \(\vartheta(F)\).
In what follows, we propose some convergence conditions on SOR and SSOR iterative methods for non-Hermitian positive definite linear systems.
Theorem 1
Proof
Theorem 2
Proof
Theorem 3
Proof
(ii) If \(\Vert T\Vert _{2}>1\) and \(0<\omega<\omega_{0}\), then by the same method we can prove that \(\rho(L_{\omega})\leq \Vert M^{-1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\). This completes the proof. □
Theorem 4
Proof
(ii) If \(\sigma_{\min}(F)>1\) and \(0<\omega<\omega_{0}\), then by the same method we can prove that \(\rho(L_{\omega})\leq \Vert M^{-1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\). This proof is completed. □
Remark 1
- (1)
It follows from Theorem 3 that whether the forward Gauss-Seidel method converges or not, there always exists a positive constant \(\omega_{0}\) such that, for \(0<\omega<\omega_{0}\), the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
- (2)
Theorem 4 shows that whether the backward Gauss-Seidel method converges or not, there always exists a positive constant \(\omega _{1}\) such that, for \(0<\omega<\omega_{1}\) the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
2.3 Convergence results for SSOR iterative method
In what follows, convergence results for the SSOR iterative method are established for non-Hermitian positive definite linear systems.
Theorem 5
Let \(A=I-L-U\in\mathbb{C}^{n\times n}\) with \(A_{L}=I-L-L^{*}\) and \(A_{U}=I-U-U^{*}\) both Hermitian positive semidefinite. Then for \(0<\omega<2\), the SSOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\), that is, \(\rho(S_{\omega})<1\) for \(0<\omega<2\), where \(S_{\omega}\) is defined in (8).
Proof
We know from (5) and (8) that the SSOR method changes into SGS method when \(\omega=1\). Thus, it follows from Theorem 5 that the following conclusion is true.
Theorem 6
Let \(A=I-L-U\in\mathbb{C}^{n\times n}\) with \(A_{L}=I-L-L^{*}\) and \(A_{U}=I-U-U^{*}\) both Hermitian positive semidefinite. Then the SGS iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
3 Numerical examples
Two numerical examples are given in this section to demonstrate that the convergence results are true and the SOR iterative and SSOR iterative methods are very effective.
Example 1
The comparison results on the spectral radius of the SOR iterative matrix of A for different ω
ω | 0.01 | 0.02 | 0.0281 | 0.05 | 0.1 | 0.15 | 0.20 | 0.2292 |
\(\rho(L_{\omega})\) | 0.9929 | 0.9858 | 0.9801 | 0.9645 | 0.9290 | 0.8936 | 0.8585 | 0.8382 |
ω | 0.25 | 0.30 | 0.42 | 0.52 | 0.6 | 0.72 | 0.82 | 0.87 |
\(\rho(L_{\omega})\) | 0.8239 | 0.7901 | 0.7155 | 0.6679 | 0.6512 | 0.7078 | 0.8769 | 1.0007 |
Table 1 shows that for the matrix A, the SOR iterative method is convergent for \(0<\omega<0.87\). By direct computations, \(\Vert T\Vert _{2}=4.0258>1\) when \(\omega_{0}=0.0281\). Thus, Theorem 3 shows that the SOR iterative method converges for the matrix A when \(0<\omega <0.0281\). The same is shown in Theorem 4 for \(0<\omega<0.2292\). It follows from Table 1 that Theorems 3 and 4 are both true. However, both intervals given by the theorems are very narrow. Sometimes these intervals fail to include the optimal value such that \(\rho(L_{\omega})\) reaches minimization.
Example 2
The comparison results on the spectral radius of the SSOR iterative matrix of B for different ω
ω | 0.125 | 0.250 | 0.375 | 0.500 | 0.625 | 0.750 | 0.875 | 0.1 |
\(\rho(S_{\omega})\) | 0.76945 | 0.56558 | 0.39654 | 0.26311 | 0.16501 | 0.10674 | 0.27623 | 0.26139 |
ω | 1.125 | 1.250 | 1.375 | 1.5000 | 1.625 | 1.750 | 1.875 | 1.925 |
\(\rho(S_{\omega})\) | 0.15931 | 0.18860 | 0.26281 | 0.36777 | 0.49742 | 0.64802 | 0.81648 | 0.88823 |
Table 2 shows that the spectral radius \(\rho(S_{\omega})\) gradually decreases to 0.10674 with ω increasing from 0.125 to 0.750, whereas \(\rho(S_{\omega})\) gradually increases from 0.15931 to 0.88823 with ω increasing from 1.125 to 1.925. However, when ω increases from 0.750 to 1.125, \(\rho(S_{\omega})\) gradually increases from 0.10674 to 0.27623 and gradually decreases from 0.27623 to 0.15931. Therefore, the optimal value of ω should be \(\omega _{opt}\in(0.625,1.250)\) such that the SSOR iterative method converges faster to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
4 Conclusions
In this paper, we study the convergence of the SOR and SSOR iterative methods for non-Hermitian positive definite linear systems. we propose some necessary and sufficient conditions for the convergence of the SOR iterative method. But, these conditions are only theoretically significant and are difficult to apply to practical computations. In what follows, two conditions are presented such that there always exists a positive constant \(\omega_{0}\) (\(\omega_{1}\)) such that, for \(0<\omega<\omega_{0}\) (\(0<\omega<\omega_{1}\)), the SOR iterative method converges for linear system (1) whether the forward or backward Gauss-Seidel method converges or not.
It is more important that a practical condition for both \(A_{L}=I-L-L^{*}\) and \(A_{U}=I-U-U^{*}\) to be Hermitian positive semidefinite is proposed such that both the SSOR iterative method for any \(\omega\in(0,2)\) and the SGS iterative method converge to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
Declarations
Acknowledgements
The work was supported by the National Natural Science Foundations of China (11201362 and 11271297), the Natural Science Foundation of Shaanxi Province (2016JM1009) and Yunnan NSF Grant (2011FZ190).
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- Golub, GH, Van Loan, CF: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996) MATHGoogle Scholar
- Saad, Y: Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston (1996) MATHGoogle Scholar
- Young, DM: Iterative Solution of Large Linear Systems. Academic Press, New York (1971) MATHGoogle Scholar
- Bai, ZZ, Golub, GH, Ng, MK: Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems. SIAM J. Matrix Anal. Appl. 24(3), 603-626 (2003) MathSciNetView ArticleMATHGoogle Scholar
- Bai, ZZ, Golub, GH, Ng, MK: On successive-overrelaxation acceleration of the Hermitian and skew-Hermitian splitting iterations. Numer. Linear Algebra Appl. 17, 319-335 (2007) MathSciNetView ArticleMATHGoogle Scholar
- Bai, ZZ, Golub, GH, Lu, LZ, Yin, JF: Block triangular and skew-Hermitian splitting methods for positive-definite linear systems. SIAM J. Sci. Comput. 26, 844-863 (2005) MathSciNetView ArticleMATHGoogle Scholar
- Bai, ZZ, Golub, GH: Accelerated Hermitian and skew-Hermitian splitting iteration methods for saddle-point problems. IMA J. Numer. Anal. 27, 1-23 (2007) MathSciNetView ArticleMATHGoogle Scholar
- Bai, ZZ, Golub, GH, Pan, JY: Preconditioned Hermitian and skew-Hermitian splitting methods for non-Hermitian positive semidefinite linear systems. Numer. Math. 98, 1-32 (2004) MathSciNetView ArticleMATHGoogle Scholar
- Li, L, Huang, TZ, Liu, XP: Modified Hermitian and skew-Hermitian splitting methods for non-Hermitian positive-definite linear systems. Numer. Linear Algebra Appl. 14, 217-235 (2007) MathSciNetView ArticleMATHGoogle Scholar
- Bai, ZZ, Golub, GH, Ng, MK: On inexact Hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems. Linear Algebra Appl. 428, 413-440 (2008) MathSciNetView ArticleMATHGoogle Scholar
- Wang, L, Bai, ZZ: Convergence conditions for splitting iteration methods for non-Hermitian linear systems. Linear Algebra Appl. 428, 453-468 (2008) MathSciNetView ArticleMATHGoogle Scholar
- Benzi, M, Gander, M, Golub, GH: Optimization of the Hermitian and skew-Hermitian splitting iteration for saddle-point problems. BIT Numer. Math. 43, 881-900 (2003) MathSciNetView ArticleMATHGoogle Scholar
- Benzi, M, Golub, GH: A preconditioner for generalized saddle point problems. SIAM J. Matrix Anal. Appl. 26, 20-41 (2004) MathSciNetView ArticleMATHGoogle Scholar
- Bertaccini, D, Golub, GH, Capizzano, SS, Tablino Possio, C: Preconditioned HSS method for the solution of non-Hermitian positive definite linear systems and applications to the discrete convection-diffusion equation. Numer. Math. 99, 441-484 (2005) MathSciNetView ArticleMATHGoogle Scholar
- Varga, RS: Matrix Iterative Analysis, 2nd edn. Springer, Berlin (2000) View ArticleMATHGoogle Scholar
- Demmel, JW: Applied Numerical Linear Algebra. SIAM, Philadelphia (1997) View ArticleMATHGoogle Scholar
- Zhang, CY, Luo, SH, Xu, CX, Jiang, HY: Schur complements of generally diagonally dominant matrices and criterion for irreducibility of matrices. Electron. J. Linear Algebra 18, 69-87 (2009) MathSciNetMATHGoogle Scholar
- Kolotilina, LY: Nonsingularity/singularity criteria for non-strictly block diagonally dominant matrices. Linear Algebra Appl. 359, 133-159 (2003) MathSciNetView ArticleMATHGoogle Scholar