 Research
 Open Access
 Published:
Convergence on successive overrelaxed iterative methods for nonHermitian positive definite linear systems
Journal of Inequalities and Applications volume 2016, Article number: 156 (2016)
Abstract
Some convergence conditions on successive overrelaxed (SOR) iterative method and symmetric SOR (SSOR) iterative method are proposed for nonHermitian positive definite linear systems. Some examples are given to demonstrate the results obtained.
Introduction
Many problems in scientific computing give rise to a system of n linear equations
where A is a large sparse nonHermitian positive definite matrix, that is, its Hermitian part \(H=(A+A^{*})/2\) is Hermitian positive definite, where \(A^{*}\) denotes the conjugate transpose of a matrix A. In order to solve system (1) by iterative methods, usually, efficient splittings of the coefficient matrix A are required. For example, the classic Jacobi and GaussSeidel iterations [1–3] split the matrix A into its diagonal and offdiagonal parts. Recently, considerable interest appears in the work on the Hermitian and skewHermitian splitting (HSS) method for this system introduced by Bai et al. [4] and some generalized HSS methods such as the NSS method [5], PSS method [6], PHSS method [7, 8], and LHSS method [9], and several significant theoretical results are proposed. Meanwhile, these methods and theoretical results are applied to this linear system directly or indirectly (as a preconditioner); see [4–14]. It is shown in [3, 15, 16] that the successive overrelaxed (SOR) iterative method and symmetric SOR (SSOR) iterative method for Hermitian positive definite linear systems are convergent. But, is the same true for these iterative methods for nonHermitian positive definite linear systems? In this paper, we mainly study the convergence of the SOR iterative and SSOR iterative method for nonHermitian positive definite linear systems and propose some convergence conditions.
Main results
The main theoretical results in this paper are given to propose some convergence conditions on the SOR and SSOR iterative methods. For convenience of presentation, in Section 2.1, we give some classic iterative methods for linear systems.
SOR iterative methods
In order to solve system (1) by iterative methods, we split the coefficient matrix A in (1) into
where \(D=\operatorname{diag}(\operatorname{Re}(a_{11}),\operatorname{Re}(a_{22}),\ldots, \operatorname{Re}(a_{nn}))\), L is a lower triangular matrix, and U is a strictly upper triangular matrix. Then,
where I is the identity matrix. Without loss of generality, in (2), we can assume that \(D=I\). Decomposition (2) becomes
The forward, backward, and symmetric GaussSeidel (FGS, BGS, and SGS) iterative methods are defined by the iteration matrices
respectively. The forward, backward, and symmetric successive overrelaxation (FSOR, BSOR, and SSOR) iterative methods are defined by the iteration matrices
and
respectively.
Convergence results for SOR iterative method
Throughout the paper, we denote the conjugate transpose of a vector \(x\in\mathbb{C}\) and a matrix \(A\in\mathbb{C}^{n\times n}\), the spectrum of A, and the set of all eigenvectors of A by \(x^{*}\) and \(A^{*}\), \(\sigma(A)=\{\lambda\in\mathbb{C}: \operatorname{det}(\lambda IA)=0\}\), and \(\vartheta(A)=\{x\in\mathbb{C}^{n}: Ax=\lambda x, \lambda\in\sigma(A)\} \), respectively. Let \(\rho(A)=\max_{\lambda\in\sigma(A)\vert \lambda \vert }\) be the spectral radius of A, and \(\vartheta_{\max}(A)=\{x\in\vartheta (A): Ax=\lambda x, \vert \lambda \vert =\rho(A)\}\). The following lemmas will be used in this paper.
Lemma 1
Let \(A=MN\in\mathbb{C}^{n\times n}\) with M nonsingular and \(F=M^{1}N\). Then \(\rho(F)<1\) if and only if \(HF^{*}HF\) is Hermitian positive definite on \(\vartheta(F)\) for any \(n\times n\) Hermitian positive definite matrix H.
Proof
Let λ be any eigenvalue of the matrix F, and \(x\in\vartheta (F)\) be a corresponding eigenvector with \(x\neq0\), that is, \(Fx=\lambda x\). Then, for all \(x\in\vartheta(F)\), we have
which indicates that this lemma is true. □
From Lemma 1 we have the following conclusion.
Lemma 2
Let \(A=MN\in\mathbb{C}^{n\times n}\) with M nonsingular and \(F=M^{1}N\). Then \(\rho(F)<1\) if and only if \(M^{*}MN^{*}N\) is Hermitian positive definite on \(\vartheta(F)\).
In what follows, we propose some convergence conditions on SOR and SSOR iterative methods for nonHermitian positive definite linear systems.
Theorem 1
Let \(A=ILU\in\mathbb{C}^{n\times n}\) be positive definite with \(H=(A^{*}+A)/2\), and \(L_{\omega}\) be defined in (6). Then the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\), that is, \(\rho(L_{\omega})<1\) if and only if \(0<\omega<1/\eta\) if \(\eta>0\) or \(1/\eta<\omega\) if \(\eta <0\) or \(0<\omega\) if \(\eta=0\), where
for any \(x\in\vartheta_{\max}(L_{\omega})\).
Proof
Let \(\alpha=1/\omega1\). Then (6) is changed as
Let \(M=(\alpha+1)IL\) and \(N=U+\alpha I\). Then (7) is changed into \(L_{\alpha}=M^{1}N\). Assume that λ is an eigenvalue of \(L_{\alpha}\) with \(\vert \lambda \vert =\rho(L_{\alpha})\) and \(x\in\vartheta_{\max }(L_{\omega})\) is its corresponding eigenvector. Then, \(M^{1}Nx=\lambda x\), \(Nx=\lambda M x\), and consequently \(x^{*}N^{*}Nx= \vert \lambda \vert ^{2} x^{*}M^{*}Mx\). Thus,
For any \(x\in\vartheta_{\max}(L_{\omega})\), we have
where
It follows from Lemma 2, (8), and (9) that \(\rho (L_{\omega})<1\) if and only if \(2x^{*}Hx(1/\omega\eta)>0\) for \(x\in \vartheta_{\max}(L_{\omega})\). Since A is positive definite, \(H=(A+A^{*})/2\) is Hermitian positive definite, that is, \(x^{*}Hx>0\) for any \(x\neq0\), \(x\in\mathbb{C}^{n}\). As a consequence, \(x^{*}Hx>0\) for any \(x\in\vartheta_{\max}(L_{\omega})\). Thus, \(\rho(L_{\omega})<1\) if and only if \(1/\omega>\eta\). Again, since \(1/\omega>\eta\) holds if and only if \(0<\omega<1/\eta\) if \(\eta>0\) or \(1/\eta<\omega\) if \(\eta<0\) or \(0<\omega\) if \(\eta=0\), this completes the proof. □
Theorem 2
Let \(A=ILU\in\mathbb{C}^{n\times n}\) be positive definite with \(H=(A^{*}+A)/2\), and \(L_{\omega}\) be defined in (6). Then the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\), that is, \(\rho(L_{\omega})<1\) if and only if \(0<\omega<1/(1\tau)\) if \(\tau<1\) or \(\omega>1/(1\eta)\) if \(\tau>1\) or \(\omega>0\) if \(\tau=1\), where
for any \(x\in\vartheta_{\max}(L_{\omega})\).
Proof
Since
yields \(\tau=1\eta\), the conclusion of the theorem follows from Theorem 1. □
Theorem 3
Let \(ALU\in\mathbb{C}^{n\times n}\) be positive definite with \(H=(A^{*}+A)/2\), and \(L_{\omega}\) be defined in (6). Suppose that A satisfies one of the two conditions: (i) \(\Vert T\Vert _{2}\leq1\) and \(0<\omega<1\); (ii) \(\Vert T\Vert _{2}>1\) and \(0<\omega<\omega_{0}\). Then the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\), that is, \(\rho (L_{\omega})<1\), where
Proof
According to the proof of Theorem 1,
(i) If \(\Vert T\Vert _{2}\leq1\) and \(0<\omega<1\), then it is obvious that \(x^{*}(M^{*}MN^{*}N)x>0\) for all \(x\neq0\), \(x\in\mathbb{C}^{n}\). Hence, \(M^{*}MN^{*}N\succ0\). As a result, \(I(M^{1}N)(M^{1}N)^{*}\succ0\) and \(\rho (L_{\omega})\leq \Vert M^{1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
(ii) If \(\Vert T\Vert _{2}>1\) and \(0<\omega<\omega_{0}\), then by the same method we can prove that \(\rho(L_{\omega})\leq \Vert M^{1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\). This completes the proof. □
Theorem 4
Let \(A=ILU\in\mathbb{C}^{n\times n}\) be positive definite with \(H=(A^{*}+A)/2\), and \(L_{\omega}\) be defined in (6). Suppose that A satisfies one of the two conditions: (i) \(\sigma _{\min}(F)\leq1\) and \(0<\omega<1\); (ii) \(\sigma_{\min}(F)>1\) and \(0<\omega<\omega_{1}\). Then the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\), that is, \(\rho(L_{\omega})<1\), where \(\sigma_{\min}(F)\) denotes the minimal singular value of the matrix F, and
Proof
According to the proofs of Theorems 1 and 3,
(i) If \(\sigma_{\min}(F)\leq1\) and \(0<\omega<1\), then it is obvious that \(x^{*}(M^{*}MN^{*}N)x>0\) for all \(x\neq0\), \(x\in\mathbb{C}^{n}\). Hence, \(M^{*}MN^{*}N\succ0\). As a result, \(I(M^{1}N)(M^{1}N)^{*}\succ0\) and \(\rho (L_{\omega})\leq \Vert M^{1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
(ii) If \(\sigma_{\min}(F)>1\) and \(0<\omega<\omega_{0}\), then by the same method we can prove that \(\rho(L_{\omega})\leq \Vert M^{1}N\Vert _{2}<1\), that is, the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\). This proof is completed. □
Remark 1

(1)
It follows from Theorem 3 that whether the forward GaussSeidel method converges or not, there always exists a positive constant \(\omega_{0}\) such that, for \(0<\omega<\omega_{0}\), the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).

(2)
Theorem 4 shows that whether the backward GaussSeidel method converges or not, there always exists a positive constant \(\omega _{1}\) such that, for \(0<\omega<\omega_{1}\) the SOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
Convergence results for SSOR iterative method
In what follows, convergence results for the SSOR iterative method are established for nonHermitian positive definite linear systems.
Theorem 5
Let \(A=ILU\in\mathbb{C}^{n\times n}\) with \(A_{L}=ILL^{*}\) and \(A_{U}=IUU^{*}\) both Hermitian positive semidefinite. Then for \(0<\omega<2\), the SSOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\), that is, \(\rho(S_{\omega})<1\) for \(0<\omega<2\), where \(S_{\omega}\) is defined in (8).
Proof
If \(0<\omega<2\), then
Let y be an eigenvector corresponding to the eigenvalue μ of \(S_{\omega}\) with \(\vert \mu \vert =\rho(S_{\omega})\). Then
and
Hence,
which shows that
is singular, where \(R_{\mu}/(I\omega L)\) denotes the Schur complement of
with respect to \(I\omega L\). It then follows from Lemma 3.13 in [17] that \(R_{\mu}\) is singular. Assume that \(\rho(S_{\omega})=\vert \mu \vert \geq1\). Since \(A_{U}=IUU^{*}\) is Hermitian positive semidefinite and \(0<\omega<2\),
is Hermitian positive semidefinite. Thus,
which implies
In the same way, the Hermitian positive semidefiniteness of \(A_{L}=ILL^{*}\) and \(0<\omega<2\) also yield the inequality
Inequalities (12) and (13) show that \(R_{\mu}\) is block diagonally dominant. By Theorem 2.1 in [18], \(R_{\mu}\) is nonsingular, which contradicts the fact that \(R_{\mu}\) is singular. This indicates that the assumption \(\rho(S_{\omega})=\vert \mu \vert \geq1\) is not correct. Thus, \(\rho(S_{\omega})=\vert \mu \vert <1\), i.e., the SSOR iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\). The proof is completed. □
We know from (5) and (8) that the SSOR method changes into SGS method when \(\omega=1\). Thus, it follows from Theorem 5 that the following conclusion is true.
Theorem 6
Let \(A=ILU\in\mathbb{C}^{n\times n}\) with \(A_{L}=ILL^{*}\) and \(A_{U}=IUU^{*}\) both Hermitian positive semidefinite. Then the SGS iterative method converges to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
Numerical examples
Two numerical examples are given in this section to demonstrate that the convergence results are true and the SOR iterative and SSOR iterative methods are very effective.
Example 1
The coefficient matrix A of the linear system (1) is given as
It is verified that A is positive definite. By Matlab 7.0 computations we get Table 1.
Table 1 shows that for the matrix A, the SOR iterative method is convergent for \(0<\omega<0.87\). By direct computations, \(\Vert T\Vert _{2}=4.0258>1\) when \(\omega_{0}=0.0281\). Thus, Theorem 3 shows that the SOR iterative method converges for the matrix A when \(0<\omega <0.0281\). The same is shown in Theorem 4 for \(0<\omega<0.2292\). It follows from Table 1 that Theorems 3 and 4 are both true. However, both intervals given by the theorems are very narrow. Sometimes these intervals fail to include the optimal value such that \(\rho(L_{\omega})\) reaches minimization.
Example 2
The coefficient matrix A of the linear system (1) is given as
It is verified that \(A_{L}\) and \(A_{U}\) are both Hermitian positive definite for \(n=100\). Matlab computations yield some comparison results on the spectral radius of SSOR iterative matrices; see Table 2.
Table 2 shows that the spectral radius \(\rho(S_{\omega})\) gradually decreases to 0.10674 with ω increasing from 0.125 to 0.750, whereas \(\rho(S_{\omega})\) gradually increases from 0.15931 to 0.88823 with ω increasing from 1.125 to 1.925. However, when ω increases from 0.750 to 1.125, \(\rho(S_{\omega})\) gradually increases from 0.10674 to 0.27623 and gradually decreases from 0.27623 to 0.15931. Therefore, the optimal value of ω should be \(\omega _{opt}\in(0.625,1.250)\) such that the SSOR iterative method converges faster to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
Conclusions
In this paper, we study the convergence of the SOR and SSOR iterative methods for nonHermitian positive definite linear systems. we propose some necessary and sufficient conditions for the convergence of the SOR iterative method. But, these conditions are only theoretically significant and are difficult to apply to practical computations. In what follows, two conditions are presented such that there always exists a positive constant \(\omega_{0}\) (\(\omega_{1}\)) such that, for \(0<\omega<\omega_{0}\) (\(0<\omega<\omega_{1}\)), the SOR iterative method converges for linear system (1) whether the forward or backward GaussSeidel method converges or not.
It is more important that a practical condition for both \(A_{L}=ILL^{*}\) and \(A_{U}=IUU^{*}\) to be Hermitian positive semidefinite is proposed such that both the SSOR iterative method for any \(\omega\in(0,2)\) and the SGS iterative method converge to the unique solution of (1) for any choice of the initial guess \(x_{0}\).
References
 1.
Golub, GH, Van Loan, CF: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996)
 2.
Saad, Y: Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston (1996)
 3.
Young, DM: Iterative Solution of Large Linear Systems. Academic Press, New York (1971)
 4.
Bai, ZZ, Golub, GH, Ng, MK: Hermitian and skewHermitian splitting methods for nonHermitian positive definite linear systems. SIAM J. Matrix Anal. Appl. 24(3), 603626 (2003)
 5.
Bai, ZZ, Golub, GH, Ng, MK: On successiveoverrelaxation acceleration of the Hermitian and skewHermitian splitting iterations. Numer. Linear Algebra Appl. 17, 319335 (2007)
 6.
Bai, ZZ, Golub, GH, Lu, LZ, Yin, JF: Block triangular and skewHermitian splitting methods for positivedefinite linear systems. SIAM J. Sci. Comput. 26, 844863 (2005)
 7.
Bai, ZZ, Golub, GH: Accelerated Hermitian and skewHermitian splitting iteration methods for saddlepoint problems. IMA J. Numer. Anal. 27, 123 (2007)
 8.
Bai, ZZ, Golub, GH, Pan, JY: Preconditioned Hermitian and skewHermitian splitting methods for nonHermitian positive semidefinite linear systems. Numer. Math. 98, 132 (2004)
 9.
Li, L, Huang, TZ, Liu, XP: Modified Hermitian and skewHermitian splitting methods for nonHermitian positivedefinite linear systems. Numer. Linear Algebra Appl. 14, 217235 (2007)
 10.
Bai, ZZ, Golub, GH, Ng, MK: On inexact Hermitian and skewHermitian splitting methods for nonHermitian positive definite linear systems. Linear Algebra Appl. 428, 413440 (2008)
 11.
Wang, L, Bai, ZZ: Convergence conditions for splitting iteration methods for nonHermitian linear systems. Linear Algebra Appl. 428, 453468 (2008)
 12.
Benzi, M, Gander, M, Golub, GH: Optimization of the Hermitian and skewHermitian splitting iteration for saddlepoint problems. BIT Numer. Math. 43, 881900 (2003)
 13.
Benzi, M, Golub, GH: A preconditioner for generalized saddle point problems. SIAM J. Matrix Anal. Appl. 26, 2041 (2004)
 14.
Bertaccini, D, Golub, GH, Capizzano, SS, Tablino Possio, C: Preconditioned HSS method for the solution of nonHermitian positive definite linear systems and applications to the discrete convectiondiffusion equation. Numer. Math. 99, 441484 (2005)
 15.
Varga, RS: Matrix Iterative Analysis, 2nd edn. Springer, Berlin (2000)
 16.
Demmel, JW: Applied Numerical Linear Algebra. SIAM, Philadelphia (1997)
 17.
Zhang, CY, Luo, SH, Xu, CX, Jiang, HY: Schur complements of generally diagonally dominant matrices and criterion for irreducibility of matrices. Electron. J. Linear Algebra 18, 6987 (2009)
 18.
Kolotilina, LY: Nonsingularity/singularity criteria for nonstrictly block diagonally dominant matrices. Linear Algebra Appl. 359, 133159 (2003)
Acknowledgements
The work was supported by the National Natural Science Foundations of China (11201362 and 11271297), the Natural Science Foundation of Shaanxi Province (2016JM1009) and Yunnan NSF Grant (2011FZ190).
Author information
Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
All authors contributed equally to this work. All authors read and approved the final manuscript.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Zhang, Cy., Miao, G. & Zhu, Y. Convergence on successive overrelaxed iterative methods for nonHermitian positive definite linear systems. J Inequal Appl 2016, 156 (2016). https://doi.org/10.1186/s1366001611006
Received:
Accepted:
Published:
MSC
 15A06
 15A15
 15A48
Keywords
 SOR iterative method
 SSOR iterative method
 nonHermitian positive definite linear system
 convergence