A predictor-corrector iterative method for solving linear least squares problems and perturbation error analysis

Suzan C. Buranay; Ovgu C. Iyikal

doi:10.1186/s13660-019-2154-z

Research
Open Access
Published: 20 July 2019

A predictor-corrector iterative method for solving linear least squares problems and perturbation error analysis

Journal of Inequalities and Applications volume 2019, Article number: 203 (2019) Cite this article

627 Accesses
1 Citations
Metrics details

Abstract

The motivation of the present work concerns two objectives. Firstly, a predictor-corrector iterative method of convergence order $p=45$ requiring 10 matrix by matrix multiplications per iteration is proposed for computing the Moore–Penrose inverse of a nonzero matrix of $\operatorname{rank}=r$. Convergence and a priori error analysis of the proposed method are given. Secondly, the numerical solution to the general linear least squares problems by an algorithm using the proposed method and the perturbation error analysis are provided. Furthermore, experiments are conducted on the ill-posed problem of one-dimensional image restoration and on some test problems from Harwell–Boeing collection. Obtained numerical results show the applicability, stability, and the estimated order of convergence of the proposed method.

Introduction

The numerical solution of many problems in mathematical physics requires the solution of an algebraic linear system of equations. When the coefficient matrix of a consistent linear system is singular or rectangular matrix, then the need for Moore–Penrose inverse occurs for the computation of minimum norm least squares solution.

Let $\mathbb{C} ^{n_{1}\times n_{2}}$ and $\mathbb{C} _{r}^{n_{1} \times n_{2}}$ denote the sets of all complex $n_{1}\times n_{2}$ matrices and all complex $n_{1}\times n_{2}$ matrices of $\operatorname{rank}=r$, respectively. By $A^{\ast }$, $R ( A )$, and $P_{R ( A ) }$ we present the conjugate transpose of A, range of A, and the orthogonal projector onto $R ( A ) $ respectively, and as usual I denotes the unit matrix of appropriate order. The generalized inverse or Moore–Penrose inverse of a matrix $A\in \mathbb{C} ^{n_{1}\times n_{2}}$ denoted by $A^{\dagger }\in \mathbb{C} ^{n_{2}\times n_{1}}$ is the unique matrix satisfying the following four Penrose equations (see [1]):

$$ AA^{\dagger }A=A,\qquad A^{\dagger }AA^{\dagger }=A^{\dagger }, \qquad \bigl(AA^{\dagger } \bigr)^{\ast }=AA^{\dagger },\qquad \bigl(A^{\dagger }A \bigr)^{ \ast }=A^{\dagger }A. $$

(1.1)

Many important approaches for computing the Moore–Penrose generalized inversion have been developed. Among these methods, direct methods usually tend to require a predictable amount of resources in terms of time and storage, which normally puts them out of interest especially for large scaled systems with sparse matrices. Special interest is given to propose and investigate iterative methods for computing the Moore–Penrose inverse $A^{\dagger }$. Let $A\in \mathbb{C} _{r}^{n _{1}\times n_{2}}$, we survey on iterative methods by which $V_{m}$ denotes the approximate Moore–Penrose inverse of A at the mth iteration, and $Mms$ stands for the matrix by matrix multiplications per iteration. The authors of [2] proposed the sequence

$$ V_{m}=\alpha \sum_{i=0}^{m}A^{\ast } \bigl( I-\alpha AA^{ \ast } \bigr) ^{i},\quad m=0,1,\ldots, $$

(1.2)

that converges to $A^{\dagger }$ as $m\rightarrow \infty $ under the assumption

$$ 0< \alpha < \frac{2}{\lambda _{1} ( A^{\ast }A ) }, $$

(1.3)

where

$$ \lambda _{1} \bigl( A^{\ast }A \bigr) \geq \lambda _{2} \bigl( A^{ \ast }A \bigr) \geq \cdots\geq \lambda _{r} \bigl( A^{\ast }A \bigr) >0 $$

(1.4)

are the nonzero eigenvalues of $A^{\ast }A$. Later in [3] (see also [4]) the iterative method (1.2) was written as follows:

$$\begin{aligned} &V_{0} =\alpha A^{\ast }, \end{aligned}$$

(1.5)

$$\begin{aligned} &V_{m+1} =V_{m}+\alpha ( I-V_{m}A ) A^{\ast }, \quad m=0,1,\ldots, \end{aligned}$$

(1.6)

and it converges to $A^{\dagger }$ linearly. The following iterative algorithm [5, 6]

$$ \begin{aligned} &V_{0} =\alpha A^{\ast }, \\ &V_{m+1} =V_{m} ( 2I-AV_{m} ) ,\quad m=0,1, \ldots, \end{aligned} $$

(1.7)

yields $A^{\dagger }$ as the limit of the sequence $\{ V_{m} \} $, $m=0,1,\ldots$ , when α satisfies (1.3). This method is a variant of the well-known Schulz method of 2nd order. A pth order iterative method for computing $A^{\dagger }$ is studied in [7] and [8] for $p\geq 2$ and p being integer as follows:

$$ \begin{aligned} &V_{0} =\alpha A^{\ast }, \\ &\overline{T}_{m} =I-V_{m}A, \\ &P_{m} =\sum_{i=0}^{p-1} \overline{T}_{m}^{i}, \\ &V_{m+1} =P_{m}V_{m}= \Biggl( \sum _{i=0}^{p-1}\overline{T}_{m} ^{i} \Biggr) V_{m},\quad m=0,1,\ldots. \end{aligned} $$

(1.8)

Method (1.8) is also called hyperpower iterative method and for $n_{1}\leq n_{2}$ is usually presented as

$$ V_{m+1}=V_{m}\sum_{j=0}^{p-1}T_{m}^{j}, \quad m=0,1,\ldots, $$

(1.9)

where $T_{m}=I-AV_{m}$, performing p $Mms$. In [7], when $p=3$, method (1.9) is rewritten in the form

$$ V_{m+1}=V_{m} \bigl( 3I-3AV_{m}+ ( AV_{m} ) ^{2} \bigr) ,\quad m=0,1,\ldots, $$

(1.10)

and in Horner’s form for the inversion of a square nonsingular matrix is given in [9] as

$$ V_{m+1}=V_{m} \bigl( 3I-AV_{m} ( 3I-AV_{m} ) \bigr) ,\quad m=0,1,\ldots, $$

(1.11)

and has at least third order of convergence [7, 10]. There are different representations of the hyperpower method (1.9) which differ in the evaluation of the power sum $\sum_{j=0}^{p-1}T_{m}^{j}$. The computational effort in the pth order method decreases when the number of matrix by matrix multiplications and additions in the polynomial are reduced through factorizations if nested loops are used (see [11] and [12]). When $p=5$, an effective form of the hyperpower iterative method for inverse of a square nonsingular matrix is presented in [11] as follows:

$$ V_{m+1}=V_{m} \bigl( I+ \bigl( I+T_{m}^{2} \bigr) \bigl( T_{m}+T _{m}^{2} \bigr) \bigr),\quad m=0,1,\ldots, $$

(1.12)

and requires 4 $Mms$. To compute the inner inverse of matrix $A\in C^{n_{1}\times n_{2}}$, the following iterative procedure is proposed in [13]:

$$ V_{m+1}=V_{m} \biggl[ pI-\frac{p(p-1)}{2}AV_{m}+ \cdots+ ( -1 ) ^{p-1} ( AV_{m} ) ^{p-1} \biggr],\quad p\geq 2, $$

(1.13)

$m=0,1,2,\ldots$ , which requires p $Mms$. Equation (1.13) is an equivalent form of (1.9). In [14] the authors showed that the iterative procedure (1.13) can be used to compute the Moore–Penrose inverse of a matrix. The authors of [15] proved that the sequences (1.13) indeed are convergent to a fixed inner inverse of the matrix which is the Moore–Penrose inverse of the matrix, and they provided numerical results for the third order iterations. The following 7th order method is given by [16] for computing generalized inverse $A_{T,S}^{(2)}$:

$$ V_{m+1}=V_{m} \bigl( I+ \bigl( T_{m}+T_{m}^{4} \bigr) \bigl( I+T _{m}+T_{m}^{2} \bigr) \bigr) ,\quad m=0,1,\ldots. $$

(1.14)

The method in (1.14) performs 5 $Mms$. Recently several systematic algorithms for factorizations of the hyperpower iterative family (1.9) of arbitrary orders have been given in [17] for computing outer inverses. Among these methods the 11th, 15th, and 19th order methods are

$$\begin{aligned}& V_{m+1}=V_{m} \bigl( I+T_{m} \bigl( I+ \bigl( T_{m}+T_{m}^{2}+T_{m} ^{3} \bigr) \bigl( I+T_{m}^{3}+T_{m}^{6} \bigr) \bigr) \bigr) ,\quad m=0,1,\ldots. \end{aligned}$$

(1.15)

$$\begin{aligned}& V_{m+1}=V_{m} \bigl(I+ \bigl(T_{m}+T_{m}^{2} \bigr) \bigl(I+ \bigl(T_{m}^{2}+T_{m}^{4} \bigr) \bigl(I+T_{m}^{4}+T _{m}^{8} \bigr) \bigr) \bigr),\quad m=0,1,\ldots \end{aligned}$$

(1.16)

$$\begin{aligned}& V_{m+1}=V_{m} \bigl(I+ \bigl(T_{m}+T_{m}^{2} \bigr) \bigl(I+T_{m}^{2}+T_{m}^{4} \bigr) \bigl(I+T_{m}^{6}+T _{m}^{12} \bigr) \bigr),\quad m=0,1,\ldots, \end{aligned}$$

(1.17)

given that each uses $7,7$, and 8 $Mms$, respectively.

In [18] for a given integer parameter $k\geq 1$, two classes of iterative methods for matrix inversion of a square nonsingular matrix were proposed, and these methods were used to construct inverse preconditioners for solving linear systems. Class 1 methods converge with order of convergence $p=3\ast 2^{k}+1$ and Class 2 methods converge with order $p=5\ast 2^{k}-1$, requiring $2k+4$ $Mms$ and $3k+4$ $Mms$, respectively. Most recently, high order iterative methods with a recurrence formula for approximate matrix inversion have been proposed in [19] with orders of convergence $p=4k+3$, where $k\geq 1$ is an integer requiring $k+4$ matrix multiplications per iteration. The methods of orders $p=7,11,15,19$ are used to construct approximate Schur-block incomplete LU preconditioners (Schur-BILU) for regularized solution of discrete ill-posed problems.

The original contribution of this study is the construction of a predictor-corrector iterative method (PCIM) of convergence order $p=45$ requiring 10 $Mms$ for computing the Moore–Penrose inverse of nonzero matrix $A\in \mathbb{C} _{r}^{n_{1}\times n_{2}}$. Also, perturbation error analysis of the proposed method for the minimum norm solution of least squares problems is provided. To verify the theoretical results, experimental analysis are conducted for the one-dimensional image restoration problem and for some well-conditioned and moderately ill-conditioned problems from Harwell–Boeing collection. Next, a benchmark example is used to numerically compute the order of convergence of the proposed method. Concluding remarks are given in the last section.

Some versions of the predictor-corrector variational iteration method for the solution of integral equations of fractional order were studied in [20] and [21].

A predictor-corrector iterative method for approximate Moore–Penrose inverse

Let I denote the $n_{1}\times n_{1}$ unit matrix and let $A\in \mathbb{C} _{r}^{n_{1}\times n_{2}}$ and the nonzero eigenvalues of $A^{\ast }A$ be denoted by (1.4). We define the following matrix-valued functions consisting of matrix multiplications and additions:

$$\begin{aligned}& \varOmega (T_{m}) =T_{m}+T_{m}^{2}, \end{aligned}$$

(2.1)

$$\begin{aligned}& \varPsi (T_{m}) =I+T_{m}^{2}, \end{aligned}$$

(2.2)

$$\begin{aligned}& \varGamma (T_{m}) =T_{m}^{4}, \end{aligned}$$

(2.3)

where $T_{m}= ( I-AV_{m} ) $. We propose the following:

Predictor-corrector iterative method (PCIM).

$$\begin{aligned}& In :V_{0}=\alpha A^{\ast } \\& G :T_{m}=I-AV_{m},\qquad \varPhi (T_{m})= \varPsi (T_{m})\varOmega (T_{m}) \\& P :V_{m+\frac{1}{2}}=V_{m} \bigl( I+\varPhi (T_{m}) \bigr) ,\qquad T _{m+\frac{1}{2}}=I-AV_{m+\frac{1}{2}} \\& C :V_{m+1}=V_{m+\frac{1}{2}} \bigl( I+\varPhi (T_{m+\frac{1}{2}}) \bigl( I+\varGamma (T_{m+\frac{1}{2}}) \bigr) \bigr) \\& \quad \text{for }m =0,1,2,\ldots. \end{aligned}$$

(2.4)

Here, In is the initial step, G is the generator, P is the predictor at the fractional step $m+\frac{1}{2}$, and C is the corrector for the approximate Moore–Penrose inverse $V_{m+1}$ of A at the $m+1$th iteration in (2.4). Let $R_{m}=P_{R ( A ) }-AV_{m}$ be the residual at the mth iteration and consider the following form of (2.4) which requires $P_{R ( A ) }$:

$$\begin{aligned}& In :V_{0}=\alpha A^{\ast }, \\& G :R_{m}=P_{R ( A ) }-AV_{m},\qquad \varPhi(R_{m})=\varPsi (R _{m})\varOmega (R_{m}), \\& P :V_{m+\frac{1}{2}}=V_{m} \bigl( P_{R(A)}+\varPhi (R_{m}) \bigr) ,\qquad R_{m+\frac{1}{2}}=P_{R ( A ) }-AV_{m+\frac{1}{2}}, \\& C :V_{m+1}=V_{m+\frac{1}{2}} \bigl( P_{R(A)}+\varPhi (R_{m+\frac{1}{2}}) \bigl( I+\varGamma (R_{m+\frac{1}{2}}) \bigr) \bigr) \\& \quad \text{for }m =0,1,2,\ldots \end{aligned}$$

(2.5)

Lemma 2.1

Let $A\in \mathbb{C} _{r}^{n_{1}\times n_{2}}$, if in the initial step In of (2.5 α satisfies (1.3), then for the sequence $\{ V_{m+1} \} $ obtained at $m+1$th iteration by PCIM (2.5) the following hold true for every $m=0,1,2,\ldots$ :

$$\begin{aligned}& \mathrm{(i)}\quad V_{m+1}P_{R ( A ) } =V_{m+1},\qquad \mathrm{(ii)}\quad P_{R ( A^{\ast } ) }V_{m+1}=V_{m+1}, \end{aligned}$$

(2.6)

$$\begin{aligned}& \mathrm{(iii)}\quad ( AV_{m+1} ) ^{\ast } = AV_{m+1},\qquad \mathrm{(iv)}\quad ( V_{m+1}A ) ^{\ast }=V_{m+1}A. \end{aligned}$$

(2.7)

Proof

The proof follows from PCIM (2.5) using mathematical induction. We give the proof of (i) and (iii) and the proof of (ii) and (iv) can be given analogously. By $( III ) $ of Theorem 1 [7], $AA^{\dagger }=P_{R ( A ) }$ and $A^{\dagger }A=P_{R ( A^{\ast } ) }$, and on the basis of Moore–Penrose equations (1.1), the projection $P_{R ( A ) }$ satisfies the following:

$$ P_{R ( A ) }^{2}=P_{R ( A ) },\qquad P_{R (A )}^{\ast }=P_{R ( A ) },\qquad P_{R (A ) }A=A;\qquad A^{\ast }P_{R ( A ) }=A^{\ast }. $$

(2.8)

(i) For $m=0$, in the initial step In of PCIM (2.5) we have $V_{0}=\alpha A^{\ast }$ implying $V_{0}P_{R ( A ) }= \alpha A^{\ast }P_{R ( A ) }=\alpha A^{\ast }=V_{0}$. Assume that $V_{m}P_{R ( A ) }=V_{m}$ holds true for m, then using (2.8) for the predictor step P of (2.5) we obtain

$$\begin{aligned} &\begin{aligned}[b] V_{m+\frac{1}{2}}P_{R ( A ) } ={}&V_{m} \bigl( P_{R(A)}+ \varPhi (R_{m}) \bigr) P_{R ( A ) } \\ ={}&V_{m} \bigl( P_{R ( A ) }+ ( P_{R ( A ) }-AV_{m} ) + ( P_{R ( A ) }-AV_{m} ) ^{2}+ ( P_{R ( A ) }-AV_{m} ) ^{3} \\ &{}+ ( P_{R ( A ) }-AV_{m} ) ^{4} \bigr) P_{R ( A ) }=V_{m+\frac{1}{2}}, \end{aligned} \end{aligned}$$

(2.9)

$$\begin{aligned} &\begin{aligned}[b] R_{m+\frac{1}{2}} &=P_{R(A)}-AV_{m+\frac{1}{2}}=P_{R(A)}-AV_{m} \bigl( P_{R(A)}+\varPhi (R_{m}) \bigr) \\ &=P_{R(A)}-AV_{m} \bigl(P_{R(A)}+R_{m}+R_{m}^{2}+R_{m}^{3}+R_{m}^{4} \bigr)=R _{m}^{5}. \end{aligned} \end{aligned}$$

(2.10)

Also in the corrector step C of (2.5) using (2.10) we get

$$\begin{aligned}& \varGamma (R_{m+\frac{1}{2}}) =R_{m+\frac{1}{2}}^{4}=R_{m}^{20}, \end{aligned}$$

(2.11)

$$\begin{aligned}& \varPhi (R_{m+\frac{1}{2}}) =\varPsi (R_{m+\frac{1}{2}})\varOmega (R_{m+ \frac{1}{2}})= \bigl( I+R_{m}^{10} \bigr) \bigl( R_{m}^{5}+R_{m} ^{10} \bigr) . \end{aligned}$$

(2.12)

Using (2.8) and from the assumption $V_{m}P_{R ( A ) }=V_{m}$, equations (2.11) and (2.12) imply

$$\begin{aligned}& \varGamma (R_{m+\frac{1}{2}})P_{R ( A ) }= ( P_{R ( A ) }-AV_{m} ) ^{20}P_{R ( A ) }=\varGamma (R_{m+ \frac{1}{2}}), \end{aligned}$$

(2.13)

$$\begin{aligned}& \varPhi (R_{m+\frac{1}{2}})P_{R ( A ) }= \bigl( I+R_{m}^{10} \bigr) \bigl( R_{m}^{5}+R_{m}^{10} \bigr) P_{R ( A ) }=\varPhi (R_{m+\frac{1}{2}}). \end{aligned}$$

(2.14)

Using (2.8)–(2.14) in the corrector step C of (2.5), we obtain

$$\begin{aligned} V_{m+1}P_{R ( A ) } =&V_{m+\frac{1}{2}} \bigl( P_{R(A)}+ \varPhi (R_{m+\frac{1}{2}}) \bigl( I+\varGamma (R_{m+\frac{1}{2}}) \bigr) \bigr) P_{R ( A ) } \\ =&V_{m+\frac{1}{2}} \bigl( P_{R(A)}+\varPhi (R_{m+\frac{1}{2}})P_{R ( A ) }+ \varPhi (R_{m+\frac{1}{2}})\varGamma (R_{m+\frac{1}{2}})P _{R ( A ) } \bigr) \\ =&V_{m+\frac{1}{2}} \bigl( P_{R(A)}+\varPhi (R_{m+\frac{1}{2}}) \bigl( I+ \varGamma (R_{m+\frac{1}{2}}) \bigr) \bigr) =V_{m+1}. \end{aligned}$$

(2.15)

(iii) For $m=0$, in the initial step In of PCIM (2.5) we have $V_{0}=\alpha A^{\ast }$ implying $( AV_{0} ) ^{\ast }= ( A\alpha A^{\ast } ) ^{\ast }=\alpha AA^{\ast }=AV_{0}$. Assume that $( AV_{m} ) ^{\ast }=AV_{m}$ holds true, then from (2.8) in the predictor step P of (2.5) gives

$$\begin{aligned}& R_{m}^{\ast } = ( P_{R(A)}-AV_{m} ) ^{\ast }=R_{m}, \end{aligned}$$

(2.16)

$$\begin{aligned}& \varPhi ^{\ast }(R_{m}) = \bigl( \bigl( I+R_{m}^{10} \bigr) \bigl( R _{m}^{5}+R_{m}^{10} \bigr) \bigr) ^{\ast }=\varPhi (R_{m}), \end{aligned}$$

(2.17)

$$\begin{aligned}& ( AV_{m+\frac{1}{2}} ) ^{\ast } = \bigl( P_{R(A)}^{ \ast }+ \varPhi ^{\ast }(R_{m}) \bigr) ( AV_{m} ) ^{\ast }=AV _{m+\frac{1}{2}}. \end{aligned}$$

(2.18)

From the assumption and also using (2.8), (2.10)–(2.12), and (2.16) gives

$$ R_{m+\frac{1}{2}}^{\ast }=R_{m+\frac{1}{2}},\qquad \varGamma ^{\ast }(R_{m+ \frac{1}{2}})=\varGamma (R_{m+\frac{1}{2}}),\qquad \varPhi ^{\ast }(R_{m+ \frac{1}{2}})=\varPhi (R_{m+\frac{1}{2}}). $$

(2.19)

Using (2.8), (2.18), and (2.19) in the corrector step C of (2.5) yields

$$\begin{aligned} ( AV_{m+1} ) ^{\ast } =& \bigl( P_{R(A)}+\varPhi (R_{m+ \frac{1}{2}}) \bigl( I+\varGamma (R_{m+\frac{1}{2}}) \bigr) \bigr) ^{\ast } ( AV_{m+\frac{1}{2}} ) ^{\ast } \\ =&AV_{m+1}. \end{aligned}$$

(2.20)

□

Remark 2.2

The validity of (2.6) and (2.7) for PCIM (2.5) implies that PCIM (2.4) is derived from PCIM (2.5). The computational significance of method (2.5) is limited by the need for knowledge of $P_{R(A)}$.Therefore, the PCIM given in (2.4) is more effective in computational aspects and is preferable for $n_{1}\leq n_{2}$. If $n_{1}>n_{2}$, then the dual version of PCIM (2.4) can be used.

Theorem 2.3

Let $A\in \mathbb{C} _{r}^{n_{1}\times n_{2}}$, if in the initial step In of PCIM (2.4) α satisfies (1.3), then the sequence $\{ V_{m+1} \} $ obtained by the proposed PCIM (2.4) converges to the Moore–Penrose inverse $A^{\dagger }$ uniformly with $p=45$ order of convergence and with asymptotic convergence factor [18] $ACF=\frac{10}{\ln 45}\approx 2.627$. Furthermore, the following error estimate is valid:

$$ \bigl\Vert A^{\dagger }-V_{m+1} \bigr\Vert _{2} \leq \frac{\alpha \Vert R_{0} \Vert _{2}^{45^{m+1}} \Vert A^{\ast } \Vert _{2}}{1- \Vert R_{0} \Vert _{2}}, $$

(2.21)

where $R_{0}=P_{R ( A ) }-\alpha AA^{\ast }$.

Proof

To show the uniform convergence of $\{ V_{m+1} \} $ obtained by PCIM (2.4) to $A^{\dagger }$, we use the approach analogous to the proof of Theorem 3 of [7]. Let us define the residual in the dual version at initial approximation by

$$ \overline{T}_{0}=I-V_{0}A=I-\alpha A^{\ast }A. $$

(2.22)

Also, for any integer $s>0$,

$$ T_{0}^{s}A=A\overline{T}_{0}^{s} $$

(2.23)

and for PCIM (2.4), for its dual version and from Lemma 2.1 using (2.10)–(2.12) for PCIM (2.5), we obtain the residual error at the $(m+1)$th iteration as

$$\begin{aligned}& T_{m+1} =I-AV_{m+1}=T_{m}^{45}, \quad m=0,1,\ldots, \end{aligned}$$

(2.24)

$$\begin{aligned}& \overline{T}_{m+1} =I-V_{m+1}A= \overline{T}_{m}^{45},\quad m=0,1,\ldots, \end{aligned}$$

(2.25)

$$\begin{aligned}& R_{m+1} =P_{R ( A ) }-AV_{m+1}=R_{m}^{45}, \quad m=0,1,\ldots, \end{aligned}$$

(2.26)

respectively. It follows from (2.24)–(2.26) that, for each m,

$$\begin{aligned}& AV_{m+1} =I-T_{0}^{45^{m+1}},\qquad V_{m+1}A=I-\overline{T}_{0} ^{45^{m+1}}, \end{aligned}$$

(2.27)

$$\begin{aligned}& V_{m+1}AV_{m+1} =V_{m+1}- \overline{T}_{0}^{45^{m+1}}V_{m+1}. \end{aligned}$$

(2.28)

From Lemma 2.1, equations (2.6), (2.7) and on the basis of Corollary 4 to Theorem 2 and Theorem 1 of [7] the passage to the limit in (2.27) and (2.28) imply the uniform convergence

$$ AV_{m+1}\rightarrow P_{R ( A ) },\qquad V_{m+1}A\rightarrow P_{R ( A^{\ast } ) },\qquad V_{m+1}AV_{m+1}\rightarrow A ^{\dagger }. $$

(2.29)

Therefore, the sequence $\{ V_{m+1} \} $ converges uniformly to $A^{\dagger }$ with order $p=45$. By denoting

$$ \widehat{V}_{m+\frac{1}{2}}=V_{m+\frac{1}{2}} \bigl( I+T_{m}^{5}+T _{m}^{10}+T_{m}^{15}+T_{m}^{20}+T_{m}^{25}+T_{m}^{30}+T_{m}^{35} \bigr), $$

(2.30)

the corrector step C of (2.4) can be rewritten as

$$ V_{m+1}=V_{m+\frac{1}{2}}+\widehat{V}_{m+\frac{1}{2}}T_{m}^{5}. $$

(2.31)

We denote the error of PCIM (2.4) at the corrector step C by $E_{m+1}=A^{\dagger }-V_{m+1}$. Next, by using that $A^{\dagger }AA ^{\dagger }=A^{\dagger }$ and from Lemma 2.1 equations (2.6) and (2.8), we obtain

$$\begin{aligned} E_{m+1}-E_{m+1}R_{m+1} =&A^{\dagger }-V_{m+1}- \bigl( A^{\dagger }-V _{m+1} \bigr) ( P_{R ( A ) }-AV_{m+1} ) \\ =&V_{m+1} \bigl( AA^{\dagger }-AV_{m+1} \bigr) =V_{m+1}R_{m+1}. \end{aligned}$$

(2.32)

Using (1.3) and on the basis of Theorem 2 of [6], we have $\rho ( R_{0} ) <1$. Since $R_{0}$ is Hermitian, this gives $\rho ( R_{0} ) = \Vert R _{0} \Vert _{2}<1$ and using (2.26) and norm properties

$$ \Vert R_{m+1} \Vert _{2}\leq \Vert R_{m} \Vert _{2}^{45}\leq \Vert R_{0} \Vert _{2}^{45^{m+1}}< 1. $$

(2.33)

From (2.32) using second norm gives

$$ \Vert E_{m+1} \Vert _{2}\leq \frac{ \Vert V_{m+1}R _{m+1} \Vert _{2}}{1- \Vert R_{m+1} \Vert _{2}} $$

(2.34)

(see [7]). By using (2.26), (2.30), and (2.31), it follows that

$$\begin{aligned} V_{m+1}R_{m+1} =&V_{m+1}R_{m}^{40}R_{m}^{5} \\ =& \bigl\{ V_{m+1} \bigl( P_{R ( A ) }+R_{m}^{5}+R_{m} ^{10}+R_{m}^{15}+R_{m}^{20}+R_{m}^{25}+R_{m}^{30}+R_{m}^{35}+R_{m} ^{40} \bigr) \\ &{}- V_{m+1} \bigl( P_{R ( A ) }+R_{m}^{5}+R_{m}^{10}+R _{m}^{15}+R_{m}^{20}+R_{m}^{25}+R_{m}^{30}+R_{m}^{35} \bigr) \bigr\} R_{m}^{5} \\ =& \bigl\{ V_{m+1} \bigl( P_{R ( A ) }+R_{m}^{5}+R_{m} ^{10}+R_{m}^{15}+R_{m}^{20}+R_{m}^{25}+R_{m}^{30}+R_{m}^{35}+R_{m} ^{40} \bigr) \\ &{}-V_{m+\frac{1}{2}} \bigl( P_{R ( A ) }+R_{m}^{5}+R_{m} ^{10}+R_{m}^{15}+R_{m}^{20}+R_{m}^{25}+R_{m}^{30}+R_{m}^{35} \bigr) \\ &{}-\widehat{V}_{m+\frac{1}{2}}T_{m}^{5} \bigl( P_{R ( A ) }+R_{m}^{5}+R_{m}^{10}+R_{m}^{15}+R_{m}^{20}+R_{m}^{25}+R _{m}^{30}+R_{m}^{35} \bigr) \bigr\} R_{m}^{5} \\ =& ( V_{m+1}-\widehat{V}_{m+\frac{1}{2}} ) \bigl( P_{R ( A ) }+R_{m}^{5}+R_{m}^{10}+R_{m}^{15}+R_{m}^{20} \\ &{} +R_{m}^{25}+R_{m}^{30}+R_{m}^{35}+R_{m}^{40} \bigr) R_{m} ^{5}. \end{aligned}$$

(2.35)

From equation (2.35) and using $\Vert P_{R ( A ) } \Vert _{2}=1$ and second norm, we get

$$\begin{aligned} \Vert V_{m+1}R_{m+1} \Vert _{2} \leq & \Vert R_{m} \Vert _{2} ^{5} \bigl( 1+ \Vert R_{m} \Vert _{2}^{5}+ \Vert R _{m} \Vert _{2}^{10}+ \Vert R_{m} \Vert _{2}^{15}+ \Vert R_{m} \Vert _{2}^{20}+ \Vert R_{m} \Vert _{2}^{25} \\ &{} + \Vert R_{m} \Vert _{2}^{30}+ \Vert R_{m} \Vert _{2}^{35}+ \Vert R_{m} \Vert _{2}^{40} \bigr) \Vert V _{m+1}-\widehat{V}_{m+\frac{1}{2}} \Vert _{2}. \end{aligned}$$

(2.36)

From (2.30) and (2.31) and using that $V_{m+\frac{1}{2}}P_{R(A)}^{s}=V_{m+\frac{1}{2}}$ is valid for every positive integer s and also using second norm, it follows that

$$\begin{aligned} \Vert V_{m+1}-\widehat{V}_{m+\frac{1}{2}} \Vert _{2} =& \bigl\Vert V_{m+\frac{1}{2}}+\widehat{V}_{m+\frac{1}{2}} \bigl( T _{m}^{5}-I \bigr) \bigr\Vert _{2} \\ =& \bigl\Vert V_{m+\frac{1}{2}}T_{m}^{40} \bigr\Vert _{2}\leq \Vert V _{m+\frac{1}{2}}R_{m} \Vert _{2} \Vert R_{m} \Vert _{2}^{39}. \end{aligned}$$

(2.37)

Also, in view of (2.33),

$$\begin{aligned} 1- \Vert R_{m+1} \Vert _{2} \geq &1- \Vert R_{m} \Vert _{2}^{45}= \bigl( 1- \Vert R_{m} \Vert _{2}^{5} \bigr) \bigl( 1+ \Vert R_{m} \Vert _{2}^{5}+ \Vert R_{m} \Vert _{2}^{10}+ \Vert R_{m} \Vert _{2}^{15} \\ &{} + \Vert R_{m} \Vert _{2}^{20}+ \Vert R_{m} \Vert _{2}^{25}+ \Vert R_{m} \Vert _{2}^{30}+ \Vert R_{m} \Vert _{2}^{35}+ \Vert R_{m} \Vert _{2}^{40}\bigr) . \end{aligned}$$

(2.38)

From the predictor step P of (2.4) and using $\Vert P _{R ( A ) } \Vert _{2}=1$ and second norm, it follows that

$$ \Vert V_{m+\frac{1}{2}}R_{m} \Vert _{2}\leq \Vert V _{m}R_{m} \Vert _{2} \bigl( 1+ \Vert R_{m} \Vert _{2}+ \Vert R_{m} \Vert _{2}^{2}+ \Vert R_{m} \Vert _{2}^{3}+ \Vert R_{m} \Vert _{2}^{4} \bigr) . $$

(2.39)

Also

$$ 1- \Vert R_{m} \Vert _{2} ^{5}= \bigl( 1- \Vert R_{m} \Vert _{2} \bigr) \bigl( 1+ \Vert R_{m} \Vert _{2}+ \Vert R _{m} \Vert _{2}^{2}+ \Vert R_{m} \Vert _{2}^{3}+ \Vert R_{m} \Vert _{2}^{4} \bigr) . $$

(2.40)

Using (2.34), (2.36)–(2.40) and by recursion, we obtain the following inequalities yielding (2.21):

$$\begin{aligned} \bigl\Vert A^{\dagger }-V_{m+1} \bigr\Vert _{2} \leq &\frac{ \Vert V _{m+1}R_{m+1} \Vert _{2}}{1- \Vert R_{m+1} \Vert _{2}}\leq \frac{ \Vert R_{m} \Vert _{2}^{44} \Vert V _{m+\frac{1}{2}}R_{m} \Vert _{2}}{1- \Vert R_{m} \Vert _{2}^{5}} \leq \frac{ \Vert R_{m} \Vert _{2}^{44} \Vert V_{m}R _{m} \Vert _{2}}{1- \Vert R_{m} \Vert _{2}} \\ \leq& \frac{ \Vert R_{0} \Vert _{2}^{45^{m+1}} \Vert V_{0} \Vert _{2}}{1- \Vert R_{0} \Vert _{2}} =\frac{\alpha \Vert R_{0} \Vert _{2}^{45^{m+1}} \Vert A ^{\ast } \Vert _{2}}{1- \Vert R_{0} \Vert _{2}}. \end{aligned}$$

(2.41)

The proposed PCIM (2.4) requires 5 $Mms$ for the evaluation of the predictor step P and 5 $Mms$ for the corrector step C. Therefore, total 10 $Mms$ are needed giving the asymptotic convergence factor $ACF$ [18] as $\frac{10}{\ln 45} \approx 2.627$. □

Perturbation error analysis

We consider the minimum norm solution of the linear system of equations

$$ Ax=b,\quad A\in \mathbb{R} ^{n_{1}\times n_{2}}, b\in \mathbb{R} ^{n_{1}}, $$

(3.1)

where $b\in R ( A ) $ and $x\in \mathbb{R} ^{n_{2}}$ denotes the unknown solutions and $\operatorname{rank}(A)=r\leq \min ( n_{1},n_{2} ) $. The general least squares solution of (3.1) or minimum norm least squares solution is the solution of the problem

$$ \min_{x\in S} \Vert x \Vert _{2},\quad S= \bigl\{ x\in \mathbb{R} ^{n_{2}}\mid \Vert Ax-b \Vert _{2}=\min \bigr\} . $$

(3.2)

System (3.2) always has a unique solution $x=A^{\dagger }b$ called the pseudoinverse solution [22]. If A is a square nonsingular matrix, then $A^{\dagger }=A^{-1}$. The condition number of A is defined by $\varkappa (A)= \Vert A \Vert _{2} \Vert A ^{\dagger } \Vert _{2}$. Next we consider the following perturbed general least squares problem:

$$ \min_{\widetilde{x}\in \widetilde{S}} \Vert \widetilde{x} \Vert _{2},\quad \widetilde{S}= \bigl\{ \widetilde{x}\in \mathbb{R} ^{n _{2}}\mid \Vert \widetilde{A}\widetilde{x}-\widetilde{b} \Vert _{2}=\min \bigr\} , $$

(3.3)

where $\widetilde{A}=A+\partial A$, $\widetilde{b}=b+\partial b$, and $\widetilde{x}=\widetilde{A}^{\dagger }\widetilde{b}$ is called the regularized pseudoinverse solution and $\partial x=\widetilde{x}-x$, $\partial \tau =\widetilde{\tau }-\tau $ and the residuals are $\widetilde{\tau } =\widetilde{b}-\widetilde{A}\widetilde{x}$, $\tau =b-Ax$. Also, we take into account the perturbations satisfying $\operatorname{rank} ( \widetilde{A} ) =\operatorname{rank} ( A ) =r$ and $\frac{ \Vert \partial A \Vert _{2}}{ \Vert A \Vert _{2}}\leq \varepsilon _{A}$, $\frac{ \Vert \partial b \Vert _{2}}{ \Vert b \Vert _{2}}\leq \varepsilon _{b}$.

Algorithm for approximate regularized pseudoinverse solution

Let $\widetilde{x}_{m+1,\partial }=\widetilde{V}_{m+1}\widetilde{b}$ be the approximate regularized pseudoinverse solution of (3.3), where $\widetilde{V}_{m+1}$ is the approximate Moore–Penrose inverse of Ã obtained by the PCIM given in (2.4) at the $m+1$th iteration by using initial approximation $\widetilde{V}_{0}=\alpha \widetilde{A}^{\ast }$ and α satisfying

$$ 0< \alpha < \frac{2}{\widetilde{\lambda }_{1} ( \widetilde{A}^{ \ast }\widetilde{A} ) } , $$

(3.4)

where

$$ \widetilde{\lambda }_{1} \bigl( \widetilde{A}^{\ast } \widetilde{A} \bigr) \geq \widetilde{\lambda } _{2} \bigl( \widetilde{A}^{\ast }\widetilde{A} \bigr) \geq \cdots\geq \widetilde{ \lambda }_{r} \bigl( \widetilde{A}^{\ast }\widetilde{A} \bigr) >0 $$

(3.5)

are the nonzero eigenvalues of $\widetilde{A}^{\ast }\widetilde{A}$. Hence the residual error at the corresponding step is $\widetilde{\tau }_{m+1}=\widetilde{b}-\widetilde{A}$ $\widetilde{x} _{m+1,\partial }$. Given a predescribed accuracy $\varepsilon >0$, we propose the following algorithm that uses PCIM (2.4) and approximates the regularized pseudoinverse solution $\widetilde{x} _{m+1,\partial }$.

Algorithm A

Choose an initial matrix $\widetilde{V}_{0} =\alpha \widetilde{A}^{\ast }$ under the assumption (3.4).

Step 1:

Let $m=0$. Evaluate $\widetilde{x}_{0,\partial }= \widetilde{V}_{0}\widetilde{b}$ and $\widetilde{\tau }_{0}=$ $\widetilde{b}-\widetilde{A}$ $x_{0,\partial }$, then calculate $\frac{ \Vert \widetilde{\tau }_{0} \Vert _{2}}{ \Vert \widetilde{b} \Vert _{2}}$.

Do Step 2–Step 6 until $\frac{ \Vert \widetilde{\tau } _{m} \Vert _{2}}{ \Vert \widetilde{b} \Vert _{2}} \leq \varepsilon $.

Step 2:

Evaluate $\widetilde{T}_{m}=I-\widetilde{A} \widetilde{V}_{m}$.

Step 3:

Apply the predictor step P of PCIM (2.4) to find $\widetilde{V}_{m+\frac{1}{2}}$.

Step 4:

Evaluate $\widetilde{T}_{m+\frac{1}{2}}=I-\widetilde{A} \widetilde{V}_{m+\frac{1}{2}}$.

Step 5:

Apply the corrector step C of PCIM (2.4) to find $\widetilde{V}_{m+1}$.

Step 6:

Evaluate $\widetilde{x}_{m+1,\partial }=\widetilde{V} _{m+1}\widetilde{b}$ and $\widetilde{\tau }_{m+1}=\widetilde{b}- \widetilde{A}$ $\widetilde{x}_{m+1,\partial }$ respectively. Then calculate $\frac{ \Vert \widetilde{\tau }_{m+1} \Vert _{2}}{ \Vert \widetilde{b} \Vert _{2}}$. Let $m=m+1$.

Step 7:

If m̃ is the number of iterations performed, then $\widetilde{y}_{\widetilde{m}}=\widetilde{x}_{ \widetilde{m},\partial }$ is the approximate regularized pseudoinverse solution satisfying $\frac{ \Vert \widetilde{\tau }_{ \widetilde{m}} \Vert _{2}}{ \Vert \widetilde{b} \Vert _{2}}\leq \varepsilon $.

Theorem 3.1

(Theorem 1.4.6 in [22])

Assume that $\operatorname{rank} ( \widetilde{A} ) =\operatorname{rank} ( A ) =r$ and let $\frac{ \Vert \partial A \Vert _{2}}{ \Vert A \Vert _{2}}\leq \varepsilon _{A}$, $\frac{ \Vert \partial b \Vert _{2}}{ \Vert b \Vert _{2}}\leq \varepsilon _{b}$. Then, if $\ \eta =\varkappa ( A ) \varepsilon _{A}<1$, the perturbations ∂x and ∂τ in the least squares solution x and the residual τ satisfy

$$\begin{aligned}& \Vert \partial x \Vert _{2}\leq \frac{\varkappa ( A ) }{1-\eta } \biggl( \varepsilon _{A} \Vert x \Vert _{2}+\varepsilon _{b}\frac{ \Vert b \Vert _{2}}{ \Vert A \Vert _{2}}+\varepsilon _{A}\varkappa ( A ) \frac{ \Vert \tau \Vert _{2}}{ \Vert A \Vert _{2}} \biggr) +\varepsilon _{A}\varkappa ( A ) \Vert x \Vert _{2}, \end{aligned}$$

(3.6)

$$\begin{aligned}& \Vert \partial \tau \Vert _{2}\leq \varepsilon _{A} \Vert x \Vert _{2} \Vert A \Vert _{2}+\varepsilon _{b} \Vert b \Vert _{2}+\varepsilon _{A}\varkappa ( A ) \Vert \tau \Vert _{2}. \end{aligned}$$

(3.7)

Theorem 3.2

Assume that $\operatorname{rank} ( \widetilde{A} ) =\operatorname{rank} ( A ) =r$ and let $\frac{ \Vert \partial A \Vert _{2}}{ \Vert A \Vert _{2}}\leq \varepsilon _{A}$, $\frac{ \Vert \partial b \Vert _{2}}{ \Vert b \Vert _{2}}\leq \varepsilon _{b}$. Let $\widetilde{x}_{\widetilde{m},\partial }=\widetilde{V}_{\widetilde{m}} \widetilde{b}$ be the approximate regularized pseudoinverse solution (3.3) obtained by Algorithm A. Then, if $\eta =\varkappa ( A ) \varepsilon _{A}<1$, the following inequality holds true:

$$\begin{aligned} \Vert x-\widetilde{x}_{\widetilde{m},\partial } \Vert _{2} \leq & \frac{\varkappa ( A ) }{1-\eta } \biggl( \varepsilon _{A} \Vert x \Vert _{2}+\varepsilon _{b}\frac{ \Vert b \Vert _{2}}{ \Vert A \Vert _{2}}+ \varepsilon _{A}\varkappa ( A ) \frac{ \Vert \tau \Vert _{2}}{ \Vert A \Vert _{2}} \biggr) \\ &{}+\varepsilon _{A}\varkappa ( A ) \Vert x \Vert _{2}+\frac{\alpha ( 1+\varepsilon _{b} ) \Vert \widetilde{R} _{0} \Vert _{2}^{45^{\widetilde{m}}} \Vert \widetilde{A} ^{\ast } \Vert _{2} \Vert A \Vert _{2} \Vert x \Vert _{2}}{1- \Vert \widetilde{R}_{0} \Vert _{2}} , \end{aligned}$$

(3.8)

where x is the pseudoinverse solution (3.2), $\tau =b-Ax$ and $\widetilde{R}_{0}=P_{R ( \widetilde{A} ) }-\widetilde{A}\widetilde{V}_{0}$.

Proof

On the basis of Theorem 2.3 and the inequality in (2.21), and from Theorem 3.1, using (3.6), it follows that

$$\begin{aligned} \Vert x-\widetilde{x}_{\widetilde{m},\partial } \Vert _{2} =& \Vert x-\widetilde{x}_{\widetilde{m},\partial }+ \widetilde{x}-\widetilde{x} \Vert _{2}\leq \Vert x- \widetilde{x} \Vert _{2}+ \Vert \widetilde{x}- \widetilde{x}_{\widetilde{m},\partial } \Vert _{2} \\ =& \Vert x-\widetilde{x} \Vert _{2}+ \bigl\Vert \widetilde{A} ^{\dagger }\widetilde{b}-\widetilde{V}_{\widetilde{m}} \widetilde{b} \bigr\Vert _{2}\leq \Vert x-\widetilde{x} \Vert _{2}+ \bigl\Vert \widetilde{A} ^{\dagger }- \widetilde{V}_{\widetilde{m}} \bigr\Vert _{2} \Vert \widetilde{b} \Vert _{2} \\ \leq &\frac{\varkappa ( A ) }{1-\eta } \biggl( \varepsilon _{A} \Vert x \Vert _{2}+\varepsilon _{b}\frac{ \Vert b \Vert _{2}}{ \Vert A \Vert _{2}}+ \varepsilon _{A}\varkappa ( A ) \frac{ \Vert \tau \Vert _{2}}{ \Vert A \Vert _{2}} \biggr) + \varepsilon _{A}\varkappa ( A ) \Vert x \Vert _{2} \\ &{}+\frac{\alpha \Vert \widetilde{R}_{0} \Vert _{2}^{45^{\widetilde{m}}} \Vert \widetilde{A}^{\ast } \Vert _{2}}{1- \Vert \widetilde{R} _{0} \Vert _{2}} \Vert \widetilde{b} \Vert _{2} \\ \leq &\frac{\varkappa ( A ) }{1-\eta } \biggl( \varepsilon _{A} \Vert x \Vert _{2}+\varepsilon _{b}\frac{ \Vert b \Vert _{2}}{ \Vert A \Vert _{2}}+ \varepsilon _{A}\varkappa ( A ) \frac{ \Vert \tau \Vert _{2}}{ \Vert A \Vert _{2}} \biggr) + \varepsilon _{A}\varkappa ( A ) \Vert x \Vert _{2} \\ &{}+\frac{\alpha ( 1+\varepsilon _{b} ) \Vert \widetilde{R} _{0} \Vert _{2}^{45^{\widetilde{m}}} \Vert \widetilde{A} ^{\ast } \Vert _{2} \Vert A \Vert _{2} \Vert x \Vert _{2}}{1- \Vert \widetilde{R}_{0} \Vert _{2}}. \end{aligned}$$

□

Numerical results

In this section all the computations are performed using a personal computer with properties AMD Ryzen 7 1800X Eight Core Processor 3.60 GHz. Calculations are carried by Mathematica program in double precision for Example 1 and Example 2 and with 900 precision for Example 3. The tables adopt the following notations:

$\mathrm{TCS} _{AL}$ is the total solution cost in seconds of Algorithm A.

$\mathrm{TCSSP} _{AL}$ is the total solution cost in seconds of Algorithm A for successive perturbations.

Example 1

(One-dimensional image restoration problem (see [18, 23]))

In this test problem we consider the first kind Fredholm integral equation

$$ \int _{\frac{-\pi }{2}}^{\frac{\pi }{2}}K(\theta ,\varphi )f( \varphi )\,d \varphi =g(\theta ),\quad \frac{-\pi }{2}\leq \theta \leq \frac{\pi }{2}. $$

(4.1)

The kernel $K(\theta ,\varphi )$ and the function $f(\varphi )$ are as follows:

$$ K(\theta ,\varphi )= \biggl[(\cos \theta +\cos \varphi )\frac{\sin \omega }{ \omega } \biggr]^{2}\text{,} $$

(4.2)

with

$$\begin{aligned}& \begin{gathered} \omega =\pi (\sin \theta +\sin \varphi ). \\ f(\varphi )=\exp \bigl(-4(\varphi +0.5)^{2} \bigr)+2\exp \bigl(-4( \varphi -0.5)^{2} \bigr) . \end{gathered} \end{aligned}$$

(4.3)

By taking the quadrature nodes $\varphi _{j}=-\frac{\pi }{2}+\frac{ ( j-0.5 ) \pi }{n_{2}}$, $j=1,\ldots,n_{2}$, and the points $\theta _{i}=-\frac{\pi }{2}+\frac{ ( i-0.5 ) \pi }{n_{1}}$, $i=1,\ldots,n_{1}$, the discretization of (4.1)–(4.3) gives the discrete ill-posed problem $Au_{h}=b$. The right-hand side vector b is calculated by multiplying A with u, where u is the trace of $f(\varphi )$ on the grid points $\varphi _{j},j=1,\ldots,n_{2}$. We apply the proposed Algorithm A by using successive perturbations to obtain the regularized pseudoinverse solution for the perturbed systems $(A+\partial A)y^{k}=b+\partial ^{k}b$, where $∂ A i, j = {δ 1 if i = j 0 if i ≠ j}$ and $\partial ^{k}b=(\delta ^{k},\ldots,\delta ^{k})^{T}\in R ^{n_{1}} $ for different values of the smoothing parameter $\delta ^{1}$, and $\delta ^{k}=0.999\delta ^{k-1}$, $k=2$,…,6. Let $\widetilde{y}_{\widetilde{m}_{k}}^{k}$, $k=1,\ldots,6$, be the approximate regularized pseudoinverse solution of

$$ \min_{\widetilde{y}^{k}\in \widetilde{S}^{k}} \bigl\Vert \widetilde{y} ^{k} \bigr\Vert _{2},\quad \widetilde{S}^{k}= \bigl\{ \widetilde{y}^{k}\in R^{n_{2}}\mid \bigl\Vert (A+\partial A) \widetilde{y}^{k}- \bigl( b+\partial ^{k}b \bigr) \bigr\Vert _{2}= \min \bigr\} , $$

(4.4)

obtained by the given Algorithm A by performing $\widetilde{m}_{k}$ iterations for an accuracy of $\frac{ \Vert \widetilde{\tau } _{\widetilde{m}_{k}} \Vert _{2}}{ \Vert \widetilde{b} \Vert _{2}}\leq 5\times 10^{-7}$ for the corresponding perturbed system. Table 1 presents the total iteration number $\widetilde{m}= \sum_{k=1}^{6}\widetilde{m}_{k}$ performed, $\mathrm{TCSSP} _{AL}$ and the relative $L_{2}$ norm of the errors using Algorithm A when $n_{1}=400$ and $n_{2}=800$.

Table 1 $\mathrm{TCSSP} _{AL}$, iteration numbers, relative $L_{2}$ norm of the errors obtained by Algorithm A for Example 1

Full size table

Example 2

(Test problems from Harwell–Boeing collection)

The experiments are carried out on nine of the matrices from the Harwell–Boeing test collection (Duff, Grimes, and Lewis, see [22]). The numerical values of the ABB and ASH matrices are random numbers uniformly distributed in $[-1,1]$, while WELL and ILLC have their original values. The WELL and ILLC matrices have the same nonzero structure but different numerical values. A set of consistent least squares problems is defined by taking the exact solution to be $u=(1,\ldots,1)^{T}\in R^{n_{2}}$, and the right-hand side b is obtained by multiplying the coefficient matrix A by u. We apply the proposed Algorithm A using the dual version of PCIM for obtaining the regularized pseudoinverse solution for each test problem, where $∂ A i, j = {δ if i = j 0 if i ≠ j}$ and $\partial b=(\delta ,\ldots,\delta )^{T}\in R^{n_{1}}$ for different values of the smoothing parameter δ. Let $\widetilde{y}_{\widetilde{m}}$ be the approximate regularized pseudoinverse solution obtained by the given Algorithm A using the dual version of PCIM by performing m̃ iterations for an accuracy of $\frac{ \Vert \widetilde{\tau }_{\widetilde{m}} \Vert _{2}}{ \Vert \widetilde{b} \Vert _{2}}\leq \varepsilon $. Table 2 presents the test problems, description of the problems, and the size of the matrices, $\mathrm{TCS} _{AL}$, iteration numbers m̃ performed, and the relative $L_{2}$ norm of the errors obtained for the test problems in Example 2 when $\delta =\varepsilon =5\times 10^{-15}$. Figure 1 illustrates the relative $L_{2}$ norm of errors for the test problems in Example 2 with respect to δ. The obtained numerical results justify the theoretical results in Theorem 3.2.

Table 2 $\mathrm{TCS} _{AL}$, iteration numbers, and relative $L_{2}$ norm of the errors obtained by Algorithm A for the test problems of Example 2

Full size table

Example 3

(A benchmark example [17])

With this example we aim to numerically compute the theoretical order of convergence of the proposed PCIM (2.4) using the known exact Moore–Penrose inverse. In this example A and $A^{\dagger }$ are

$$ A= \begin{pmatrix} 1 & 0 & 0 & -6 \\ 2 & 6 & 0 & -6 \\ 7 & 8 & 9 & -6 \end{pmatrix} ,\qquad A^{\dagger }= \begin{pmatrix} \frac{28}{1931} & \frac{-143}{3862} & \frac{84}{1931} \\ \frac{-653}{3862} & \frac{1335}{7724} & \frac{-14}{1931} \\ \frac{57}{1931} & \frac{-249}{1931} & \frac{171}{1931} \\ \frac{-1903}{11\mbox{,}586} & \frac{-143}{23\mbox{,}172} & \frac{14}{1931} \end{pmatrix}, $$

and we use

$$ \widetilde{p}=\ln \biggl( \frac{ \Vert V_{m+1}-A^{\dagger } \Vert _{2}}{ \Vert V_{m}-A^{\dagger } \Vert _{2}} \biggr) \biggm/ \ln \biggl( \frac{ \Vert V_{m}-A^{\dagger } \Vert _{2}}{ \Vert V_{m-1}-A^{\dagger } \Vert _{2}} \biggr) $$

to estimate the order of convergence p. The maximum norm of the errors and the calculated order of convergence p̃ are presented in Table 3. The fourth column of Table 3 shows that the numerically calculated order of convergence p̃ of PCIM (2.4) is at least 45.

Table 3 Second norm errors and the estimated order of convergence of PCIM for Example 3

Full size table

Conclusion

The PCIM with convergence order $p=45$ is proposed for computing the Moore–Penrose inverse of a nonzero matrix $A\in \mathbb{C} _{r}^{n _{1}\times n_{2}}$. Also we give the Algorithm A which uses the PCIM and approximates the regularized pseudoinverse solution of general least squares problem. The economical computational efficiency and stability of Algorithm A are useful for the numerical regularized pseudoinverse solution of difficult problems such as the first kind Fredholm integral equations.

References

1.
Penrose, R.: A generalized inverse for matrices. Proc. Camb. Philos. Soc. 51, 406–413 (1955)
Article Google Scholar
2.
Ben-Israel, A., Charnes, A.: Contributions to the theory of generalized inverses. J. Soc. Ind. Appl. Math. 11, 667–699 (1963)
MathSciNet Article Google Scholar
3.
Ben-Israel, A., Cohen, D.: On iterative computation of generalized inverses and associated projections. SIAM J. Numer. Anal. 3, 410–419 (1966)
MathSciNet Article Google Scholar
4.
Ben-Israel, A., Greville, T.N.E.: Generalized Inverses: Theory and Applications. Springer, Berlin (2003)
Google Scholar
5.
Ben-Israel, A.: An iterative method for computing the generalized inverse of an arbitrary matrix. Math. Compet. 19, 452 (1965)
MathSciNet Article Google Scholar
6.
Ben-Israel, A.: A note on an iterative method for generalized inversion of matrices. Math. Compet. 20, 439–440 (1966)
Article Google Scholar
7.
Petryshyn, W.V.: On generalized inverses and on the uniform convergence of $( I-\beta K ) ^{n}$ with application to iterative methods. J. Math. Anal. Appl. 18, 417–439 (1967)
MathSciNet Article Google Scholar
8.
Zlobec, S.: On computing the generalized inverse of a linear operator. Glasnik Mat-Fiz. Astronom. Ser. II Drushtvo Mat. Fiz. Hrvatske 22, 265–271 (1967)
MathSciNet MATH Google Scholar
9.
Amat, S., Busquier, S., Gutierrez, J.M.: Geometric constructions of iterative functions to solve nonlinear equations. J. Comput. Appl. Math. 157(1), 197–205 (2003)
MathSciNet Article Google Scholar
10.
Li, H.-B., Huang, T.-Z., Zhang, Y., Liu, X.-P., Gu, T.-X.: Chebyshev-type methods and preconditioning techniques. Appl. Math. Comput. 218(2), 260–270 (2011)
MathSciNet MATH Google Scholar
11.
Stickel, E.: On a class of high order methods for inverting matrices. Z. Angew. Math. Mech. 67(7), 334–336 (1987)
MathSciNet Article Google Scholar
12.
Herzberger, J.: Efficient algorithms for the inclusion of the inverse matrix using error-bounds for hyperpower methods. Computing 46, 279–288 (1991)
MathSciNet Article Google Scholar
13.
Li, W., Li, Z.: A family of iterative methods for computing the approximate inverse of a square matrix and inner inverse of a non-square matrix. Appl. Math. Comput. 215, 3433–3442 (2010)
MathSciNet MATH Google Scholar
14.
Chen, H., Wang, Y.: A family of higher-order convergent iterative methods for computing the Moore-Penrose inverse. Appl. Math. Comput. 218, 4012–4016 (2011)
MathSciNet MATH Google Scholar
15.
Weiguo, L., Juan, L., Tiantian, Q.: A family of iterative methods for computing Moore-Penrose inverse of a matrix. Linear Algebra Appl. 438, 47–56 (2013)
MathSciNet Article Google Scholar
16.
Soleymani, F.: An efficient and stable Newton-type iterative method for computing generalized inverse $A_{T,S}^{(2)}$. Numer. Algorithms 69(3), 569–578 (2015)
MathSciNet Article Google Scholar
17.
Soleymani, F., Stanimirović, P.S., Haghani, F.K.: On hyperpower family of iterations for computing outer inverses possessing high efficiencies. Linear Algebra Appl. 484, 477–495 (2015)
MathSciNet Article Google Scholar
18.
Buranay, S.C., Subasi, D., Iyikal, O.C.: On the two classes of high order convergent methods of approximate inverse preconditioners for solving linear systems. Numer. Linear Algebra Appl. 24(6), Article ID e2111 (2017)
MathSciNet Article Google Scholar
19.
Buranay, S.C., Iyikal, O.C.: Approximate Schur-block ILU preconditioners for regularized solution of discrete ill-posed problems. Math. Probl. Eng. 2019, Article ID 1912535 (2019). https://doi.org/10.1155/2019/1912535
MathSciNet Article Google Scholar
20.
Wang, Y.-H., Wu, G.-C., Baleanu, D.: Variational iteration method-a promising technique for constructing equivalent integral equations of fractional order. Cent. Eur. J. Phys. 11(10), 1392–1398 (2013)
Google Scholar
21.
Wu, G.-C., Baleanu, D.: Variational iteration method for the Burgers’ flow with fractional derivatives-new Lagrange multipliers. Appl. Math. Model. 37, 6183–6190 (2013)
MathSciNet Article Google Scholar
22.
Björck, A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Google Scholar
23.
Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34(4), 561–580 (1992)
MathSciNet Article Google Scholar

Download references

Availability of data and materials

In Example 2 data are used from “matrix market” a repository organized by the National Institute of Standards and Technology to support this study.

Funding

Not applicable.

Author information

Affiliations

Department of Mathematics, Faculty of Arts and Sciences, Eastern Mediterranean University, Famagusta, North Cyprus, Turkey
Suzan C. Buranay & Ovgu C. Iyikal

Authors

Suzan C. Buranay
View author publications
You can also search for this author in PubMed Google Scholar
Ovgu C. Iyikal
View author publications
You can also search for this author in PubMed Google Scholar

Contributions

The authors contributed equally to the writing of this paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Suzan C. Buranay.

Ethics declarations

Competing interests

The authors declare that they have no competing interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Cite this article

Buranay, S.C., Iyikal, O.C. A predictor-corrector iterative method for solving linear least squares problems and perturbation error analysis. J Inequal Appl 2019, 203 (2019). https://doi.org/10.1186/s13660-019-2154-z

Download citation

Received: 15 March 2019
Accepted: 09 July 2019
Published: 20 July 2019
DOI: https://doi.org/10.1186/s13660-019-2154-z

MSC

65F20
65F30

Keywords

Linear least squares problems
Perturbation error analysis
Moore–Penrose inverse
Image restoration problem
Matrix algorithms