 Research
 Open Access
 Published:
Adaptively relaxed algorithms for solving the split feasibility problem with a new step size
Journal of Inequalities and Applications volume 2014, Article number: 448 (2014)
Abstract
In the present paper, we propose several kinds of adaptively relaxed iterative algorithms with a new step size for solving the split feasibility problem in real Hilbert spaces. The proposed algorithms never terminate, while the known algorithms existing in the literature may terminate. Several weak and strong convergence theorems of the proposed algorithms have been established. Some numerical experiments are also included to illustrate the effectiveness of the proposed algorithms.
MSC:46E20, 47J20, 47J25.
1 Introduction
Since its inception in 1994, the split feasibility problem SFP [1] has been attracting researchers’ interest [2, 3] due to its extensive applications in signal processing and image reconstruction [4], with particular progress in intensitymodulated radiation therapy [5, 6].
Let {H}_{1} and {H}_{2} be real Hilbert spaces, C and Q be nonempty closed convex subsets of {H}_{1} and {H}_{2}, respectively, and A:{H}_{1}\to {H}_{2} a bounded linear operator. Then SFP can be formulated as finding a point \stackrel{\u02c6}{x} with the property
The set of solutions for SFP (1.1) is denote by \mathrm{\Gamma}=C\cap {A}^{1}(Q).
Over the past two decades years or so, the researchers invested and designed various iterative algorithms for solving SFP (1.1); see [6–13]. The most popular algorithm, among them, is Byrne’s CQ algorithm, which generates a sequence \{{x}_{n}\} by the recursive procedure
where the step size {\tau}_{n} is chosen in the open interval (0,2/{\parallel A\parallel}^{2}), while {P}_{C} and {P}_{Q} are the orthogonal projections onto C and Q, respectively.
We remark in passing that Byrne’s CQ algorithm (1.2) is indeed a special case of the classical gradient projection method (GPM). To see this, let us define f:{H}_{1}\to \mathrm{R} by
then the convex objective f is differentiable and has a Lipschitz gradient given by
We consider the following convex minimization problem:
It is well known that \stackrel{\u02c6}{x}\in C is a solution of problem (1.5) if and only if
Also, we know that (1.6) holds true if and only if
Note that if \mathrm{\Gamma}\ne \mathrm{\varnothing}, then \stackrel{\u02c6}{x}\in \mathrm{\Gamma}\iff f(\stackrel{\u02c6}{x})={min}_{x\in C}f(x)\iff \text{(1.6) holds}\iff \text{(1.7) holds}. Consequently, we can utilize the classical gradient projection method (GPM) below to solve SFP (1.1):
where {\tau}_{n}\in (0,2/L), while L is the Lipschitz constant of ∇f. Noting that L={\parallel A\parallel}^{2}, we see immediately that (1.8) is exactly CQ algorithm (1.2).
We note that, in algorithms (1.2) and (1.8) mentioned above, the choice of the step size {\tau}_{n} depends heavily on the operator (matrix) norm \parallel A\parallel. This means that for actual implementation of CQ algorithm (1.2), one has first to know at least an upper bound of the operator (matrix) norm \parallel A\parallel, which is in general difficult. To overcome this difficulty, several authors proposed several various of adaptive methods, which permit the step size {\tau}_{n} to be selected selfadaptively; see [7–9].
Yang [10] considered the following step size:
where \{{\rho}_{n}\} is a sequence of positive real numbers such that
Very recently, López et al. [11] introduced another choice of the step size sequence \{{\tau}_{n}\} as follows:
where \{{\rho}_{n}\} is chosen in the open interval (0,4). By virtue of the step size (1.11), López et al. [11] introduced four kinds of algorithms for solving SFP (1.1).
We observe that if \mathrm{\nabla}f({x}_{n})=0 for some n\ge 1, then the algorithms introduced by López et al. [11] have to terminate in the n th step of iterations. In this case {x}_{n} is not necessarily a solution of the SFP (1.1), since {x}_{n} may be not in C, Algorithm 4.1 in [11] is such a case. To make up the flaw, we introduce a new choice of the step size sequence \{{\tau}_{n}\} as follows:
where \{{\rho}_{n}\} is chosen in the open interval (0,4) and \{{\sigma}_{n}\} is a sequence of positive numbers in (0,1), while {f}_{n} and \mathrm{\nabla}{f}_{n} are given by, respectively,
where \{{Q}_{n}\} will be defined in Section 3.
The purpose of this paper is to introduce a new choice of the step size sequence \{{\tau}_{n}\} that makes the associated algorithms never terminate. A new stop rule is also given, which ensures that the (n+1)th iteration {x}_{n+1} is a solution of SFP (1.1) and the iterative process stops. Several weak and strong convergence results are presented. Numerical experiments are included to illustrate the effectiveness of the proposed algorithms and the applications in signal processing of the CQ algorithm with the step size selected in this paper.
The rest of this paper is organized as follows. In the next section, some necessary concepts and important facts are collected. The weak and strong convergence theorems of the proposed algorithms with step size (1.12) are established in Section 3. Finally in Section 4, we provide some numerical experiments to illustrate the effectiveness and applications of the proposed algorithms with step size (1.12) to inverse problems arising from signal processing.
2 Preliminaries
Throughout this paper, we assume that SFP (1.1) is consistent, i.e., \mathrm{\Gamma}\ne \mathrm{\varnothing}. We denote by ℝ the set of real numbers. Let {H}_{1} and {H}_{2} real Hilbert spaces and the letter I the identity mapping on {H}_{1} or {H}_{2}. If f:{H}_{1}\to \mathbb{R} is a differentiable (subdifferentiable) functional, then we denote by ∇f (∂f) the gradient (subdifferential) of f. Given a sequence \{{x}_{n}\} in H, {w}_{w}({x}_{n}) (resp. ‘{x}_{n}\rightharpoonup x’) denotes the strong (resp. weak) convergence of \{{x}_{n}\} to x. The symbols \u3008\cdot ,\cdot \u3009 and \parallel \cdot \parallel denote inner product and norm of Hilbert spaces {H}_{1} and {H}_{2}, respectively. Let T:{H}_{1}\to {H}_{1} be a mapping. We use Fix(T) to denote the set of fixed points of T. We also denote by dom(T) the domain of T.
Some equalities in Hilbert space {H}_{1} play very important roles for solving linear and nonlinear problems arising from real world.
It is well known that in a real Hilbert space {H}_{1}, the following two equalities hold:
for all x,y\in {H}_{1}.
for all x,y\in {H}_{1} and t\in \mathbb{R}.
Recall that a mapping T:dom(T)\subset {H}_{1}\to {H}_{1} is said to be

(i)
nonexpansive if
\parallel TxTy\parallel \le \parallel xy\parallel ,(2.3)
for all x,y\in dom(T);

(ii)
firmly nonexpansive if
{\parallel TxTy\parallel}^{2}\le {\parallel xy\parallel}^{2}{\parallel (IT)x(IT)y\parallel}^{2},(2.4)
for all x,y\in dom(T);

(iii)
λaveraged if there exist some \lambda \in (0,1) and another nonexpansive mapping S:{H}_{1}\to {H}_{2} such that
T=(1\lambda )I+\lambda S.(2.5)
The following proposition describes the characterizations of firmly nonexpansive mappings (see [12]).
Proposition 2.1 Let T:dom(T)\subset {H}_{1}\to {H}_{1} be a mapping. Then the following statements are equivalent.

(i)
T is firmly nonexpansive;

(ii)
IT is firmly nonexpansive;

(iii)
{\parallel TxTy\parallel}^{2}\le \u3008xy,TxTy\u3009 for all x,y\in {H}_{1};

(iv)
T is \frac{1}{2}averaged;

(v)
2TI is nonexpansive.
Recall that the metric (nearest point) projection form {H}_{1} onto a nonempty closed convex subset C of {H}_{1} is defined as follows: for each x\in {H}_{1}, there exists a unique point {P}_{C}x\in C with the property:
Now we list some basic properties of {P}_{C} below; see [12] for details.
Proposition 2.2
(p_{1}) Given x\in H and z\in C. Then z={P}_{C}x if and only if we have the inequality
(p_{2})
(p_{3})
for all x,y\in {H}_{1}.
(p_{4}) 2{P}_{C}I is nonexpansive;
(p_{5})
in particular,
(p_{6})
From (p_{2}), (p_{3}), and (p_{4}), we see immediately that both {P}_{C} and (I{P}_{C}) are firmly nonexpansive and \frac{1}{2}averaged.
Recall that a function f:{H}_{1}\to \mathbb{R} is called convex if
It is well known that a differentiable function f is convex if and only if we have the relation
Recall that an element \xi \in {H}_{1} is said to be a subgradient of f:{H}_{1}\to \mathbb{R} at x if
If the function f:{H}_{1}\to \mathbb{R} has at least one subgradient at x, it is said to be subdifferentiable at x. The set of subgradients of f at the point x is called the subdifferential of f at x, and is denoted by \partial f(x). A function f is called subdifferentiable if it is subdifferentiable at every x\in {H}_{1}. If f is convex and differentiable, then \partial f(x)=\{\mathrm{\nabla}f(x)\} for every x\in {H}_{1}. A function f is called subdifferentiable if it is subdifferentiable at every x\in {H}_{1}. If f is convex and differentiable, then \partial f(x)=\{\mathrm{\nabla}f(x)\} for every x\in {H}_{1}. A function f:{H}_{1}\to \mathbb{R} is said to be weakly lower semicontinuous (wlsc) at x if {x}_{n}\rightharpoonup x implies
f is said to be wlsc on {H}_{1} if it is wlsc at every point x\in {H}_{1}.
It is well known that for a convex function f:{H}_{1}\to \mathbb{R}, it is wlsc on {H}_{1} if and only if it is lsc on {H}_{1}.
It is an easy exercise to prove the following conclusions (see [13, 14]).
Proposition 2.3 Let f be given as in (1.3). Then the following conclusions hold.

(i)
f is convex and differentiable;

(ii)
\mathrm{\nabla}f(x)={A}^{\ast}(I{P}_{Q})Ax, x\in {H}_{1};

(iii)
f is wlsc on {H}_{1};

(iv)
∇f is {\parallel A\parallel}^{2}Lipschitz:
\parallel \mathrm{\nabla}f(x)\mathrm{\nabla}f(y)\parallel \le {\parallel A\parallel}^{2}\parallel xy\parallel ,\phantom{\rule{1em}{0ex}}x,y\in H.
The concept of Fejér monotonicity plays a key role in establishing weak convergence theorems. Recall that a sequence \{{x}_{n}\} in {H}_{1} is said to be Fejér monotone with respect to (w.r.t.) a nonempty closed convex subset C in {H}_{1} if
Proposition 2.4 (see [11, 15])
Let C be a nonempty closed convex in {H}_{1}. If the sequence \{{x}_{n}\} is Fejér monotone w.r.t. C, then the following hold:

(i)
{x}_{n}\rightharpoonup \stackrel{\u02c6}{x} if and only if {w}_{w}({x}_{n})\subset C;

(ii)
the sequence \{{P}_{C}{x}_{n}\} converges strongly;

(iii)
if {x}_{n}\rightharpoonup \stackrel{\u02c6}{x}\in C, then \stackrel{\u02c6}{x}={lim}_{n}{P}_{C}{x}_{n}.
Proposition 2.5 (see [16])
Let \{{\alpha}_{n}\} be a sequence of nonnegative real numbers such that
where \{{t}_{n}\} is a sequence in (0,1) and {b}_{n} is a sequence in ℝ such that

(i)
{\sum}_{n=1}^{\mathrm{\infty}}{t}_{n}=\mathrm{\infty};

(ii)
{\overline{lim}}_{n}{b}_{n}\le 0 or {\sum}_{n=1}^{\mathrm{\infty}}{t}_{n}{b}_{n}<\mathrm{\infty}. Then {\alpha}_{n}\to 0 (n\to \mathrm{\infty}).
3 Main results
Let c:{H}_{1}\to \mathbb{R} and q:{H}_{2}\to \mathbb{R} be convex functions and define level sets of c and q as follows:
Assume that both c and q are subdifferentiable on {H}_{1} and {H}_{2}, respectively, and that ∂c and ∂q are bounded mappings. Given an arbitrary initial data {x}_{1}\in {H}_{1}. Assume that {x}_{n} is the current value for n\ge 1. We introduce two sequences of halfspaces as follows:
where {\xi}_{n}\in \partial c({x}_{n}), and
where {\eta}_{n}\in \partial q(A{x}_{n}). Clearly, C\subseteq {C}_{n} and Q\subseteq {Q}_{n} for all n\ge 1.
Construct {x}_{n+1} via the formula
where \{{\tau}_{n}\} is given as (1.12),
and
More precisely, we introduce the following relaxed CQ algorithm in an adaptive way.
Algorithm 3.1 Choose an initial data {x}_{1}\in {H}_{1} arbitrarily. Assume that the n th iterate {x}_{n} has been constructed then we compute the (n+1)th iteration {x}_{n+1} via the formula:
where the step size {\tau}_{n} is chosen in such a way that
with 0<{\rho}_{n}<4 and 0<{\sigma}_{n}<1. If {x}_{n+1}={x}_{n} for some n\ge 1, then {x}_{n} is a solution of the SFP (1.1) and the iterative process stops; otherwise, we set n:=n+1 and go on to (3.7) to compute the next iteration {x}_{n+2}.
We remark in passing that if {x}_{n+1}={x}_{n} for some n\ge 1, then {x}_{m}={x}_{n} for all m\ge n+1, consequently, {lim}_{m\to \mathrm{\infty}}{x}_{m}={x}_{n} is a solution of SFP (1.1). Thus, we may assume that the sequence \{{x}_{n}\} generated by Algorithm 3.1 is infinite.
Theorem 3.2 Assume that {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})\ge \rho >0. Then the sequence \{{x}_{n}\} generated by Algorithm 3.1 converges weakly to a solution \stackrel{\u02c6}{x} of SFP (1.1), where \stackrel{\u02c6}{x}={lim}_{n\to \mathrm{\infty}}{P}_{\mathrm{\Gamma}}{x}_{n}.
Proof Let z\in \mathrm{\Gamma} be fixed, and set {y}_{n}={x}_{n}{\tau}_{n}\mathrm{\nabla}{f}_{n}({x}_{n}). By virtue of (2.1), (3.7), and Proposition 2.2(p_{6}), we have
In view of Proposition 2.2, we know that I{P}_{{Q}_{n}} are firmly nonexpansive for all n\ge 1, and from this one derives
from which it turns out that
which in turn allows us to deduce the following conclusions:

(i)
\{{x}_{n}\} is Fejér monotone w.r.t. Γ; in particular,

(ii)
\{{x}_{n}\} is a bounded sequence;

(iii)
{\sum}_{n=1}^{\mathrm{\infty}}{\rho}_{n}(4{\rho}_{n}){f}_{n}^{2}({x}_{n})/{(\parallel \mathrm{\nabla}{f}_{n}({x}_{n})\parallel +{\sigma}_{n})}^{2}<\mathrm{\infty}; and

(iv)
\sum _{n=1}^{\mathrm{\infty}}{\parallel {y}_{n}{P}_{{C}_{n}}{y}_{n}\parallel}^{2}<\mathrm{\infty}.(3.12)
By Proposition 2.4(i), to show that {x}_{n}\rightharpoonup x, it suffices to show that {w}_{w}({x}_{n})\subset \mathrm{\Gamma}. To see this, take {x}^{\ast}\in {w}_{w}({x}_{n}) and let \{{x}_{{n}_{k}}\} be a sequence of \{{x}_{n}\} weakly converging to {x}^{\ast}. By our assumption that {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})\ge \rho >0, without loss of generality, we can assume that {\rho}_{n}(4{\rho}_{n})\ge \frac{\rho}{2} for all n\ge 1. It follows from (3.12) (iii) that
Note that \parallel \mathrm{\nabla}{f}_{n}({x}_{n})\parallel +{\sigma}_{n}\le {\parallel A\parallel}^{2}\parallel {x}_{n}z\parallel +1 for z\in \mathrm{\Gamma}. This, together with (3.13), implies that {f}_{n}({x}_{n})\to 0, that is, \parallel (I{P}_{{Q}_{n}})A{x}_{n}\parallel \to 0. By our assumption that ∂q is a bounded mapping, we see that there exists a constant M>0 such that \parallel {\eta}_{n}\parallel \le M, \mathrm{\forall}{\eta}_{n}\in \partial q(A{x}_{n}).
Since {P}_{{Q}_{n}}(A{x}_{n})\in {Q}_{n}, by the definition of {Q}_{n}, we have
Noting that A{x}_{{n}_{k}}\rightharpoonup A\stackrel{\u02c6}{x} and using the wlsc of q, we have q(A{x}^{\ast})\le {\underline{lim}}_{k}q(A{x}_{{n}_{k}})\le 0, which implies that A{x}^{\ast}\in Q. We next prove {x}^{\ast}\in C. Firstly, from (3.12) (iv), we know that \parallel {y}_{n}{P}_{{C}_{n}}{y}_{n}\parallel \to 0. Notice that
we have
Since ∂c is a bounded mapping, we have {M}_{1}>0 such that
Since {P}_{{C}_{n}}({x}_{n})\in {C}_{n}, by the definition of {C}_{n}, we have
Then wlsc of C implies that c({x}^{\ast})\le {\underline{lim}}_{k}c({x}_{{n}_{k}})\le 0, thus {x}^{\ast}\in C and {w}_{w}({x}_{n})\subset \mathrm{\Gamma}, completing the proof. □
We introduce a little more general algorithm as follows.
Algorithm 3.3 Choose an initial data {x}_{1}\in {H}_{1} arbitrarily. Assume that the n th iteration {x}_{n} has been constructed; then we compute the (n+1)th iteration {x}_{n+1} via the formula:
where the step size \{{\tau}_{n}\} is as before and \{{\beta}_{n}\} is a sequence in (0,1) satisfying {\overline{lim}}_{n}{\beta}_{n}<1. If {x}_{n+1}={x}_{n} for some n\ge 1, then {x}_{n} is a solution of the SFP (1.1) and the iterative process stops; otherwise, we set n:=n+1 and go on to (3.15) to compute the next iteration {x}_{n+2}.
We have the following weak convergence theorem.
Theorem 3.4 Assume that {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})\ge \rho >0. Then the sequence \{{x}_{n}\} generated by Algorithm 3.3 converges weakly to a solution \stackrel{\u02c6}{x} of the SFP (1.1) where \stackrel{\u02c6}{x}={lim}_{n\to \mathrm{\infty}}{P}_{\mathrm{\Gamma}}{x}_{n}.
Proof Let z\in \mathrm{\Gamma} be fixed and set {y}_{n}={x}_{n}{\tau}_{n}\mathrm{\nabla}{f}_{n}({x}_{n}). By virtue of (2.1), (2.2), (3.15), (3.10), and Proposition 2.2(p_{6}), we have
which implies that

(i)
\{{x}_{n}\} is Fejér monotone w.r.t. Γ; in particular,

(ii)
\{{x}_{n}\} is a bounded sequence;

(iii)
{\sum}_{n=1}^{\mathrm{\infty}}(1{\beta}_{n}){\rho}_{n}(4{\rho}_{n}){f}_{n}^{2}({x}_{n})/{(\parallel \mathrm{\nabla}{f}_{n}({x}_{n})\parallel +{\sigma}_{n})}^{2}<\mathrm{\infty}; and

(iv)
{\sum}_{n=1}^{\mathrm{\infty}}(1{\beta}_{n}){\parallel {y}_{n}{P}_{{C}_{n}}{y}_{n}\parallel}^{2}<\mathrm{\infty}.
By our assumptions on \{{\beta}_{n}\} and \{{\rho}_{n}\}, we have \frac{{f}_{n}({x}_{n})}{\parallel \mathrm{\nabla}{f}_{n}({x}_{n})\parallel +{\sigma}_{n}}\to 0 and {y}_{n}{P}_{{C}_{n}}{y}_{n}\to 0, the rest of the arguments follow exactly form the corresponding parts of Theorem 3.2, we omit its details. This completes the proof. □
We remark that Theorem 3.4 generalizes Theorem 3.2, that is, if we take {\beta}_{n}\equiv 0 in Theorem 3.4, then we can obtain Theorem 3.2. It is really interesting work to compare convergence rate of Algorithms 3.1 and 3.3.
Generally speaking, Algorithms 3.1 and 3.3 have only the weak convergence in the frame work of infinitedimensional spaces, and therefore the modifications of Algorithms 3.1 and 3.3 are needed in order to realize the strong convergence. Considerable efforts have been made and several interesting results have been reported recently; see [17–20]. Below is our modification of Algorithms 3.1 and 3.3.
Algorithm 3.5 Choose an arbitrary initial data {x}_{1}\in {H}_{1}. Assume that the n th iteration {x}_{n}\in {H}_{1} has been constructed. Set
with the step size {\tau}_{n} given by (1.12), and define two halfspaces {Y}_{n} and {Z}_{n} by
The (n+1)th iterate {x}_{n+1} is then constructed in the formula:
If {x}_{n+1}={x}_{n} for some n\ge 1, then {x}_{n} is a solution of SFP (1.1) and the iterative process stops; otherwise, we set n:=n+1 and go on to (3.17)(3.20) to compute the next iteration {x}_{n+2}.
Theorem 3.6 Assume that {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})\ge \rho >0. Then the sequence \{{x}_{n}\} generated by Algorithm 3.5 converges strongly to a solution {x}^{\ast} of SFP (1.1), where {x}^{\ast}={P}_{\mathrm{\Gamma}}({x}_{1}).
Proof Firstly, we show that
for all n\ge 1. Indeed, in view of (3.11), we have \mathrm{\Gamma}\subset {Y}_{n} for all n\ge 1. To show (3.21) holds, it suffices to show that \mathrm{\Gamma}\subset {Z}_{n} for all n\ge 1. We complete the proof by induction. Since {Z}_{1}={H}_{1}, \mathrm{\Gamma}\subset {Z}_{1}. Assume that \mathrm{\Gamma}\subset {Z}_{k} form some k\ge 1; we plan to show \mathrm{\Gamma}\subset {Z}_{k+1}. Since \mathrm{\Gamma}\subset {Y}_{k}\cap {Z}_{k}, and {Y}_{k}\cap {Z}_{k}\ne \mathrm{\varnothing} closed convex, {x}_{n+1}={P}_{{Y}_{n}\cap {Z}_{n}}({x}_{1}) is well defined. It follows from Proposition 2.2(p_{1}) that
This implies that z\in {Z}_{k+1} and hence \mathrm{\Gamma}\subset {Z}_{k+1}. Consequently, \mathrm{\Gamma}\subset {Z}_{n} for all n\ge 1, and thus (3.21) holds true.
From the definition of {Z}_{n} and Proposition 2.2(p_{1}), we see that {x}_{n}={P}_{{Z}_{n}}{x}_{1}. It then follows from (3.20) that
This derives that {lim}_{n}\parallel {x}_{n}{x}_{1}\parallel exists, dented by d.
Noting that {x}_{n+1}\in {Z}_{n}, we have
By virtue of (2.1) and (3.24), we obtain
From this one derives that {x}_{n+1}{x}_{n}\to 0 (n\to \mathrm{\infty}).
Since {x}_{n+1}\in {Y}_{n}, we have
from which it turns out that
and
At this point, we show {w}_{w}({x}_{n})\subset \mathrm{\Gamma}. To end this, take \stackrel{\u02c6}{x}\in {w}_{w}({x}_{n}). Then there exists a subsequence \{{x}_{{n}_{j}}\} of \{{x}_{n}\} such that {x}_{{n}_{j}}\stackrel{w}{\u27f6}\stackrel{\u02c6}{x}. By our assumption that {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})\ge \rho >0, from (3.28) we conclude that
which implies that {f}_{n}({x}_{n})\to 0, since \{\parallel \mathrm{\nabla}{f}_{n}({x}_{n})\parallel +{\sigma}_{n}\} is bounded. Notice that {P}_{{Q}_{n}}(A{x}_{n})\in {Q}_{n}, and ∂q is a bounded mapping, we have
Since A{x}_{{n}_{j}}\rightharpoonup A\stackrel{\u02c6}{x} and q is wlsc on {H}_{2}, we derive
which implies that A\stackrel{\u02c6}{x}\in Q.
We next show \stackrel{\u02c6}{x}\in C. Indeed, from (3.27) we have
From (3.29), we have also
Consequently, it follows from (3.31) and (3.32) that
Since {P}_{{C}_{n}}({x}_{n})\in {C}_{n}, noting ∂c is a bounded mapping, we immediately obtain
Then the wlsc of c ensures that
from which it turns out that \stackrel{\u02c6}{x}\in C, and thus \stackrel{\u02c6}{x}\in \mathrm{\Gamma}. It follows from (3.21) that \stackrel{\u02c6}{x}\in {Z}_{n}, which implies that
for all n\ge 1. Thus, from (3.34) we obtain
in particular, we have
consequently, {x}_{{n}_{j}}\to \stackrel{\u02c6}{x}, since {x}_{{n}_{j}}\stackrel{w}{\u27f6}\stackrel{\u02c6}{x}.
At this point, by virtue of (3.19), we have
in particular, we have
Thus, upon taking the limit as j\to \mathrm{\infty} in (3.37), we obtain
This implies that \stackrel{\u02c6}{x}={P}_{\mathrm{\Gamma}}{x}_{1} by Proposition 2.2(p_{1}). Therefore \{{x}_{n}\} converges strongly to \stackrel{\u02c6}{x}={P}_{\mathrm{\Gamma}}{x}_{1} because of the uniqueness of {P}_{\mathrm{\Gamma}}{x}_{1}. This completes the proof. □
Algorithm 3.7 Choose an arbitrary initial data {x}_{1}\in {H}_{1}. Assume that the n th iteration {x}_{n}\in {H}_{1} has been constructed. Set
with the step size {\tau}_{n} given by (1.12) and the relaxed factor {\beta}_{n} in [0,1) satisfying {\overline{lim}}_{n}{\beta}_{n}<1. Define two halfspaces {Y}_{n} and {Z}_{n} by
The (n+1)th iteration {x}_{n+1} is then constructed by the formula:
If {x}_{n+1}={x}_{n} for some n\ge 1, then {x}_{n} is a solution of SFP (1.1) and the iterative process stops; otherwise, we set n:=n+1 and go on to (3.39)(3.41) to compute the next iteration {x}_{n+2}.
Along the proof lines of Theorem 3.6 we can prove the following.
Theorem 3.8 Assume that {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})\ge \rho >0; then the sequence \{{x}_{n}\} generated by Algorithm 3.7 converges strongly to a solution {x}^{\ast} of SFP (1.1), where {x}^{\ast}={P}_{\mathrm{\Gamma}}({x}_{1}).
The proof of Theorem 3.8 is similar to that of Theorem 3.6, and therefore we omit its details.
We next turn our attention to another kind of algorithm.
Algorithm 3.9 Choose an arbitrary initial data {x}_{1}\in {H}_{1}. Assume that the n th iteration {x}_{n}\in {H}_{1} has been constructed; then we compute the (n+1)th iteration {x}_{n+1} via the recursion:
where the step size {\tau}_{n} is given by (1.12), g:{H}_{1}\to {H}_{1} is a contraction with contractive coefficient \delta \in (0,1), and \{{\alpha}_{n}\} is a real sequence in (0,1). If {x}_{n+1}={x}_{n} for some n\ge 1, then {x}_{n} is an approximate solution of SFP (1.1) (he approximate rule will be given below) and the iterative process stops; otherwise, we set n:=n+1 and go on to (3.43) to compute the next iteration {x}_{n+2}.
We point out that if {x}_{n+1}={x}_{n} for some n\ge 1, then (3.43) reduces to
This implies that {x}_{n}\in {C}_{n} and hence {x}_{n}\in C. Write
Then it follows from (3.44) that
Such an {x}_{n} is called an approximate solution of SFP (1.1). If e({x}_{n},{\tau}_{n})=0, then {x}_{n} is a solution of SFP (1.1).
Theorem 3.10 Assume that \{{\alpha}_{n}\} and \{{\rho}_{n}\} satisfy conditions (C_{1}) {\alpha}_{n}\to 0, {\sum}_{n=1}^{\mathrm{\infty}}{\alpha}_{n}=\mathrm{\infty} and (C_{2}) {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})>0, respectively. Then the sequence \{{x}_{n}\} generated by Algorithm 3.9 converges strongly to a solution {x}^{\ast} of SFP (1.1), where {x}^{\ast}={P}_{\mathrm{\Gamma}}g({x}^{\ast}), equivalently, {x}^{\ast} solves the following variational inequality:
Proof First of all, we show there exists a unique {x}^{\ast}\in \mathrm{\Gamma} such that {x}^{\ast}={P}_{\mathrm{\Gamma}}g({x}^{\ast}). Indeed, since {P}_{\mathrm{\Gamma}}g:{H}_{1}\to {H}_{1} is a contraction with the contractive coefficient \delta \in (0,1), by the Banach contractive mapping principle, we conclude that there exists a unique {x}^{\ast}\in {H}_{1} such that {x}^{\ast}={P}_{\mathrm{\Gamma}}g({x}^{\ast})\in \mathrm{\Gamma}, equivalently, {x}^{\ast} solves the following variational inequality:
Write {y}_{n}={x}_{n}{\tau}_{n}\mathrm{\nabla}{f}_{n}({x}_{n}) and {z}_{n}={\alpha}_{n}g{x}_{n}+(1{\alpha}_{n}){y}_{n}. Then (3.43) can be rewritten as
Noting that {x}^{\ast}\in \mathrm{\Gamma} and Q\subseteq {Q}_{n} for all n\ge 1, we have A{x}^{\ast}\in {Q}_{n} for all n\ge 1, and hence
Since I{P}_{{Q}_{n}} is firmly nonexpansive, we have
By virtue of (2.1) and (3.47), we obtain
in particular, we have
for all n\ge 1.
We now estimate {\parallel {z}_{n}{x}^{\ast}\parallel}^{2}. By virtue of definition of the norm \parallel \cdot \parallel and Schwarz’s inequality, we obtain
from which it turns out that
Substituting (3.48) into (3.50) yields
By virtue of Proposition 2.2(p_{6}), (3.48), and (3.51), noting that {x}^{\ast}\in C\subset {C}_{n} for all n\ge 1, we have
We next show that \{{x}_{n}\} is bounded. Using (3.43) and (3.49), we have
for all n\ge 1, therefore \{{x}_{n}\} is bounded; so are \{{y}_{n}\} and \{{z}_{n}\}.
Finally, we show that {x}_{n}\to {x}^{\ast} (n\to \mathrm{\infty}).
Set {s}_{n}={\parallel {x}_{n}{x}^{\ast}\parallel}^{2} and assume that {\rho}_{n}(4{\rho}_{n})\ge \rho for all n\ge 1. Then (3.52) reduces to
We consider two possible cases.
Case 1. \{{s}_{n}\} is eventually decreasing, i.e., there exists some integer {n}_{0}\ge 1 such that
which means that {lim}_{n}{s}_{n} exists. Note that \{{z}_{n}\} is bounded and {\alpha}_{n}\to 0. Letting n\to \mathrm{\infty} in (3.53) yields \frac{{f}_{n}({x}_{n})}{\parallel \mathrm{\nabla}{f}_{n}({x}_{n})\parallel +{\sigma}_{n}}\to 0 and (I{P}_{{Q}_{n}}){z}_{n}\to 0. Since \{\parallel \mathrm{\nabla}{f}_{n}({x}_{n})+{\sigma}_{n}\parallel \} is a bounded sequence, we conclude that {f}_{n}({x}_{n})\to 0 and hence
Observe that \parallel {z}_{n}{y}_{n}\parallel \le {\alpha}_{n}\parallel g({x}_{n}){y}_{n}\parallel \le {\alpha}_{n}{M}_{1}\to 0,
and
We may assume that
Without loss of generality, we assume that {x}_{{n}_{k}}\rightharpoonup \stackrel{\u02c6}{x} (k\to \mathrm{\infty}); then A{x}_{{n}_{k}}\rightharpoonup A\stackrel{\u02c6}{x} (k\to \mathrm{\infty}). Since {P}_{{Q}_{{n}_{k}}}A{x}_{{n}_{k}}\in {Q}_{{n}_{k}}, \{{\eta}_{{n}_{k}}\}\subset \partial q(A{x}_{{n}_{k}}) is a bounded sequence and (I{P}_{{Q}_{{n}_{k}}})A{x}_{{n}_{k}}\to 0 (k\to \mathrm{\infty}) by (3.54), we deduce that
as k\to \mathrm{\infty}, then wlsc of q implies that
and thus A\stackrel{\u02c6}{x}\in Q.
On the other hand, since {P}_{{C}_{{n}_{k}}}({x}_{{n}_{k}})\in {C}_{{n}_{k}}, \{{\xi}_{{n}_{k}}\}\subset \partial c({x}_{{n}_{k}}) is a bounded sequence, and (I{C}_{{Q}_{{n}_{k}}}){x}_{{n}_{k}}\to 0 by (3.55), we derive
as k\to \mathrm{\infty}, then wlsc of c implies that
and thus \stackrel{\u02c6}{x}\in C. Consequently, \stackrel{\u02c6}{x}\in C\cap {A}^{1}(Q)=\mathrm{\Gamma}. It follows from (3.56) and Proposition 2.2(p_{1}) that
Taking into account of (3.53), we have
Applying Proposition 2.5 to (3.58), we derive that {s}_{n}\to 0 as n\to \mathrm{\infty}, i.e., {x}_{n}\to {x}^{\ast} as n\to \mathrm{\infty}.
Case 2. \{{s}_{n}\} is not eventually decreasing. In this case, we can find an integer {n}_{0}\ge 1 such that {s}_{{n}_{0}}<{s}_{{n}_{0}+1}. Define J(n):=\{{n}_{0}\le k\le n:{s}_{k}<{s}_{k+1}\}, n>{n}_{0}. Then J(n)\ne \mathrm{\varnothing} and J(n)\subseteq J(n+1). Define \tau :\mathrm{N}\to \mathrm{N} by
Then \tau (n)\to \mathrm{\infty} as n\to \mathrm{\infty}, {s}_{\tau (n)}\le {s}_{\tau (n)+1} for all n>{n}_{0} and {s}_{n}\le {s}_{\tau (n)+1} for all n>{n}_{0}; see [20] for details.
Since {s}_{\tau (n)}\le {s}_{\tau (n)+1} for all n>{n}_{0}, it follows from (3.53) that
and \parallel (I{P}_{{C}_{\tau (n)}}){z}_{\tau (n)}\parallel \to 0 as n\to \mathrm{\infty}.
At this point, by virtue of a similar reasoning to the corresponding parts in case 1, we can deduce that {\overline{lim}}_{n}\u3008(gI){x}^{\ast},{z}_{\tau (n)}{x}^{\ast}\u3009={\overline{lim}}_{n}\u3008(gI){x}^{\ast},{x}_{\tau (n)}{x}^{\ast}\u3009\le 0.
Noting that {s}_{\tau (n)}\le {s}_{\tau (n)+1}, it follows from (3.53) that
from which one derives that {\overline{lim}}_{n}{s}_{\tau (n)}\le 0, and hence {s}_{\tau (n)}\to 0 as n\to \mathrm{\infty}. From this it turns out that {s}_{\tau (n)+1}\to 0 as n\to \mathrm{\infty}, since {s}_{\tau (n)+1}{s}_{\tau (n)}\to 0 as n\to \mathrm{\infty}. Consequently, {s}_{n}\to 0, as n\to \mathrm{\infty}, since 0\le {s}_{n}\le {s}_{\tau (n)+1}\to 0 as n\to \mathrm{\infty}. This completes the proof. □
By using an argument like the method in Theorem 3.10, we have the following more general algorithm and convergence theorem.
Algorithm 3.11 Choose an arbitrary initial data {x}_{1}\in {H}_{1}. Assume that the n th iteration {x}_{n}\in {H}_{1} has been constructed; then we compute the (n+1)th iteration {x}_{n+1} via the recursion
where \{{\beta}_{n}\} is a real sequence in [0,1) satisfying {\overline{lim}}_{n}{\beta}_{n}<1, \{{\alpha}_{n}\} is a real sequence in (0,1) satisfying conditions (C_{1}) {\alpha}_{n}\to 0 and (C_{2}) {\sum}_{n=1}^{\mathrm{\infty}}{\alpha}_{n}=\mathrm{\infty}, g:{H}_{1}\to {H}_{1} is a contraction with contractive coefficient \delta \in (0,1) and {\tau}_{n} is given by (1.12). If {x}_{n+1}={x}_{n} for some n\ge 1, then {x}_{n} is an approximate solution of SFP (1.1) and the iterative process stops; otherwise, we set n:=n+1 and go on to (3.59) to compute the next iteration {x}_{n+2}.
Theorem 3.12 Assume that {\underline{lim}}_{n}{\rho}_{n}(4{\rho}_{n})>0; the sequence generated by algorithm (3.59) converges strongly to a solution {x}^{\ast} of the SFP (1.1), where {x}^{\ast}={P}_{\mathrm{\Gamma}}g({x}^{\ast}), equivalently, {x}^{\ast} is a solution of the following variational inequality:
4 Numerical experiments
In this section, we consider two typical numerical experiments to illustrate the performance of step size (1.12) with CQlike algorithms. Firstly, we introduce a linear observation model as follows, which covers many problems in signal and image processing:
where y\in {\mathrm{R}}^{M} is the observed or measured data with noisy ε. A:{\mathrm{R}}^{N}\to {\mathrm{R}}^{M} denotes the bounded linear observation operator. A is sparse and the range of it is not closed in most inverse problems, thus A is often illcondition and the problem is also illposed. When x is a sparse expansion, finding the solutions of (4.1) can be seen as finding a solution to the leastsquare problem
for any real number t>0.
When we set C=\{x\in {\mathrm{R}}^{N}:{\parallel x\parallel}_{1}\le t\} and Q=\{y\}, it is a particular case of SFP (1.1); see [11]. Therefore, we continue by applying the CQ algorithm to solve (4.2). We compute the projection onto C through a soft thresholding method; see [11, 21–23].
Next, according to the examples in [11, 22], we also choose two similar particular problems: compressed sensing and image deconvolution, which can be covered by (4.1). The experiments compare the performances of the proposed step size (1.12) with the step size in [11], and analysis some properties of (1.12).
4.1 Compressed sensing
In a general compressed sensing model, we set the hits of a signal x\in {\mathrm{R}}^{N} is N={2}^{12}. There exist m=50 spikes with amplitude ±1 distributed in the whole domain randomly. The plot can be seen on the top of Figure 1. Then we set the observation dimension M={2}^{10} and a matrix A with M\times N order is also generated arbitrarily. A standard Gaussian distribution noise with variance {\sigma}_{\epsilon}^{2}={10}^{4} is added. Let t=50 in (4.2).
For the step sizes (1.11) and (1.12), we always set the constant \rho =2. For Algorithm 3.3, we set {\beta}_{n}=0.5. All the processes are started with initial signal {x}_{0}=0 and finished with the stop rule
We calculated the mean squared error (MSE) for the results
where {x}^{\ast} is an estimated signal of x.
The second and third plots in Figure 1 correspond to the results with step sizes (1.11) and (1.12) to Algorithm 3.1, respectively. The recovered result by Algorithm 3.3 with step size (1.12) is shown in the fourth plot. Especially for the fifth, when we set {\beta}_{n}={(n+1)}^{k}, k=1,2,3,\dots , when we have k\ge 3 the iteration steps of Algorithm 3.3 start to approach the number in the second plot, and the restored precision is a little poorer than the others.
For (1.12) we firstly set {\sigma}_{n}=\sigma =0.5; then in order to study its effect to the convergence speed of the CQ algorithm, we let it be {\sigma}_{n}={(n+1)}^{l}, l\ge 1 is an integer. In Figure 2 we can find that when l\ge 4 the best MSE curves can be obtained, and it starts to change less. Therefore, {\sigma}_{n} should be as little as possible.
4.2 Image deconvolution
In this subsection, we continue by applying Algorithms 3.1 and 3.3 to recover the blurred Cameraman image. In the experiments, from [22, 24] we employ Haar wavelets and the blur point spread function {h}_{ij}={(1+{i}^{2}+{j}^{2})}^{1}, for i,j=4,\dots ,4; the noise variance is {\sigma}^{2}=2. The size of the image is N=M={256}^{2}. The threshold value is handtuned for the best SNR improvement. t is the sum of all the original pixel values.
We observe the performance of {\sigma}_{n} in (1.12); see Figure 3. We find that at the beginning several steps there are similar SNR curves, however, after 19 iterations, \sigma =0.5 is similar with (1.11). When 1\le l<6, the curves are worse than the others. While we set l\ge 6 the curve starts to be consistent with the curve of (1.11). Therefore, we also know that {\sigma}_{n} should be better as little as possible.
5 Conclusion remarks
In this paper we have proposed several kinds of adaptively relaxed iterative algorithms with a new variable step size {\tau}_{n} for solving SFP (1.1). The feature is that the new variable step size {\tau}_{n} contains a sequence of positive numbers in its denominator. Because of this, the proposed algorithms with relaxed iterations will never terminate at any iteration step. On the other hand, unlike the previous known algorithms, our stop rule is that the related iteration process will stop if {x}_{n+1}={x}_{n} for some n\ge 1.
By means of new analysis techniques, we have proved several kinds of weak and strong convergence theorems of the proposed algorithms for solving SFP (1.1), which improved, extended, and complemented those existing in the literature. We remark that all convergence results in this paper still hold true if we use the step size {\tau}_{n} given by (1.11) to replace the step size given by (1.12). In such a case, the stop rules should be modified. We would like to point out that our Theorems 3.10 and 3.12 are closely related to a sort of variational inequalities.
Finally, numerical experiments have been presented to illustrate the effectiveness of the proposed algorithms and applications in signal processing of the algorithms with the step size selected in this paper. The numerical results tell us that the changes of the choice of the step size {\sigma}_{n} given by (1.12) may affect the convergence rate of the iterative algorithms, and {\sigma}_{n} should be chosen as small as possible; for instance, we can choose {\sigma}_{n} such that {\sigma}_{n}\to 0 as n\to \mathrm{\infty}.
References
Censor Y, Elfving T: A multiprojection algorithm using Bregman projections in a product space. Numer. Algorithms 1994, 8: 221–239. 10.1007/BF02142692
López G, MartínMárquez V, Xu HK: Iterative algorithms for the multiplesets split feasibility problem. In Biomedical Mathematics: Promising Directions in Imaging, Therapy Planning and Inverse Problems. Edited by: Censor Y, Jiang M, Wang G. Medical Physics Publishing, Madison; 2009:243–279.
Chang SS, Kim JK, Cho YJ, Sim JY: Weak and strong convergence theorems of solutions to split feasibility problem for nonspreading type mapping in Hilbert spaces. Fixed Point Theory Appl. 2014., 2014: Article ID 11
Stark H, Yang Y: Vector Space Projections: a Numerical Approach to Signal and Image Processing, Neural Nets and Optics. Wiley, New York; 1998.
Censor Y, Bortfeld T, Martin B, Bortfeld T: A unified approach for inversion problems in intensitymodulated radiation therapy. Phys. Med. Biol. 2003, 51: 2353–2365.
Censor Y, Elfving T, Kopf N, Bortfeld T: The multiplesets split feasibility problem and its applications for inverse problems. Inverse Probl. 2005, 21: 2071–2084. 10.1088/02665611/21/6/017
Qu B, Xiu N: A note on the CQ algorithm for the split feasibility problem. Inverse Probl. 2005, 21: 1655–1665. 10.1088/02665611/21/5/009
Li M: Improved relaxed CQ methods for solving the split feasibility problem. Adv. Model. Optim. 2011, 13: 305–318.
Abdellah B, Muhammad AN, Mohamed K, Sheng ZH: On descentprojection method for solving the split feasibility problems. J. Glob. Optim. 2012, 54: 627–639. 10.1007/s1089801197822
Yang Q: On variablestep relaxed projection algorithm for variational inequalities. J. Math. Anal. Appl. 2005, 302: 166–179. 10.1016/j.jmaa.2004.07.048
López G, MartínMárquez V, Wang FH, Xu HK: Solving the split feasibility problem without prior knowledge of matrix norms. Inverse Probl. 2012., 28: Article ID 085004
Göebel K, Kirk WA: Topics on Metric Fixed Point Theory. Cambridge University Press, Cambridge; 1990.
Aubin JP: Optima and Equilibria: An Introduction to Nonlinear Analysis. Springer, Berlin; 1993.
Byrne C: A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Probl. 2004, 20: 103–120. 10.1088/02665611/20/1/006
Bauschke HH, Borwein JM: On projection algorithms for solving convex feasibility problems. SIAM Rev. 1996, 38: 367–426. 10.1137/S0036144593251710
Xu HK: Iterative algorithms for nonlinear operators. J. Lond. Math. Soc. 2011, 66: 240–256.
Wang FH, Xu HK: Approximating curve and strong convergence of the CQ algorithm for the split feasibility problem. J. Inequal. Appl. 2010. 10.1155/2010/102085
Dang YZ, Gao Y: The strong convergence of a KMCQlike algorithm for a split feasibility problem. Inverse Probl. 2011., 27: Article ID 015007
Yu X, Shahzad N, Yao YH: Implicit and explicit algorithms for solving the split feasibility problem. Optim. Lett. 2012, 6: 1447–1462. 10.1007/s1159001103400
Yao YH, Postolache M, Liou YC: Strong convergence of a selfadaptive method for the split feasibility problem. Fixed Point Theory Appl. 2013., 2013: Article ID 201 10.1186/168718122013201
Daubechies I, Fornasier M, Loris I: Accelerated projected gradient method for linear inverse problems with sparsity constraints. J. Fourier Anal. Appl. 2008, 14: 764–792. 10.1007/s0004100890398
Figueiredo MA, Nowak RD, Wright SJ: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 2007, 1: 586–598.
Starck JL, Murtagh F, Fadili JM: Sparse Image and Signal Processing Wavelets, Curvelets, Morphological Diversity. Cambridge University Press, Cambridge; 2010:166.
Mário ATF: An EM algorithm for waveletbased image restoration. IEEE Trans. Image Process. 2003, 12: 906–917. 10.1109/TIP.2003.814255
Acknowledgements
This research was supported by the National Natural Science Foundation of China (11071053).
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
All the authors contributed equally. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Zhou, H., Wang, P. Adaptively relaxed algorithms for solving the split feasibility problem with a new step size. J Inequal Appl 2014, 448 (2014). https://doi.org/10.1186/1029242X2014448
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1029242X2014448
Keywords
 split feasibility problem
 adaptive step size
 adaptively relaxed iterative algorithm
 convergence