# On relaxed and contraction-proximal point algorithms in hilbert spaces

## Abstract

We consider the relaxed and contraction-proximal point algorithms in Hilbert spaces. Some conditions on the parameters for guaranteeing the convergence of the algorithm are relaxed or removed. As a result, we extend some recent results of Ceng-Wu-Yao and Noor-Yao.

## 1. Introduction

Throughout, H denotes a real Hilbert space and A a multi-valued operator with domain D(A). We know that A is called monotone if 〈u - v, x - y〉 ≥ 0, for any u Ax, v Ay; maximal monotone if its graph G(A) = {(x,y): x D(A), y Ax} is not properly contained in the graph of any other monotone operator. Denote by S: = {x D(A): 0 Ax} the zero set and by J c : = (I + cA)-1 the resolvent of A. It is well known that J c is single valued and D(J c ) = H for any c > 0.

A fundamental problem of monotone operators is that of finding an element x so that 0 Ax. This problem is essential because it includes many concrete examples, such as convex programming and monotone variational inequalities. A successful and powerful algorithm for solving this problem is the well-known proximal point algorithm (PPA), which generates, for any initial guess, x0 H, an iterative sequence as

$x n + 1 = J c n ( x n + e n ) ,$
(1.1)

where (c n ) is a positive real sequence and (e n ) is the error sequence (see ). To guarantee the convergence of PPA, there are two kinds of accuracy criterion posed on the error sequence:

$(I) e n ≤ ε n , ∑ n = 0 ∞ ε n < ∞ or$
$(II) e n ≤ η n x ̃ n - x n , ∑ n = 0 ∞ η n < ∞ ,$

where $x ̃ n = J c n ( x n + e n ) .$ In 2001, Han and He  proved that in finite dimensional Hilbert space criterion (II) can be replaced by

$(II’) e n ≤ η n x ̃ n - x n , ∑ n = 0 ∞ η n 2 < ∞ .$

The infinite version was obtained by Marino and Xu .

There are various generations or modifications on the PPA. Among them Eckstein and Bertsekas  proposed the relaxed proximal point algorithm (RPPA):

$x n + 1 = ( 1 - ρ n ) x n + ρ n J c n ( x n ) + e n ,$
(1.2)

where (ρ n ) (0, 2) is a relaxation factor. The weak convergence of (1.2) is guaranteed provided that (e n ) satisfies criterion (I),

$c n ≥ c ̄ > 0 , 0 < δ ≤ ρ n ≤ 2 - δ .$
(1.3)

On the other hand, since the PPA does not necessarily converge strongly (see ), many authors have conducted worthwhile studies on modifying the PPA so that the strong convergence is guaranteed (see, for instance, ). In particular, Marino and Xu  proposed the contraction-proximal point algorithm (CPPA):

$x n + 1 = λ n u + ( 1 - λ n ) J c n ( x n ) + e n ,$
(1.4)

where the parameters above satisfy (i) lim n λ n = 0, Σ n λ n = ∞; (ii) either Σ n |λ n +1- λ n | < ∞; or lim n λn/λ n +1 = 1; (iii) $0< c ≤ c n ≤ c ̄ <∞, ∑ n | c n + 1 - c n |<∞;$ (iv) Σ n ||e n || < ∞. Under these assumptions, the CPPA converges strongly to P S (u), the projection of u onto S.

In this article, we shall focus on the RPPA and CPPA. We note that the resolvent is in fact the arithmetic mean of the identity and a nonexpansive operator. By using this fact, we relax or remove some sufficient conditions to guarantee the convergence of the algorithms. As a result, we extend and improve some recent results on the PPA.

## 2. Some lemmas

We know that an operator T : HH is called (i) nonexpansive if ||Tx - Ty|| ≤ || x - y|| x,y H; and (ii) firmly nonexpansive if 〈Tx - Ty, x - y〉 ≥ ||Tx - Ty||2 x,y H. Denote by Fix(T) = {x H : x = Tx} the fixed point set of T. It is well known that firmly nonexpansive operators have the following properties.

Lemma 1 (Goebel-Kirk ). Let T be firmly nonexpansive. Then (1) 2T - I is nonexpansive; (2) 〈Tx - x, Tx - z〉 ≤ 0 for all x H and for all z H Fix(T).

It is well known that J c is firmly nonexpansive and consequently nonexpansive; moreover, S = Fix (J c ). Since the fixed point set of nonexpansive operators is closed convex, the projection P s onto the solution set S is well defined whenever S ≠ . Hereafter, we assume that S is nonempty. The following lemmas play an important role in our convergence analysis.

Lemma 2 (resolvent identity ). Let c, t > 0. Then for any x H,

$J c x = J t t c x + 1 - t c J c x .$

Lemma 3 (). Let (ρ n ) be real sequence satisfying

$0 < l i m i n f n → ∞ ρ n ≤ l i m s u p n → ∞ ρ n < 1.$

Assume that (x n ) and (y n ) are bounded sequences in H satisfying x n +1 = (1 - ρ n )x n + ρ n y n . If

$limsup n → ∞ ( y n + 1 - y n - x n + 1 - x n ) ≤ 0 ,$

then lim n →∞||x n -y n || = 0.

Lemma 4 For r, s, > 0, let T r = 2J r - I. Then for any x H,

$T s x - T r x ≤ 1 - s r x - T r x .$
(2.1)

Proof. Using the resolvent identity, we have

$T s x - T r x = 2 J s x - J s s r x + 1 - s r J r x ≤ 2 x - s r x + 1 - s r J r x = 2 1 - s r x - J r x = 1 - s r x - T r x ,$

where the inequality uses the nonexpansive property of the resolvent.

Lemma 5 (). Let (ε n ) and (s n ) be positive real sequences. Assume that Σ n ε n < ∞. If either (i) s n+ 1≤ (1 + ε n )s n , or (ii) s n+ 1≤ ε n , then the limit of (s n ) exists.

## 3. The relaxed proximal point algorithm

Under criterion (II'), Ceng et al.  considered another type, RPPA:

$x ̃ n = J c n ( x n + e n ) , x n + 1 = ( 1 - ρ n ) x n + ρ n x ̃ n ,$
(3.1)

and proved the weak convergence of (3.1) under the assumptions:

$c n ≥ c ̄ > 0 , 0 < δ ≤ ρ n ≤ 1 .$

We note that the choice of (ρ n ) excludes the case whenever ρ n (1,2), the overrelaxation. The overrelaxation, however, may indeed speed up the convergence of the algorithm (see ). Below, we shall improve their conditions on the relaxation factor from 0 < δρ n ≤ 1 to 0 < δρ n ≤ 2 - δ.

Theorem 6. Assume that the following conditions hold:

(a) $c n ≥ c ̄ >0;$

(b) 0 < δρ n ≤ 2 - δ;

(c) $∑ n e n ≤ η n x ̃ n - x n , ∑ n η n 2 <∞.$

Then the sequence generated by (3.1) converges weakly to a point in S.

Proof. The key point of our proof is to show lim n s n = 0, where $s n = x n - J c n ( x n ) .$ To see this, let z S be fixed. Since $J c n$ is firmly nonexpansive and $z∈ Fix ( J c n ) ,$ applying Lemma 1 yields $⟨ x ̃ n - z , x ̃ n - x n - e n ⟩ ≤0.$ This together with (3.1) enables us to get

$x n + 1 - z 2 - x n - z 2 = ( x n - z ) + ρ n ( x ̃ n - x n ) 2 - x n - z 2 = 2 ρ n ⟨ x n - z , x ̃ n - x n ⟩ + ρ n 2 x ̃ n - x n 2 = 2 ρ n ⟨ x ̃ n - z , x ̃ n - x n ⟩ - ρ n ( 2 - ρ n ) x ̃ n - x n 2 ≤ 2 ρ n ⟨ x ̃ n - z , e n ⟩ - ρ n ( 2 - ρ n ) x ̃ n - x n 2 = 2 ρ n ⟨ x ̃ n - x n , e n ⟩ + 2 ρ n ⟨ x n - z , e n ⟩ - ρ n ( 2 - ρ n ) x ̃ n - x n 2 ≤ 2 ρ n e n x ̃ n - x n + 2 ρ n e n x n - z - ρ n ( 2 - ρ n ) x ̃ n - x n 2 ≤ 2 ρ n η n x ̃ n - x n 2 + 2 ρ n η n x ̃ n - x n x n - z - ρ n ( 2 - ρ n ) x ̃ n - x n 2 .$

Using the basic inequality 2aba2 / ε + εb2 (a,b , ε > 0), we arrive at

$2 ρ n η n x n - z x ̃ n - x n ≤ 2 ρ n 2 - ρ n η n x n - z 2 + 2 - ρ n 2 ρ n ρ n x ̃ n - x n 2 = 2 ρ n η n 2 2 - ρ n x n - z 2 + ρ n ( 2 - ρ n ) 2 x ̃ n - x n 2 ≤ 2 ( 2 - δ ) η n 2 δ x n - z 2 + ρ n ( 2 - ρ n ) 2 x ̃ n - x n 2 = ε n x n - z 2 + ρ n ( 2 - ρ n ) 2 x ̃ n - x n 2 ,$

where $ε n =2 ( 2 - δ ) η n 2 ∕δ$ is a summable sequence. Substituting this into above yields

$x n + 1 - z 2 ≤ ( 1 + ε n ) x n - z 2 - ρ n ( 2 - ρ n - 4 η n ) 2 x ̃ n - x n 2 .$

Since by Lemma 5 the limit of ||x n - z ||2 exists and lim inf n ρ n (2 - ρ n -4η n ) ≥ δ (2 - δ), this implies that $x ̃ n - x n →0.$ On the other hand, we note that for all n

$s n ≤ ( 1 + η n ) x n - x ̃ n → 0 ;$

therefore, lim n s n = 0. The rest proof is similar to that of [12, Theorem 3.1].

We now turn to the RPPA (1.2). Under the criterion (I), the assumptions on relaxation factors can be relaxed to Σρ n (2 - ρ n ) = ∞ (see [3, Theorem 3.3]). Since the proof there is very technical, we wang to restate this result with a simple proof.

Theorem 7. Assume that the following conditions hold:

(a) Σ n ||e n || < ∞;

(b) Σ n ρn(2 - ρ n ) = ∞;

(c) $0< c ̄ ≤ c n ≤ c ̃ <∞;$

(d) Σ n |c n +1- c n | < ∞.

Then the sequence generated by (1.2) converges weakly to a point in S.

Proof. The key step is to show lim n s n = 0, where $s n = x n - J c n ( x n ) .$ It has been shown that Σ n ρ n (2 - ρ n )s n < ∞ (see [3, Lemma 3.2]). Therefore, it remains to show that lim n s n exists. By letting T n = 2J n - I, we rewrite (2) as

$x n + 1 = 1 - ρ n 2 x n + ρ n 2 T n x n + e n .$

In view of Lemma 4 and condition (c),

$∥ T n + 1 x n + 1 - T n x n ∥ ≤ ∥ T n + 1 x n + 1 - T n + 1 x n ∥ + ∥ T n + 1 x n - T n x n ∥ ≤ ∥ x n - x n + 1 ∥ + ∥ T n + 1 x n - T n x n ∥ ≤ ∥ x n - x n + 1 ∥ + 1 - c n + 1 c n ∥ T n x n - x n ∥ ≤ ∥ x n - x n + 1 ∥ + | c n + 1 - c n | c ̄ ∥ T n x n - x n ∥ ≤ ∥ x n - x n + 1 ∥ + M | c n + 1 - c n | ,$

where M > 0 is a suitable number. Consequently,

$x n + 1 - T n + 1 x n + 1 = 1 - ρ n 2 x n + ρ n 2 T n x n + e n - T n + 1 x n + 1 = 1 - ρ n 2 ( x n - T n x n ) + ( T n x n - T n + 1 x n + 1 ) + e n ≤ 1 - ρ n 2 x n - T n x n + T n x n - T n + 1 x n + 1 + e n ≤ 1 - ρ n 2 x n - T n x n + x n - x n + 1 + M | c n + 1 - c n | + e n = 1 - ρ n 2 x n - T n x n + ρ n 2 ( x n - T n x n ) + e n + M | c n + 1 - c n | + e n ≤ x n - T n x n + M | c n + 1 - c n | + 2 e n .$

Using s n = || x n - T n x n ||/2, we therefore arrive at

$s n + 1 ≤ s n + σ n ,$

where σ n = 2M |c n +1- c n | + 4||e n || satisfying Σ n σ n < ∞ (due to (a) and (d)). By Lemma 5, we finally conclude that lim n s n = 0.

## 4. The contraction-proximal point algorithm

Recently, Yao and Noor  extended the CPPA to the following form:

$x n + 1 = λ n u + r n x n + δ n J c n ( x n ) + e n ,$
(4.1)

where (λ n ),(r n ),(δ n ) (0,1) and λ n + r n + δ n = 1. They proved the strong convergence of the algorithm provided that (i) $c n ≥ c ̄ >0, lim n | c n + 1 - c n |=0;$ (ii) 0 < lim inf n r n ≤ lim sup n r n < 1; and (iii) Σ n ||e n || < ∞. Also, they claimed that their algorithm includes the CPPA as a special case. This is, however, not the case, because condition (ii) excludes the special case r n ≡ 0. To overcome this drawback, we shall show the same result by replacing condition (ii) with the weak condition:

$l i m s u p n → ∞ r n <1 ⇔ l i m i n f n → ∞ δ n >0.$

In this situation, the CPPA is evidently a special case of algorithm (4.1). The idea of the following proof is followed by the second author .

Theorem 8. Let be (λ n ), (r n ) and (δ n ) be parameters in (4.1). Assume that the following conditions hold:

(a) lim n λ n = 0, Σ n λ n = ∞;

(b) lim sup n r n < 1 lim inf n δ n > 0;

(c) $c n ≥ c ̄ >0,| c n + 1 - c n |→0;$

(d) Σ n ||e n || < ∞.

Then the sequence generated by (4.1) converges strongly to P S (u).

Proof. All we need to do is to prove ||x n +1- x n || → 0, since the rest proof is similar to that of [14, Theorem 3.3]. To this end, set $J n = J c n$ and T n = 2J n - I. It then follows from (4.1) that

$x n + 1 = λ n u + r n x n + δ n 2 ( I + T n ) x n + e n = r n + δ n 2 x n + λ n u + δ n 2 T n x n + e n .$

Let ρ n = λ n + (δ n /2). Then the algorithm has the form:

$x n + 1 = ( 1 - ρ n ) x n + ρ n y n ,$
(4.2)

where y n = (2λ n u + δ n T n x n + 2e n )/2ρ n. Using nonexpansiveness of T n and Lemma 4, we have

$T n + 1 x n + 1 - T n x n ≤ T n + 1 x n + 1 - T n + 1 x n + T n + 1 x n - T n x n ≤ x n + 1 - x n + 1 - c n + 1 c n T n x n - x n ≤ x n + 1 - x n + | c n - c n + 1 | c ̄ T n x n - x n .$
(4.3)

On the other hand, it follows from the definition of y n that

(4.4)

Since (x n ) is bounded and T n is nonexpansive, we can find M > 0 so that (||T n x n || + ||x n || + ||u||) ≤ M for all n Adding (4.3) and (4.4) and noting δ n ≤ 2ρ n yield

With the knowledge that ||e n ||→ 0 and

$λ n ρ n = 2 λ n 2 λ n + δ n → 0 , δ n 2 ρ n = δ n 2 λ n + δ n → 1 ,$

we therefore deduce from (b) and (c) that

$l i m s u p n → ∞ ( ‖ y n + 1 − y n ‖ − ‖ x n + 1 − x n ‖ ) ≤ l i m s u p n → ∞ M ( | λ n + 1 ρ n + 1 − λ n ρ n | + ‖ e n + 1 ‖ ρ n + 1 + ‖ e n ‖ ρ n + | δ n + 1 2 ρ n + 1 − δ n 2 ρ n | + | c n − c n + 1 | c ¯ ) → 0.$

Note that lim inf n ρ n = lim inf n (δ n /2)> 0 and lim sup n ρ n = lim sup n (δ n /2) ≤ 1/2 < 1. On the other hand, it is easy to check that (x n ) is bounded and so is (y n ) We therefore apply Lemma 3 to yield lim n ||x n - y n || = 0. By means of (4.2), we finally have

$x n + 1 - x n = ρ n x n - y n → ,$

and thus the required result at once follows.

As a corollary, we improve [3, Theorem 4.1] as follows.

Theorem 9. Assume that the following conditions hold:

(a) lim n λ n = 0, Σ n λ n = ∞;

(b) $c n ≥ c ̄ >0,| c n + 1 - c n |→0;$

(c) Σ n ||e n || < ∞.

Then the sequence generated by (1.4) converges strongly to P S (u).

## Abbreviations

CPPA:

contraction-proximal point algorithm

PPA:

proximal point algorithm

RPPA:

relaxed proximal point algorithm.

## References

1. 1.

Rockafellar RT: Monotone operators and the proximal point algorithm. SIAM J Control Optim 1976, 14: 877–898. 10.1137/0314056

2. 2.

Han D, He BS: A new accuracy criterion for approximate proximal point algorithms. J Math Anal Appl 2001, 263: 343–354. 10.1006/jmaa.2001.7535

3. 3.

Marino G, Xu HK: Convergence of generalized proximal point algorithm. Comm Pure Appl Anal 2004, 3: 791–808.

4. 4.

Eckstein J, Bertsekas DP: On the Douglas-Rachford splitting method and the proximal points algorithm for maximal monotone operators. Math Programming 1992, 55: 293–318. 10.1007/BF01581204

5. 5.

Güler O: On the convergence of the proximal point algorithm for convex optimization. SIAM J Control Optim 1991, 29: 403–419. 10.1137/0329022

6. 6.

Bauschke HH, Combettes PL: A weak-to-strong convergence principle for Fejér-monotone methods in Hilbert spaces. Math Oper Res 2001, 26: 248–264. 10.1287/moor.26.2.248.10558

7. 7.

Solodov MV, Svaiter BF: Forcing strong convergence of proximal point iterations in a Hilbert space. Math Programming Ser 2000, 87: 189–202.

8. 8.

Xu HK: Iterative algorithms for nonlinear operators. J Lond Math Soc 2002, 66: 240–256. 10.1112/S0024610702003332

9. 9.

Goebel K, Kirk WA: Topics on Metric Fixed Point Theory. Cambridge University Press, Cambridge; 1990.

10. 10.

Suzuki T: A sufficient and necessary condition for Halpern-type strong convergence to fixed points of nonexpansive mappings. Proc Am Math Soc 2007, 135: 99–106.

11. 11.

Tan KK, Xu HK: Approximating fixed points of nonexpansive mappings by the Ishikawa iteration process. J Math Anal Appl 1993, 178: 301–308. 10.1006/jmaa.1993.1309

12. 12.

Ceng LC, Wu SY, Yao JC: New accuracy criteria for modified approximate proximal point algorithms in Hilbert space. Taiwan J Math 2008, 12: 1691–1705.

13. 13.

Eckstein J, Ferris MC: Operator-splitting methods for monotone affine variational inequalities, with a parallel application to optimal control. INFORMS J Comput 1998, 10: 218–235. 10.1287/ijoc.10.2.218

14. 14.

Yao Y, Noor MA: On convergence criteria of generalized proximal point algorithms. J Comput Appl Math 2008, 217: 46–55. 10.1016/j.cam.2007.06.013

15. 15.

Wang F: A note on the regularized proximal point algorithm. J Global Optim 2011, 50: 531–535. 10.1007/s10898-010-9611-z

## Acknowledgements

The authors would like to express thier sincere thanks to the referees for their valuable suggestions. This study is supported by the Natural Science Foundation of Department of Education, Henan(2011B110023).

## Author information

Authors

### Corresponding author

Correspondence to Shuyu Wang.

### Authors' contributions

Both authors contributed equally to this work. All authors read and approved the final manuscript.

### Competing interests

The authors declare that they have no competing interests.

## Rights and permissions

Reprints and Permissions

Wang, S., Wang, F. On relaxed and contraction-proximal point algorithms in hilbert spaces. J Inequal Appl 2011, 41 (2011). https://doi.org/10.1186/1029-242X-2011-41 