Skip to main content
Journal of Inequalities and Applications
Journal of Inequalities and Applications Cover Image
Download PDF

On relaxed and contraction-proximal point algorithms in hilbert spaces

Journal of Inequalities and Applications volume 2011, Article number: 41 (2011) Cite this article

Abstract

We consider the relaxed and contraction-proximal point algorithms in Hilbert spaces. Some conditions on the parameters for guaranteeing the convergence of the algorithm are relaxed or removed. As a result, we extend some recent results of Ceng-Wu-Yao and Noor-Yao.

1. Introduction

Throughout, H denotes a real Hilbert space and A a multi-valued operator with domain D(A). We know that A is called monotone if 〈u - v, x - y〉 ≥ 0, for any u ∈ Ax, v ∈ Ay; maximal monotone if its graph G(A) = {(x,y): x ∈ D(A), y ∈ Ax} is not properly contained in the graph of any other monotone operator. Denote by S: = {x ∈ D(A): 0 ∈ Ax} the zero set and by J c : = (I + cA)-1 the resolvent of A. It is well known that J c is single valued and D(J c ) = H for any c > 0.

A fundamental problem of monotone operators is that of finding an element x so that 0 ∈ Ax. This problem is essential because it includes many concrete examples, such as convex programming and monotone variational inequalities. A successful and powerful algorithm for solving this problem is the well-known proximal point algorithm (PPA), which generates, for any initial guess, x0 ∈ H, an iterative sequence as

x n + 1 = J c n ( x n + e n ) ,
(1.1)

where (c n ) is a positive real sequence and (e n ) is the error sequence (see [1]). To guarantee the convergence of PPA, there are two kinds of accuracy criterion posed on the error sequence:

(I) e n ≤ ε n , ∑ n = 0 ∞ ε n < ∞ or
(II) e n ≤ η n x ̃ n - x n , ∑ n = 0 ∞ η n < ∞ ,

where x ̃ n = J c n ( x n + e n ) . In 2001, Han and He [2] proved that in finite dimensional Hilbert space criterion (II) can be replaced by

(II’) e n ≤ η n x ̃ n - x n , ∑ n = 0 ∞ η n 2 < ∞ .

The infinite version was obtained by Marino and Xu [3].

There are various generations or modifications on the PPA. Among them Eckstein and Bertsekas [4] proposed the relaxed proximal point algorithm (RPPA):

x n + 1 = ( 1 - ρ n ) x n + ρ n J c n ( x n ) + e n ,
(1.2)

where (ρ n ) ⊂ (0, 2) is a relaxation factor. The weak convergence of (1.2) is guaranteed provided that (e n ) satisfies criterion (I),

c n ≥ c ̄ > 0 , 0 < δ ≤ ρ n ≤ 2 - δ .
(1.3)

On the other hand, since the PPA does not necessarily converge strongly (see [5]), many authors have conducted worthwhile studies on modifying the PPA so that the strong convergence is guaranteed (see, for instance, [6–8]). In particular, Marino and Xu [3] proposed the contraction-proximal point algorithm (CPPA):

x n + 1 = λ n u + ( 1 - λ n ) J c n ( x n ) + e n ,
(1.4)

where the parameters above satisfy (i) lim n λ n = 0, Σ n λ n = ∞; (ii) either Σ n |λ n +1- λ n | < ∞; or lim n λn/λ n +1 = 1; (iii) 0< c ≤ c n ≤ c ̄ <∞, ∑ n | c n + 1 - c n |<∞; (iv) Σ n ||e n || < ∞. Under these assumptions, the CPPA converges strongly to P S (u), the projection of u onto S.

In this article, we shall focus on the RPPA and CPPA. We note that the resolvent is in fact the arithmetic mean of the identity and a nonexpansive operator. By using this fact, we relax or remove some sufficient conditions to guarantee the convergence of the algorithms. As a result, we extend and improve some recent results on the PPA.

2. Some lemmas

We know that an operator T : H → H is called (i) nonexpansive if ||Tx - Ty|| ≤ || x - y|| ∀x,y ∈ H; and (ii) firmly nonexpansive if 〈Tx - Ty, x - y〉 ≥ ||Tx - Ty||2 ∀x,y ∈ H. Denote by Fix(T) = {x ∈ H : x = Tx} the fixed point set of T. It is well known that firmly nonexpansive operators have the following properties.

Lemma 1 (Goebel-Kirk [9]). Let T be firmly nonexpansive. Then (1) 2T - I is nonexpansive; (2) 〈Tx - x, Tx - z〉 ≤ 0 for all x ∈ H and for all z ∈ H Fix(T).

It is well known that J c is firmly nonexpansive and consequently nonexpansive; moreover, S = Fix (J c ). Since the fixed point set of nonexpansive operators is closed convex, the projection P s onto the solution set S is well defined whenever S ≠ ∅. Hereafter, we assume that S is nonempty. The following lemmas play an important role in our convergence analysis.

Lemma 2 (resolvent identity [3]). Let c, t > 0. Then for any x ∈ H,

J c x = J t t c x + 1 - t c J c x .

Lemma 3 ([10]). Let (ρ n ) be real sequence satisfying

0 < l i m i n f n → ∞ ρ n ≤ l i m s u p n → ∞ ρ n < 1.

Assume that (x n ) and (y n ) are bounded sequences in H satisfying x n +1 = (1 - ρ n )x n + ρ n y n . If

limsup n → ∞ ( y n + 1 - y n - x n + 1 - x n ) ≤ 0 ,

then lim n →∞||x n -y n || = 0.

Lemma 4 For r, s, > 0, let T r = 2J r - I. Then for any x ∈ H,

T s x - T r x ≤ 1 - s r x - T r x .
(2.1)

Proof. Using the resolvent identity, we have

T s x - T r x = 2 J s x - J s s r x + 1 - s r J r x ≤ 2 x - s r x + 1 - s r J r x = 2 1 - s r x - J r x = 1 - s r x - T r x ,

where the inequality uses the nonexpansive property of the resolvent.

Lemma 5 ([11]). Let (ε n ) and (s n ) be positive real sequences. Assume that Σ n ε n < ∞. If either (i) s n+ 1≤ (1 + ε n )s n , or (ii) s n+ 1≤ ε n , then the limit of (s n ) exists.

3. The relaxed proximal point algorithm

Under criterion (II'), Ceng et al. [12] considered another type, RPPA:

x ̃ n = J c n ( x n + e n ) , x n + 1 = ( 1 - ρ n ) x n + ρ n x ̃ n ,
(3.1)

and proved the weak convergence of (3.1) under the assumptions:

c n ≥ c ̄ > 0 , 0 < δ ≤ ρ n ≤ 1 .

We note that the choice of (ρ n ) excludes the case whenever ρ n ∈ (1,2), the overrelaxation. The overrelaxation, however, may indeed speed up the convergence of the algorithm (see [13]). Below, we shall improve their conditions on the relaxation factor from 0 < δ ≤ ρ n ≤ 1 to 0 < δ ≤ ρ n ≤ 2 - δ.

Theorem 6. Assume that the following conditions hold:

(a) c n ≥ c ̄ >0;

(b) 0 < δ ≤ ρ n ≤ 2 - δ;

(c) ∑ n e n ≤ η n x ̃ n - x n , ∑ n η n 2 <∞.

Then the sequence generated by (3.1) converges weakly to a point in S.

Proof. The key point of our proof is to show lim n s n = 0, where s n = x n - J c n ( x n ) . To see this, let z ∈ S be fixed. Since J c n is firmly nonexpansive and z∈ Fix ( J c n ) , applying Lemma 1 yields ⟨ x ̃ n - z , x ̃ n - x n - e n ⟩ ≤0. This together with (3.1) enables us to get

x n + 1 - z 2 - x n - z 2 = ( x n - z ) + ρ n ( x ̃ n - x n ) 2 - x n - z 2 = 2 ρ n ⟨ x n - z , x ̃ n - x n ⟩ + ρ n 2 x ̃ n - x n 2 = 2 ρ n ⟨ x ̃ n - z , x ̃ n - x n ⟩ - ρ n ( 2 - ρ n ) x ̃ n - x n 2 ≤ 2 ρ n ⟨ x ̃ n - z , e n ⟩ - ρ n ( 2 - ρ n ) x ̃ n - x n 2 = 2 ρ n ⟨ x ̃ n - x n , e n ⟩ + 2 ρ n ⟨ x n - z , e n ⟩ - ρ n ( 2 - ρ n ) x ̃ n - x n 2 ≤ 2 ρ n e n x ̃ n - x n + 2 ρ n e n x n - z - ρ n ( 2 - ρ n ) x ̃ n - x n 2 ≤ 2 ρ n η n x ̃ n - x n 2 + 2 ρ n η n x ̃ n - x n x n - z - ρ n ( 2 - ρ n ) x ̃ n - x n 2 .

Using the basic inequality 2ab ≤ a2 / ε + εb2 (a,b ∈ ℝ, ε > 0), we arrive at

2 ρ n η n x n - z x ̃ n - x n ≤ 2 ρ n 2 - ρ n η n x n - z 2 + 2 - ρ n 2 ρ n ρ n x ̃ n - x n 2 = 2 ρ n η n 2 2 - ρ n x n - z 2 + ρ n ( 2 - ρ n ) 2 x ̃ n - x n 2 ≤ 2 ( 2 - δ ) η n 2 δ x n - z 2 + ρ n ( 2 - ρ n ) 2 x ̃ n - x n 2 = ε n x n - z 2 + ρ n ( 2 - ρ n ) 2 x ̃ n - x n 2 ,

where ε n =2 ( 2 - δ ) η n 2 ∕δ is a summable sequence. Substituting this into above yields

x n + 1 - z 2 ≤ ( 1 + ε n ) x n - z 2 - ρ n ( 2 - ρ n - 4 η n ) 2 x ̃ n - x n 2 .

Since by Lemma 5 the limit of ||x n - z ||2 exists and lim inf n ρ n (2 - ρ n -4η n ) ≥ δ (2 - δ), this implies that x ̃ n - x n →0. On the other hand, we note that for all n ∈ ℕ

s n ≤ ( 1 + η n ) x n - x ̃ n → 0 ;

therefore, lim n s n = 0. The rest proof is similar to that of [12, Theorem 3.1].

We now turn to the RPPA (1.2). Under the criterion (I), the assumptions on relaxation factors can be relaxed to Σρ n (2 - ρ n ) = ∞ (see [3, Theorem 3.3]). Since the proof there is very technical, we wang to restate this result with a simple proof.

Theorem 7. Assume that the following conditions hold:

(a) Σ n ||e n || < ∞;

(b) Σ n ρn(2 - ρ n ) = ∞;

(c) 0< c ̄ ≤ c n ≤ c ̃ <∞;

(d) Σ n |c n +1- c n | < ∞.

Then the sequence generated by (1.2) converges weakly to a point in S.

Proof. The key step is to show lim n s n = 0, where s n = x n - J c n ( x n ) . It has been shown that Σ n ρ n (2 - ρ n )s n < ∞ (see [3, Lemma 3.2]). Therefore, it remains to show that lim n s n exists. By letting T n = 2J n - I, we rewrite (2) as

x n + 1 = 1 - ρ n 2 x n + ρ n 2 T n x n + e n .

In view of Lemma 4 and condition (c),

∥ T n + 1 x n + 1 - T n x n ∥ ≤ ∥ T n + 1 x n + 1 - T n + 1 x n ∥ + ∥ T n + 1 x n - T n x n ∥ ≤ ∥ x n - x n + 1 ∥ + ∥ T n + 1 x n - T n x n ∥ ≤ ∥ x n - x n + 1 ∥ + 1 - c n + 1 c n ∥ T n x n - x n ∥ ≤ ∥ x n - x n + 1 ∥ + | c n + 1 - c n | c ̄ ∥ T n x n - x n ∥ ≤ ∥ x n - x n + 1 ∥ + M | c n + 1 - c n | ,

where M > 0 is a suitable number. Consequently,

x n + 1 - T n + 1 x n + 1 = 1 - ρ n 2 x n + ρ n 2 T n x n + e n - T n + 1 x n + 1 = 1 - ρ n 2 ( x n - T n x n ) + ( T n x n - T n + 1 x n + 1 ) + e n ≤ 1 - ρ n 2 x n - T n x n + T n x n - T n + 1 x n + 1 + e n ≤ 1 - ρ n 2 x n - T n x n + x n - x n + 1 + M | c n + 1 - c n | + e n = 1 - ρ n 2 x n - T n x n + ρ n 2 ( x n - T n x n ) + e n + M | c n + 1 - c n | + e n ≤ x n - T n x n + M | c n + 1 - c n | + 2 e n .

Using s n = || x n - T n x n ||/2, we therefore arrive at

s n + 1 ≤ s n + σ n ,

where σ n = 2M |c n +1- c n | + 4||e n || satisfying Σ n σ n < ∞ (due to (a) and (d)). By Lemma 5, we finally conclude that lim n s n = 0.

4. The contraction-proximal point algorithm

Recently, Yao and Noor [14] extended the CPPA to the following form:

x n + 1 = λ n u + r n x n + δ n J c n ( x n ) + e n ,
(4.1)

where (λ n ),(r n ),(δ n )⊆ (0,1) and λ n + r n + δ n = 1. They proved the strong convergence of the algorithm provided that (i) c n ≥ c ̄ >0, lim n | c n + 1 - c n |=0; (ii) 0 < lim inf n r n ≤ lim sup n r n < 1; and (iii) Σ n ||e n || < ∞. Also, they claimed that their algorithm includes the CPPA as a special case. This is, however, not the case, because condition (ii) excludes the special case r n ≡ 0. To overcome this drawback, we shall show the same result by replacing condition (ii) with the weak condition:

l i m s u p n → ∞ r n <1 ⇔ l i m i n f n → ∞ δ n >0.

In this situation, the CPPA is evidently a special case of algorithm (4.1). The idea of the following proof is followed by the second author [15].

Theorem 8. Let be (λ n ), (r n ) and (δ n ) be parameters in (4.1). Assume that the following conditions hold:

(a) lim n λ n = 0, Σ n λ n = ∞;

(b) lim sup n r n < 1 ⇔ lim inf n δ n > 0;

(c) c n ≥ c ̄ >0,| c n + 1 - c n |→0;

(d) Σ n ||e n || < ∞.

Then the sequence generated by (4.1) converges strongly to P S (u).

Proof. All we need to do is to prove ||x n +1- x n || → 0, since the rest proof is similar to that of [14, Theorem 3.3]. To this end, set J n = J c n and T n = 2J n - I. It then follows from (4.1) that

x n + 1 = λ n u + r n x n + δ n 2 ( I + T n ) x n + e n = r n + δ n 2 x n + λ n u + δ n 2 T n x n + e n .

Let ρ n = λ n + (δ n /2). Then the algorithm has the form:

x n + 1 = ( 1 - ρ n ) x n + ρ n y n ,
(4.2)

where y n = (2λ n u + δ n T n x n + 2e n )/2ρ n. Using nonexpansiveness of T n and Lemma 4, we have

T n + 1 x n + 1 - T n x n ≤ T n + 1 x n + 1 - T n + 1 x n + T n + 1 x n - T n x n ≤ x n + 1 - x n + 1 - c n + 1 c n T n x n - x n ≤ x n + 1 - x n + | c n - c n + 1 | c ̄ T n x n - x n .
(4.3)

On the other hand, it follows from the definition of y n that

y n + 1 - y n = 1 2 ρ n + 1 ( 2 λ n + 1 u + δ n + 1 T n + 1 x n + 1 + 2 e n + 1 ) - 1 2 ρ n ( 2 λ n u + δ n T n x n + 2 e n ) ≤ λ n + 1 ρ n + 1 - λ n ρ n u + e n + 1 ρ n + 1 + e n ρ n + δ n + 1 2 ρ n + 1 T n + 1 x n + 1 - δ n 2 ρ n T n x n ≤ λ n + 1 ρ n + 1 - λ n ρ n u + e n + 1 ρ n + 1 + e n ρ n + δ n + 1 2 ρ n + 1 - δ n 2 ρ n T n + 1 x n + 1 + δ n 2 ρ n T n + 1 x n + 1 - T n x n .
(4.4)

Since (x n ) is bounded and T n is nonexpansive, we can find M > 0 so that (||T n x n || + ||x n || + ||u||) ≤ M for all n ∈ ℕ Adding (4.3) and (4.4) and noting δ n ≤ 2ρ n yield

y n + 1 - y n ≤ λ n + 1 ρ n + 1 - λ n ρ n u + e n + 1 ρ n + 1 + e n ρ n + δ n + 1 2 ρ n + 1 - δ n 2 ρ n T n + 1 x n + 1 + x n + 1 - x n + | c n - c n + 1 | c ̄ T n x n - x n ≤ x n + 1 - x n + M λ n + 1 ρ n + 1 - λ n ρ n + e n + 1 ρ n + 1 + e n ρ n + δ n + 1 2 ρ n + 1 - δ n 2 ρ n + | c n - c n + 1 | c ̄ .

With the knowledge that ||e n ||→ 0 and

λ n ρ n = 2 λ n 2 λ n + δ n → 0 , δ n 2 ρ n = δ n 2 λ n + δ n → 1 ,

we therefore deduce from (b) and (c) that

l i m s u p n → ∞ ( ‖ y n + 1 − y n ‖ − ‖ x n + 1 − x n ‖ ) ≤ l i m s u p n → ∞ M ( | λ n + 1 ρ n + 1 − λ n ρ n | + ‖ e n + 1 ‖ ρ n + 1 + ‖ e n ‖ ρ n + | δ n + 1 2 ρ n + 1 − δ n 2 ρ n | + | c n − c n + 1 | c ¯ ) → 0.

Note that lim inf n ρ n = lim inf n (δ n /2)> 0 and lim sup n ρ n = lim sup n (δ n /2) ≤ 1/2 < 1. On the other hand, it is easy to check that (x n ) is bounded and so is (y n ) We therefore apply Lemma 3 to yield lim n ||x n - y n || = 0. By means of (4.2), we finally have

x n + 1 - x n = ρ n x n - y n → ,

and thus the required result at once follows.

As a corollary, we improve [3, Theorem 4.1] as follows.

Theorem 9. Assume that the following conditions hold:

(a) lim n λ n = 0, Σ n λ n = ∞;

(b) c n ≥ c ̄ >0,| c n + 1 - c n |→0;

(c) Σ n ||e n || < ∞.

Then the sequence generated by (1.4) converges strongly to P S (u).

Abbreviations

CPPA:

contraction-proximal point algorithm

PPA:

proximal point algorithm

RPPA:

relaxed proximal point algorithm.

References

  1. 1.

    Rockafellar RT: Monotone operators and the proximal point algorithm. SIAM J Control Optim 1976, 14: 877–898. 10.1137/0314056

    MathSciNet  Article  Google Scholar 

  2. 2.

    Han D, He BS: A new accuracy criterion for approximate proximal point algorithms. J Math Anal Appl 2001, 263: 343–354. 10.1006/jmaa.2001.7535

    MathSciNet  Article  Google Scholar 

  3. 3.

    Marino G, Xu HK: Convergence of generalized proximal point algorithm. Comm Pure Appl Anal 2004, 3: 791–808.

    MathSciNet  Article  Google Scholar 

  4. 4.

    Eckstein J, Bertsekas DP: On the Douglas-Rachford splitting method and the proximal points algorithm for maximal monotone operators. Math Programming 1992, 55: 293–318. 10.1007/BF01581204

    MathSciNet  Article  Google Scholar 

  5. 5.

    Güler O: On the convergence of the proximal point algorithm for convex optimization. SIAM J Control Optim 1991, 29: 403–419. 10.1137/0329022

    MathSciNet  Article  Google Scholar 

  6. 6.

    Bauschke HH, Combettes PL: A weak-to-strong convergence principle for Fejér-monotone methods in Hilbert spaces. Math Oper Res 2001, 26: 248–264. 10.1287/moor.26.2.248.10558

    MathSciNet  Article  Google Scholar 

  7. 7.

    Solodov MV, Svaiter BF: Forcing strong convergence of proximal point iterations in a Hilbert space. Math Programming Ser 2000, 87: 189–202.

    MathSciNet  Google Scholar 

  8. 8.

    Xu HK: Iterative algorithms for nonlinear operators. J Lond Math Soc 2002, 66: 240–256. 10.1112/S0024610702003332

    Article  Google Scholar 

  9. 9.

    Goebel K, Kirk WA: Topics on Metric Fixed Point Theory. Cambridge University Press, Cambridge; 1990.

    Google Scholar 

  10. 10.

    Suzuki T: A sufficient and necessary condition for Halpern-type strong convergence to fixed points of nonexpansive mappings. Proc Am Math Soc 2007, 135: 99–106.

    Article  Google Scholar 

  11. 11.

    Tan KK, Xu HK: Approximating fixed points of nonexpansive mappings by the Ishikawa iteration process. J Math Anal Appl 1993, 178: 301–308. 10.1006/jmaa.1993.1309

    MathSciNet  Article  Google Scholar 

  12. 12.

    Ceng LC, Wu SY, Yao JC: New accuracy criteria for modified approximate proximal point algorithms in Hilbert space. Taiwan J Math 2008, 12: 1691–1705.

    MathSciNet  Google Scholar 

  13. 13.

    Eckstein J, Ferris MC: Operator-splitting methods for monotone affine variational inequalities, with a parallel application to optimal control. INFORMS J Comput 1998, 10: 218–235. 10.1287/ijoc.10.2.218

    MathSciNet  Article  Google Scholar 

  14. 14.

    Yao Y, Noor MA: On convergence criteria of generalized proximal point algorithms. J Comput Appl Math 2008, 217: 46–55. 10.1016/j.cam.2007.06.013

    MathSciNet  Article  Google Scholar 

  15. 15.

    Wang F: A note on the regularized proximal point algorithm. J Global Optim 2011, 50: 531–535. 10.1007/s10898-010-9611-z

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgements

The authors would like to express thier sincere thanks to the referees for their valuable suggestions. This study is supported by the Natural Science Foundation of Department of Education, Henan(2011B110023).

Author information

Affiliations

  1. Department Of Mathematics, Luoyang Normal University, Luoyang, 471022, China

    Shuyu Wang & Fenghui Wang

Authors
  1. Shuyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fenghui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuyu Wang.

Additional information

Authors' contributions

Both authors contributed equally to this work. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Wang, S., Wang, F. On relaxed and contraction-proximal point algorithms in hilbert spaces. J Inequal Appl 2011, 41 (2011). https://doi.org/10.1186/1029-242X-2011-41

Download citation

Keywords

  • maximal monotone operator
  • proximal point algorithm
  • firmly nonexpansive operator