A family of conjugate gradient methods for largescale nonlinear equations
 Dexiang Feng^{1, 2},
 Min Sun^{3} and
 Xueyong Wang^{3}Email author
https://doi.org/10.1186/s1366001715100
© The Author(s) 2017
Received: 10 July 2017
Accepted: 11 September 2017
Published: 22 September 2017
Abstract
In this paper, we present a family of conjugate gradient projection methods for solving largescale nonlinear equations. At each iteration, it needs low storage and the subproblem can be easily solved. Compared with the existing solution methods for solving the problem, its global convergence is established without the restriction of the Lipschitz continuity on the underlying mapping. Preliminary numerical results are reported to show the efficiency of the proposed method.
Keywords
MSC
1 Introduction
In this paper, motivated by the projection methods in [3, 11] and the conjugate gradient methods in [13, 14], we propose a new family of conjugate gradient projection methods for solving nonlinear problem (1.1). The new designed method is derivativefree as it does not need to compute the Jacobian matrix or its approximation of the underlying function (1.1). Further, the new method does not need to solve any linear equations at each iteration, thus it is suitable to solve large scale problem (1.1).
The remainder of this paper is organized as follows. Section 2 describes the new method and presents its convergence. The numerical results are reported in Section 3. Some concluding remarks are drawn in the last section.
2 Algorithm and convergence analysis
Now, we describe the new conjugate gradient projection method for nonlinear constrained equations.
Algorithm 2.1
 Step 0.:

Given an arbitrary initial point \(x_{0}\in R^{n}\), parameters \(0<\rho<1,\sigma>0,t>0,\beta>0,\epsilon>0\), and set \(k:=0\).
 Step 1.:

If \(\Vert F(x_{k}) \Vert <\epsilon\), stop; otherwise go to Step 2.
 Step 2.:

Computewhere \(\beta_{k}\) is such that$$ d_{k}= \textstyle\begin{cases}F(x_{k})& \text{if } k=0,\\  ( 1+\beta_{k}\frac{F(x_{k})^{\top}d_{k1}}{ \Vert F(x_{k}) \Vert ^{2}} )F(x_{k})+\beta_{k}d_{k1} &\text{if } k\geq1, \end{cases} $$(2.3)$$ \vert \beta_{k} \vert \leq t\frac{ \Vert F(x_{k}) \Vert }{ \Vert d_{k1} \Vert },\quad \forall k\geq1. $$(2.4)
 Step 3.:

Find the trial point \(y_{k}=x_{k}+\alpha_{k} d_{k}\), where \(\alpha_{k}=\beta\rho^{m_{k}}\) and \(m_{k}\) is the smallest nonnegative integer m such that$$ \bigl\langle F\bigl(x_{k}+\beta\rho^{m}d_{k} \bigr),d_{k}\bigr\rangle \geq\sigma\beta\rho^{m} \Vert d_{k} \Vert ^{2}. $$(2.5)
 Step 4.:

Computewhere$$ x_{k+1}=P_{H_{k}}\bigl[x_{k} \xi_{k}F(y_{k})\bigr], $$(2.6)with$$H_{k}=\bigl\{ x\in R^{n}h_{k}(x)\leq0\bigr\} , $$and$$ h_{k}(x)=\bigl\langle F(y_{k}),xy_{k} \bigr\rangle , $$(2.7)Set \(k:=k+1\) and go to Step 1.$$\xi_{k}=\frac{\langle F(y_{k}), x_{k}y_{k}\rangle}{ \Vert F(y_{k}) \Vert ^{2}}. $$
Obviously, Algorithm 2.1 is different from the methods in [13, 14].
Lemma 2.1
Proof
Lemma 2.1 indicates that the hyperplane \(\partial H_{k}=\{x\in R^{n}h_{k}(x)=0\}\) strictly separates the current iterate from the solutions of problem (1.1) if \(x_{k}\) is not a solution. In addition, from Lemma 2.1, we also can derive that the solution set \(S^{*}\) of problem (1.1) is included in \(H_{k}\) for all k.
Certainly, if Algorithm 2.1 terminates at step k, then \(x_{k}\) is a solution of problem (1.1). So, in the following analysis, we assume that Algorithm 2.1 always generates an infinite sequence \(\{x_{k}\}\). Based on the lemma, we can establish the convergence of the algorithm.
Theorem 2.1
If F is continuous and condition (2.1) holds, then the sequence \(\{x_{k}\}\) generated by Algorithm 2.1 globally converges to a solution of problem (1.1).
Proof
If \(\liminf_{k\rightarrow\infty} \Vert d_{k} \Vert =0\), then from (2.9) it holds that \(\liminf_{k\rightarrow\infty} \Vert F(x_{k}) \Vert =0\). From the boundedness of \(\{x_{k}\}\) and the continuity of \(F(\cdot)\), \(\{x_{k}\}\) has some accumulation point x̄ such that \(F(\bar{x})=0\). Then from (2.11), \(\{ \Vert x_{k}\bar{x} \Vert \}\) converges, and thus the sequence \(\{x_{k}\}\) globally converges to x̄.
3 Numerical results
Problem 1
Problem 2
Problem 3
Numerical results with different dimensions of Problem 1
Dimension  10  50  100  500  1,000 

Iter.  20  21  22  23  23 
CPU  0.3594  0.8750  1.7031  41.4844  257.1406 
Numerical results with different initial points of Problem 3
Initial point  Iter.  CPU  \(\boldsymbol {\Vert F(x_{k}) \Vert } \) 

(0,0,0,0)  20  0.9219  9.9320×10^{−7} 
(3,0,0,0)  22  0.8438  7.0942×10^{−7} 
(1,1,1,0)  41  1.6250  7.2649×10^{−7} 
(0,1,1,1)  28  1.1875  3.8145×10^{−7} 
(0,100,100,1)  34  1.4063  6.6320×10^{−7} 
4 Conclusion
In this paper, we extended the conjugate gradient method to nonlinear equations. The major advantage of the method is that it does not need to compute the Jacobian matrix or any linear equations at each iteration, thus it is suitable to solve largescale nonlinear constrained equations. Under mild conditions, the proposed method possesses global convergence.
In Step 4 of Algorithm 2.1, we have to compute a projection onto the intersection of the feasible set C and a halfspace at each iteration, which is equivalent to quadratic programming, quite timeconsuming work. Hence, how to remove this projection step is one of our future research topics.
Declarations
Acknowledgements
The authors thank anonymous referees for valuable comments and suggestions, which helped to improve the manuscript. This work is supported by the Natural Science Foundation of China (11671228).
Authors’ contributions
DXF and MS organized and wrote this paper. XYW examined all the steps of the proofs in this research and gave some advice. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Wang, YJ, Caccetta, L, Zhou, GL: Convergence analysis of a block improvement method for polynomial optimization over unit spheres. Numer. Linear Algebra Appl. 22, 10591076 (2015) MathSciNetView ArticleMATHGoogle Scholar
 Zeidler, E: Nonlinear Functional Analysis and Its Applications. Springer, Berlin (1990) View ArticleMATHGoogle Scholar
 Wang, CW, Wang, YJ: A superlinearly convergent projection method for constrained systems of nonlinear equations. J. Glob. Optim. 40, 283296 (2009) MathSciNetView ArticleMATHGoogle Scholar
 Wang, YJ, Caccetta, L, Zhou, GL: Convergence analysis of a block improvement method for polynomial optimization over unit spheres. Numer. Linear Algebra Appl. 22, 10591076 (2015) MathSciNetView ArticleMATHGoogle Scholar
 Wood, AJ, Wollenberg, BF: Power Generation, Operation, and Control. Wiley, New York (1996) Google Scholar
 Chen, HB, Wang, YJ, Zhao, HG: Finite convergence of a projected proximal point algorithm for the generalized variational inequalities. Oper. Res. Lett. 40, 303305 (2012) MathSciNetView ArticleMATHGoogle Scholar
 Dirkse, SP, Ferris, MC: MCPLIB: A collection of nonlinear mixed complementarity problems. Optim. Methods Softw. 5, 319345 (1995) View ArticleGoogle Scholar
 Wang, YJ, Qi, L, Luo, S, Xu, Y: An alternative steepest direction method for the optimization in evaluating geometric discord. Pac. J. Optim. 10, 137149 (2014) MathSciNetMATHGoogle Scholar
 Zhang, L, Zhou, W: Spectral gradient projection method for solving nonlinear monotone equations. J. Comput. Appl. Math. 196, 478484 (2006) MathSciNetView ArticleMATHGoogle Scholar
 Yu, ZS, Lin, J, Sun, J, Xiao, YH, Liu, LY, Li, ZH: Spectral gradient projection method for monotone nonlinear equations with convex constraints. Appl. Numer. Math. 59, 24162423 (2009) MathSciNetView ArticleMATHGoogle Scholar
 Ma, FM, Wang, CW: Modified projection method for solving a system of monotone equations with convex constraints. J. Appl. Math. Comput. 34, 4756 (2010) MathSciNetView ArticleMATHGoogle Scholar
 Zheng, L: A new projection algorithm for solving a system of nonlinear equations with convex constraints. Bull. Korean Math. Soc. 50, 823832 (2013) MathSciNetView ArticleMATHGoogle Scholar
 Sun, M, Wang, YJ, Liu, J: Generalized PeacemanRachford splitting method for multipleblock separable convex programming with applications to robust PCA. Calcolo 54, 7794 (2017) MathSciNetView ArticleMATHGoogle Scholar
 Li, M, Qu, AP: Some sufficient descent conjugate gradient methods and their global convergence. Comput. Appl. Math. 33, 333347 (2014) MathSciNetView ArticleMATHGoogle Scholar
 Karamardian, S: Complementarity problems over cones with monotone and pseudomonotone maps. J. Optim. Theory Appl. 18, 445454 (1976) MathSciNetView ArticleMATHGoogle Scholar
 Xiao, YH, Zhu, H: A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 405, 310319 (2013) MathSciNetView ArticleMATHGoogle Scholar