A trust region spectral method for largescale systems of nonlinear equations
 Meilan Zeng^{1} and
 Guanghui Zhou^{2}Email authorView ORCID ID profile
https://doi.org/10.1186/s136600161117x
© Zeng and Zhou 2016
Received: 27 February 2016
Accepted: 14 June 2016
Published: 7 July 2016
Abstract
The spectral gradient method is one of the most effective methods for solving largescale systems of nonlinear equations. In this paper, we propose a new trust region spectral method without gradient. The trust region technique is a globalization strategy in our method. The global convergence of the proposed algorithm is proved. The numerical results show that our new method is more competitive than the spectral method of La Cruz et al. (Math. Comput. 75(255):14291448, 2006) for largescale nonlinear equations.
Keywords
nonlinear equations trust region spectral method largescale problemMSC
65H10 90C061 Introduction
The above trust region methods are particularly effective for small to mediumsized systems of nonlinear equations; however, the computation and storage loads can greatly increase with increased dimension.
The paper is organized as follows. Section 2 introduces the new algorithm. The convergence theory is presented in Section 3. Section 4 demonstrates preliminary numerical results on test problems.
2 New algorithm
Now we present our algorithm for solving (1). The algorithm is given as follows.
Algorithm 1
 Step 0.:

Choose \(0<\eta_{1} <\eta_{2} < 1\), \(0<\beta_{1}<1 <\beta_{2}\), \(\epsilon>0\). Initialize \(x_{0}\), \(0<\Delta_{0} <\bar{\Delta}\). Set \(k:=0\).
 Step 1.:

Evaluate \(F_{k}\), if \(\Vert F_{k}\Vert \leq\epsilon\), then terminate.
 Step 2.:

Solve the trust region subproblem (5) to obtain \(d_{k}\).
 Step 3.:

ComputeIf \(r_{k} < \eta_{1}\), then \(\Delta_{k}= \beta_{1} \Delta_{k}\), go to Step 2. Otherwise, go to Step 4.$$ r_{k} = \frac{\mathit{Ared}_{k}(d_{k})}{\mathit{Pred}_{k}(d_{k})}. $$(10)
 Step 4.:

\(x_{k+1}=x_{k} + d_{k}\);Compute \(\gamma_{k+1}\) by (7). Set \(k:=k+1\), go to Step 1.$$\Delta_{k + 1} = \textstyle\begin{cases} \min\{\beta_{2} \Delta_{k}, \bar{\Delta}\}, & \text{if } r_{k} \geq\eta_{2},\\ \Delta_{k}, & \text{otherwise}. \end{cases} $$
3 Convergence analysis
In this section, we prove the global convergence of Algorithm 1. The global convergence of Algorithm 1 needs the following assumptions.
Assumption A
 (1)
The level set \(\Omega=\{x\in R^{n} \vert f(x)\leq f(x_{0}) \} \) is bounded.
 (2)The following relation holds:$$\bigl\Vert [J_{k}\gamma_{k}I]^{T} F_{k}\bigr\Vert =O\bigl(\Vert d_{k}\Vert \bigr). $$
Then we get the following lemmas.
Lemma 3.1
\(\vert \mathit{Ared}_{k}(d_{k})\mathit{Pred}_{k}(d_{k})\vert =O(\Vert d_{k}\Vert ^{2})\).
Proof
Similar to Zhang and Wang [15], or Yuan et al. [16], we obtain the following result.
Lemma 3.2
Proof
Lemma 3.3
Algorithm 1 does not circle between Step 2 and Step 3 infinitely.
Proof
If Algorithm 1 circles between Step 2 and Step 3 infinitely, then for all \(i=1,2,\ldots\) , we have \(x_{k+i}=x_{k}\), and \(\Vert F_{k}\Vert > \epsilon\), which implies that \(r_{k} < \eta_{1}\), \(\Delta_{k}\to0\).
Lemma 3.4
Let Assumption A hold and \(\{x_{k}\}\) be generated by Algorithm 1, then \(\{x_{k}\}\subset\Omega\). Moreover, \(\{ f(x_{k})\}\) converges.
Proof
The following theorem shows that Algorithm 1 is global convergent under the conditions of Assumption A.
Theorem 3.5
Proof
4 Numerical experiments
In this section, the recent spectral method in [1] is called Algorithm 2. We report results of some numerical experiments of Algorithms 1 and 2. We choose 14 test functions as follows (see [4, 6, 17]).
Function 1
Function 2
Initial guess: \(x_{0}=(50,0,\ldots,50,0)\).
Function 3
Function 4
Function 5
Function 6
Function 7
Function 8
Function 9
Function 10
Function 11
Function 12
Function 13
Function 14
In the experiments, the parameters are chosen as \(\Delta_{0}=1\), \(\bar {\Delta}=10\), \(\epsilon=10^{5}\), \(\eta_{1}=0.001\), \(\eta_{2}=0.75\), \(\beta_{1} =0.5\), \(\beta_{2} =2.0\), \(M=10\), \(\eta_{k}=1/(k+1)^{2}\), \(\alpha=0.5\), where ϵ is the stop criterion. The program is also stopped if the iteration number is larger than 5,000. We obtain \(d_{k}\) by (5) from the Dogleg method in [18]. The program is coded in MATLAB 2009a.
To show the performance of two algorithms, we use the performance profile proposed by Dolan and Moré [19]. The dimensions of 14 test functions are 100, 1,000, 10,000. According to the numerical results, we plot two figures based on the total number of iterations and the CPU time, respectively.
Declarations
Acknowledgements
We thank the reviewers and the editors for their valuable suggestions and comments which improve this paper greatly. This work is supported by the Science and Technology Foundation of the Department of Education of Hubei Province (D20152701) and the Foundations of Education Department of Anhui Province (KJ2016A651; 2014jyxm161).
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 La Cruz, W, Jose, MM, Marcos, R: Spectral residual method without gradient information for solving largescale nonlinear systems of equations. Math. Comput. 75(255), 14291448 (2006) MathSciNetView ArticleMATHGoogle Scholar
 Bouaricha, A, Schnabel, RB: Tensor methods for large sparse systems of nonlinear equations. Math. Program. 82(3), 377400 (1998) MathSciNetView ArticleMATHGoogle Scholar
 Bergamaschi, L, Moret, I, Zilli, G: Inexact quasiNewton methods for sparse systems of nonlinear equations. Future Gener. Comput. Syst. 18(1), 4153 (2001) View ArticleMATHGoogle Scholar
 La Cruz, W, Raydan, M: Nonmonotone spectral methods for largescale nonlinear systems. Optim. Methods Softw. 18(5), 583599 (2003) MathSciNetView ArticleMATHGoogle Scholar
 Li, Q, Li, D: A class of derivativefree methods for largescale nonlinear monotone equations. IMA J. Numer. Anal. 31, 16251635 (2011) MathSciNetView ArticleMATHGoogle Scholar
 Yuan, GL, Zhang, MJ: A threeterms PolakRibièPolyak conjugate gradient algorithm for largescale nonlinear equations. J. Comput. Appl. Math. 286, 186195 (2015) MathSciNetView ArticleMATHGoogle Scholar
 Yuan, GL, Meng, ZH, Li, Y: A modified Hestenes and Stiefel conjugate gradient algorithm for largescale nonsmooth minimizations and nonlinear equations. J. Optim. Theory Appl. 168, 129152 (2016) MathSciNetView ArticleMATHGoogle Scholar
 Barzilai, J, Borwein, JM: Twopoint step size gradient methods. IMA J. Numer. Anal. 8(1), 141148 (1988) MathSciNetView ArticleMATHGoogle Scholar
 Birgin, EG, Martinez, JM, Raydan, M: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23(4), 539559 (2003) MathSciNetView ArticleMATHGoogle Scholar
 Dai, YH, Zhang, H: Adaptive twopoint stepsize gradient algorithm. Numer. Algorithms 27(4), 377385 (2001) MathSciNetView ArticleMATHGoogle Scholar
 Dai, YH: Modified twopoint stepsize gradient methods for unconstrained optimization. Comput. Optim. Appl. 22(1), 103109 (2002) MathSciNetView ArticleMATHGoogle Scholar
 Raydan, M: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Control Optim. 7(1), 2633 (1997) MathSciNetView ArticleMATHGoogle Scholar
 Yuan, GL, Lu, XW, Wei, ZX: BFGS trustregion method for symmetric nonlinear equations. J. Comput. Appl. Math. 230, 4458 (2009) MathSciNetView ArticleMATHGoogle Scholar
 Yuan, YX: Trust region algorithm for nonlinear equations. Information 1, 721 (1998) MathSciNetMATHGoogle Scholar
 Zhang, JL, Wang, Y: A new trust region method for nonlinear equations. Math. Methods Oper. Res. 58(2), 283298 (2003) MathSciNetView ArticleMATHGoogle Scholar
 Yuan, GL, Wei, ZX, Liu, XW: A BFGS trustregion method for nonlinear equations. Computing 92(4), 317333 (2011) MathSciNetView ArticleMATHGoogle Scholar
 Moré, JJ, Garbow, BS, Hillstrom, KE: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 1741 (1981) MathSciNetView ArticleMATHGoogle Scholar
 Wang, YJ, Xiu, NH: Theory and Algorithm for Nonlinear Programming. Shanxi Science and Technology Press, Xi’an (2004) Google Scholar
 Dolan, ED, Moré, JJ: Benchmarking optimization software with performance profiles. Math. Program. 91, 201213 (2002) MathSciNetView ArticleMATHGoogle Scholar