 Research
 Open Access
A tensor trustregion model for nonlinear system
 Songhua Wang^{1}Email author and
 Shulun Liu^{2}
https://doi.org/10.1186/s1366001819350
© The Author(s) 2018
 Received: 17 October 2018
 Accepted: 3 December 2018
 Published: 13 December 2018
Abstract
It has turned out that the tensor expansion model has better approximation to the objective function than models of the normal second Taylor expansion. This paper conducts a study of the tensor model for nonlinear equations and it includes the following: (i) a three dimensional symmetric tensor trustregion subproblem model of the nonlinear equations is presented; (ii) the three dimensional symmetric tensor is replaced by interpolating function and gradient values from the most recent past iterate, which avoids the storage of the three dimensional symmetric tensor and decreases the workload of the computer; (iii) the limited BFGS quasiNewton update is used instead of the second Jacobian matrix, which generates an inexpensive computation of a complex system; (iv) the global convergence is proved under suitable conditions. Numerical experiments are done to show that this proposed algorithm is competitive with the normal algorithm.
Keywords
 Tensor model
 Trust region
 Nonlinear equations
 BFGS formula
 Convergence
MSC
 65K05
 90C26
1 Introduction
It is not difficult to see that the above models only get the second Taylor expansion and approximation. Can we get the approximation to reach one more level, namely the third expansion, or even the fourth? The answer is positive and a third Taylor expansion is used and a three dimensional symmetric tensor model is stated. In the next section, the motivation and the tensor TR model are stated. The algorithm and its global convergence are presented in Sect. 3. In Sect. 4, we do the experiments of the algorithms. One conclusion is given in the last section.
2 Motivation and the tensor trustregion model
Algorithm 1
 Initial: :

Constants ρ, \(c\in (0,1)\), \(p=0\), \(\epsilon >0\), \(x_{0}\in \Re^{n},\,m>0\), and \(B_{0}=H_{0}^{1}\in \Re^{n}\times \Re ^{n}\) is a symmetric and positive definite matrix. Let \(k:=0\);
 Step 1: :

Stop if \(\Vert S(x_{k}) \Vert <\epsilon \) holds;
 Step 2: :

Solve (2.3) with \(\triangle =\triangle_{k}\) to obtain \(d_{k}^{p}\);
 Step 3: :

Compute \(Ad_{k}(d_{k}^{p})\), \(Pd_{k}(d_{k}^{p})\), and the radio \(r_{k}^{p}\). If \(r_{k}^{p}<\rho \), let \(p=p+1\), go to Step 2. If \(r_{k}^{p}\geq \rho \), go to the next step;
 Step 4: :

Set \(x_{k+1}=x_{k}+d_{k}^{p}\), \(y_{k}=S(x_{k+1})S(x _{k})\), update \(B_{k+1}=H_{k+1}^{1}\) by (2.4) if \(y_{k}^{T}d _{k}^{p}>0\), otherwise set \(B_{k+1}=B_{k}\);
 Step 5: :

Let \(k:=k+1\) and \(p=0\). Go to Step 1.
Remark
The procedure of “Step 2–Step 3–Step 2” is called the inner cycle in the above algorithm. It is necessary for us to prove that the inner cycle is finite, which generates the circumstance that Algorithm 1 is well defined.
3 Convergence results
This section focuses on convergence results of Algorithm 1 under the following assumptions.
Assumption i
 (A) :

The level set Ω defined byis bounded.$$ \varOmega =\bigl\{ x\mid\beta (x)\leq \beta (x_{0})\bigr\} $$(3.1)
 (B) :

On an open convex set \(\varOmega_{1}\) containing Ω, the nonlinear system \(S(x)\) is twice continuously differentiable.
 (C) :

The approximation relationis true, where \(d_{k}^{p}\) is the solution of the model (2.3).$$ \bigl\Vert \bigl[\nabla S(x_{k})B_{k} \bigr]S(x_{k}) \bigr\Vert =O\bigl(\bigl\Vert d_{k}^{p} \bigr\Vert \bigr) $$(3.2)
 (D) :

On \(\varOmega_{1}\), the sequence matrices \(\{B_{k}\}\) are uniformly bounded, namely there exist constants \(0< M_{0}\leq M\) satisfying$$ M_{s}\leq \Vert B_{k} \Vert \le M_{l} \quad \forall k. $$(3.3)
Based on the above assumptions and the definition of the model (2.3), we have the following lemma.
Lemma 3.1
Proof
Lemma 3.2
Proof
Lemma 3.3
Let the conditions of Lemma 3.2 hold. We conclude that Algorithm 1 does not infinitely circle in the inner cycle (“Step 2–Step 3–Step 2”).
Proof
This lemma will be proved by contradiction. Suppose, at \(x_{k}\), that Algorithm 1 infinitely circles in the inner cycle, namely, \(r_{k}^{p}<\rho \) and \(c^{p}\rightarrow 0\) with \(p\rightarrow \infty \). This implies that \(\Vert g_{k} \Vert \geq \epsilon \), or the algorithm stops. Thus we conclude that \(\Vert d_{k}^{p} \Vert \leq \triangle_{k}=c^{p}\Vert g_{k} \Vert \rightarrow 0\) is true.
Lemma 3.4
Suppose that the conditions of Lemma 3.3 holds. Then we conclude that \(\{x_{k}\}\subset \varOmega \) is true and \(\{\beta (x_{k})\}\) converges.
Proof
Theorem 3.5
Proof
4 Numerical results
This section reports some numerical results of Algorithm 1 and the algorithm of [35] (Algorithm YL).
4.1 Problems
Problem 1
Problem 2
Problem 3
Problem 4
Problem 5
Problem 6
Problem 7
Problem 8
Problem 9
Problem 10
Parameters: \(\rho =0.05\), \(\epsilon =10^{4}\), \(c=0.5\), \(p=3\), \(m=6\), \(H_{0}\) is the unit matrix.
The method for ( 1.3 ) and ( 2.3 ): the \(Dogleg\) method [22].
Codes experiments: run on a PC with an Intel Pentium(R) Xeon(R) E5507 CPU @2.27 GHz, 6.00 GB of RAM, and the Windows 7 operating system.
Codes software: MATLAB r2017a.
Stop rules: the program stops if \(\Vert S(x) \Vert \leq 1e{}4\) holds.
Other cases: we will stop the program if the iteration number is larger than a thousand.
4.2 Results and discussion
The column meaning of the tables is as follows.
Dim: the dimension.
NI: the iterations number.
NG: the norm function number.
Time: the CPUtime in s.
 (i)
Both of these algorithms can successfully solve all these ten nonlinear problems;
 (ii)
the NI and the NG of these two algorithm do not increase when the dimension becomes large;
 (iii)
the NI and the NG of Algorithm 1 are competitive to those of Algorithm YL and the Time of Algorithm YL is better than that of Algorithm 1. To directly show their the efficiency, the tool of [5] is used and three figures for NI, NG and Time are listed.
Experiment results
Nr  Dim  Algorithm 1  Algorithm YL  

Ni  NG  Time  NI  NG  Time  
1  400  9  18  10.93567  11  22  1.778411 
800  9  18  52.46314  11  22  7.176046  
1600  8  14  215.453  11  22  42.57267  
2  400  4  10  11.27887  6  7  1.185608 
800  4  10  45.94229  6  7  4.071626  
1600  4  10  251.38  6  7  22.58894  
3  400  4  10  2.808018  64  125  8.642455 
800  4  10  10.74847  78  129  52.26034  
1600  4  10  70.80885  68  99  262.5653  
4  400  2  2  0.8112052  6  17  1.092007 
800  2  2  2.839218  6  22  3.08882  
1600  2  2  14.08689  6  22  13.27569  
5  400  3  6  1.731611  6  7  0.936006 
800  3  6  5.616036  6  7  3.650423  
1600  3  6  30.32659  6  7  22.44854  
6  400  3  6  1.279208  5  6  0.7176046 
800  3  6  5.397635  5  16  2.88601  
1600  3  6  29.88979  5  16  16.39571  
7  400  5  14  3.790824  12  49  1.435209 
800  5  14  22.52654  12  49  4.69563  
1600  5  14  102.0403  17  83  19.23492  
8  400  1  2  1.294808  3  6  0.2808018 
800  1  2  5.694037  3  6  0.8580055  
1600  1  2  31.091  3  6  3.775224  
9  400  13  19  11.01367  12  15  1.60681 
800  9  15  40.95026  11  17  7.191646  
1600  10  19  299.3191  10  16  38.07984  
10  400  3  9  2.558416  40  50  12.44888 
800  3  9  11.62207  40  50  49.43672  
1600  3  9  73.07087  41  53  365.7911 
5 Conclusions
 (1)
a tensor trustregion model is established and discussed.
 (2)
the low workload update is used in this tensor trustregion model. In the future, we think this tensor trustregion model shall be more significant.
Declarations
Acknowledgements
The authors would like to thank the above the support funding. The authors also thank the referees and the editor for their valuable suggestions which greatly improve our paper.
Authors’ information
Songhua Wang and Shulun Liu are cofirst authors.
Funding
This work was supported by the National Natural Science Fund of China (Grant No. 11661009).
Authors’ contributions
Dr. Songhua Wang mainly analyzed the theory results and Shulun Liu has done the numerical experiments. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1995) MATHGoogle Scholar
 Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasiNewton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994) MathSciNetView ArticleGoogle Scholar
 Chow, T., Eskow, E., Schnabel, R.: Algorithm 783: a software package for unstrained optimization using tensor methods. ACM Trans. Math. Softw. 20, 518–530 (1994) View ArticleGoogle Scholar
 Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Method. SIAM, Philadelphia (2000) View ArticleGoogle Scholar
 Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002) MathSciNetView ArticleGoogle Scholar
 Fan, J.Y.: A modified Levenberg–Marquardt algorithm for singular system of nonlinear equations. J. Comput. Math. 21, 625–636 (2003) MathSciNetMATHGoogle Scholar
 GomezRuggiero, M., Martinez, J.M., Moretti, A.: Comparing algorithms for solving sparse nonlinear systems of equations. SIAM J. Sci. Stat. Comput. 23, 459–483 (1992) MathSciNetView ArticleGoogle Scholar
 Griewank, A.: The ’global’ convergence of Broydenlike methods with a suitable line search. J. Aust. Math. Soc. Ser. B 28, 75–92 (1986) MathSciNetView ArticleGoogle Scholar
 Levenberg, K.: A method for the solution of certain nonlinear problem in least squares. Q. Appl. Math. 2, 164–166 (1944) MathSciNetView ArticleGoogle Scholar
 Li, D., Fukushima, M.: A global and superlinear convergent Gauss–Newtonbased BFGS method for symmetric nonlinear equations. SIAM J. Numer. Anal. 37, 152–172 (1999) MathSciNetView ArticleGoogle Scholar
 Li, D., Qi, L., Zhou, S.: Descent directions of quasiNewton methods for symmetric nonlinear equations. SIAM J. Numer. Anal. 40(5), 1763–1774 (2002) MathSciNetView ArticleGoogle Scholar
 Li, X., Wang, S., Jin, Z., Pham, H.: A conjugate gradient algorithm under Yuan–Wei–Lu line search technique for largescale minimization optimization models. Math. Probl. Eng. 2018, Article ID 4729318 (2018) MathSciNetGoogle Scholar
 Li, Y., Yuan, G., Wei, Z.: A limitedmemory BFGS algorithm based on a trustregion quadratic model for largescale nonlinear equations. PLoS ONE 10(5), Article ID e0120993 (2015) View ArticleGoogle Scholar
 Marquardt, D.W.: An algorithm for leastsquares estimation of nonlinear inequalities. SIAM J. Appl. Math. 11, 431–441 (1963) View ArticleGoogle Scholar
 Moré, J.J., Garbow, B.S., Hillström, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981) MathSciNetView ArticleGoogle Scholar
 Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999) View ArticleGoogle Scholar
 Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970) MATHGoogle Scholar
 Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997) MathSciNetView ArticleGoogle Scholar
 Sheng, Z., Yuan, G.: An effective adaptive trust region algorithm for nonsmooth minimization. Comput. Optim. Appl. 71, 251–271 (2018) MathSciNetView ArticleGoogle Scholar
 Sheng, Z., Yuan, G., et al.: An adaptive trust region algorithm for largeresidual nonsmooth least squares problems. J. Ind. Manag. Optim. 14, 707–718 (2018) MathSciNetGoogle Scholar
 Sheng, Z., Yuan, G., et al.: A new adaptive trust region algorithm for optimization problems. Acta Math. Sci. 38B(2), 479–496 (2018) MathSciNetView ArticleGoogle Scholar
 Wang, Y.J., Xiu, N.H.: Theory and Algoithm for Nonlinear Programming. Shanxi Science and Technology Press, Xian (2004) Google Scholar
 Wei, Z., Yuan, G., Lian, Z.: An approximate Gauss–Newtonbased BFGS method for solving symmetric nonlinear equations. Guangxi Sciences 11(2), 91–99 (2004) Google Scholar
 Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt method. Computing 15, 239–249 (2001) MathSciNetMATHGoogle Scholar
 Yuan, G., Duan, X., Liu, W., Wang, X., Cui, Z., Sheng, Z.: Two new PRP conjugate gradient algorithms for minimization optimization models. PLoS ONE 10(10), Article ID e0140071 (2015). https://doi.org/10.1371/journal.pone.0140071 View ArticleGoogle Scholar
 Yuan, G., Lu, S., Wei, Z.: A new trustregion method with line search for solving symmetric nonlinear equations. Int. J. Comput. Math. 88, 2109–2123 (2011) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Lu, X.: A new backtracking inexact BFGS method for symmetric nonlinear equations. Comput. Math. Appl. 55, 116–129 (2008) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Lu, X.: An active set limited memory BFGS algorithm for bound constrained optimization. Appl. Math. Model. 35, 3561–3573 (2011) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Lu, X., Wei, Z.: BFGS trustregion method for symmetric nonlinear equations. J. Comput. Appl. Math. 230(1), 44–58 (2009) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Meng, Z., Li, Y.: A modified Hestenes and Stiefel conjugate gradient algorithm for largescale nonsmooth minimizations and nonlinear equations. J. Optim. Theory Appl. 168, 129–152 (2016) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Sheng, Z.: Nonsmooth Optimization Algorithms. Press of Science, Beijing (2017) Google Scholar
 Yuan, G., Sheng, Z., Wang, P., Hu, W., Li, C.: The global convergence of a modified BFGS method for nonconvex functions. J. Comput. Appl. Math. 327, 274–294 (2018) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Wei, Z.: Convergence analysis of a modified BFGS method on convex minimizations. Comput. Optim. Appl. 47, 237–255 (2010) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Wei, Z., Lu, S.: Limited memory BFGS method with backtracking for symmetric nonlinear equations. Math. Comput. Model. 54, 367–377 (2011) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Wei, Z., Lu, X.: A BFGS trustregion method for nonlinear equations. Computing 92, 317–333 (2011) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Wei, Z., Lu, X.: Global convergence of the BFGS method and the PRP method for general functions under a modified weak Wolfe–Powell line search. Appl. Math. Model. 47, 811–825 (2017) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Wei, Z., Wu, Y.: Modified limited memory BFGS method with nonmonotone line search for unconstrained optimization. J. Korean Math. Soc. 47, 767–788 (2010) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Wei, Z., Yang, Y.: The global convergence of the Polak–Ribiere–Polyak conjugate gradient algorithm under inexact line search for nonconvex functions. J. Comput. Appl. Math. (2018). https://doi.org/10.1016/j.cam.2018.10.057 View ArticleGoogle Scholar
 Yuan, G., Yao, S.: A BFGS algorithm for solving symmetric nonlinear equations. Optimization 62, 45–64 (2013) MathSciNetView ArticleGoogle Scholar
 Yuan, G., Zhang, M.: A threeterms Polak–Ribière–Polyak conjugate gradient algorithm for largescale nonlinear equations. J. Comput. Appl. Math. 286, 186–195 (2015) MathSciNetView ArticleGoogle Scholar
 Yuan, Y.: Trust region algorithm for nonlinear equations. Information 1, 7–21 (1998) MathSciNetMATHGoogle Scholar
 Zhang, J., Wang, Y.: A new trust region method for nonlinear equations. Math. Methods Oper. Res. 58, 283–298 (2003) MathSciNetView ArticleGoogle Scholar
 Zhu, D.: Nonmonotone backtracking inexact quasiNewton algorithms for solving smooth nonlinear equations. Appl. Math. Comput. 161, 875–895 (2005) MathSciNetMATHGoogle Scholar