DaiKou type conjugate gradient methods with a line search only using gradient
 Yuanyuan Huang^{1}Email author and
 Changhe Liu^{1}
https://doi.org/10.1186/s136600171341z
© The Author(s) 2017
Received: 17 August 2016
Accepted: 22 March 2017
Published: 4 April 2017
Abstract
In this paper, the DaiKou type conjugate gradient methods are developed to solve the optimality condition of an unconstrained optimization, they only utilize gradient information and have broader application scope. Under suitable conditions, the developed methods are globally convergent. Numerical tests and comparisons with the PRP+ conjugate gradient method only using gradient show that the methods are efficient.
Keywords
conjugate gradient optimality condition line search sufficient descent condition global convergence1 Introduction
Recently, Dai and Kou [7] designed a family of conjugate gradient methods for the unconstrained nonlinear problems, the corresponding search direction is close to the direction of the scaled memoryless BFGS method. More importantly, they satisfied the sufficient descent condition (5). Numerical experiments illustrated that the DaiKou type conjugate gradient methods are more efficient than the HagerZhang type methods [8] presented by Hager and Zhang [8, 9]. For other descent conjugate gradient methods proposed by researchers, please see [7, 9–11] and the references therein.
The rest of this paper is organized as follows. In the next section, we simply review the DaiKou type conjugate gradient methods for unconstrained minimization and develop them to solve problem (1). In Section 3, we prove the global convergence of the improved methods under some suitable conditions. In Section 4, we select two classes of test problems to test the improved methods. One class is composed of test problems from the CUTEst test environment, and the other class is composed of some boundary value problems. The numerical performance is used to confirm their broader application and to compare with that of the PRP+ conjugate gradient method in [13]. Finally, some conclusions are given in Section 5.
2 Algorithm
The DaiKou type methods are very efficient in solving the unconstrained minimization, so we hope they can be used to solve problem (1) only requiring gradient information. Now we describe the improved methods in detail.
Algorithm 2.1
 Step 0.:

Choose \(x_{0}\in R^{n}\), constants \(\sigma\in(0,1)\), \(\delta\in (0,\sigma)\), \(\lambda\in[0,1]\), \(\eta\in[0,1)\), \(\varepsilon>0\). Set \(g_{0}:=g(x_{0})\) and \(k:=0\).
 Step 1.:

If \(\g_{k}\_{\infty}\leq\varepsilon\), then stop.
 Step 2.:

Generate the search direction \(d_{k}\) by (4) with \(\beta_{k}\) from (12), where \(\tau_{k1}\) is defined by (14).
 Step 3.:

Find \(\alpha_{k}\) such that condition (11) holds, then compute the new iterate \(x_{k+1}=x_{k}+\alpha_{k} d_{k}\). Set \(k:=k+1\) and go to Step 1.
In Step 3, the steplength \(\alpha_{k}\) is determined following the inexact line search strategies of Algorithm 2.6 in [17]. Detailed steps are described in the following line search algorithm.
Algorithm 2.2
 Step 0.:

Set \(u=0\) and \(v=+\infty\). Choose \(\alpha>0\). Set \(j:=0\).
 Step 1.:

If α does not satisfythen set j:=j+1, and go to Step 2. If α does not satisfy$$g(x_{k}+\alpha d_{k})^{T} d_{k}\leq \delta g_{k}^{T} d_{k}, $$then set \(j:=j+1\), and go to Step 3. Otherwise, set \(\alpha_{k}:=\alpha\), and return.$$\sigma g_{k}^{T} d_{k}\leq g(x_{k}+ \alpha d_{k})^{T} d_{k}, $$
 Step 2.:

Set \(v=\alpha\), \(\alpha=(u+v)/2\). Then go to Step 1.
 Step 3.:

Set \(u=\alpha\), \(\alpha=2u\). Then go to Step 1.
The choice of the initial steplength is important for a line search. For conjugate gradient methods, it is important to make an initial guess of the steplength by utilizing the current iterative information about the problem. So, in Algorithm 2.2, we choose the initial steplength \(\alpha=1/\g_{0}\\) if \(k=0\), and \(\alpha=\alpha _{k1}g_{k1}^{T} d_{k1}/y_{k1}^{T} d_{k1}\) if \(k\geq1\).
3 Convergence analysis
Assumption 1
Lemma 3.1
Proof
Lemma 3.2
Suppose that \(f:R^{n}\rightarrow R\) is bounded below along the ray \(\{ x_{k}+\alpha d_{k}  \alpha>0\}\), its gradient g is continuous, \(d_{k}\) is a search direction at \(x_{k}\), and \(g_{k}^{T} d_{k}<0\). Then if \(0<\delta <\sigma<1\), there exists \(\alpha_{k}>0\) satisfying the line search (11).
Proof
Lemma 3.3
Proof
Now, we state the Zoutendijk condition [18] for the line search (11).
Lemma 3.4
Proof
Theorem 3.1
Proof
Theorem 3.2
Proof
4 Numerical experiments
In this section, we did some numerical experiments to test the performance of the proposed method and compared it with the PRP+ conjugate gradient method in [13]. All codes were written in Matlab and run on a notebook computer with an Intel(R) Core(TM) i55200U 2.20 GHz CPU, 8.00 GB of RAM and Linux operation system Ubuntu 12.04. All test problems were drawn from the CUTEst test library [19, 20] and the literature [12]. For the test problems from the CUTEst test library, we particularly chose the unconstrained optimization problems whose dimensions were at least 50. Different from the work in the literature such as [5, 7], we solved them only using gradient information. In order to confirm the broader application scope of the proposed method, some boundary value problems were selected from [12]. See Chapter 1 in [21] for the background of the boundary value problems.
In practical implementations, the stopping criterion used was \(\g_{k}\_{\infty}\leq10^{3}\). For the proposed method in this paper, the values of σ and δ in the line search (11) were taken to be 0.9 and 0.0001, respectively, \(\lambda=0.5\), and \(\eta=0.5\). For the PRP+ conjugate gradient, all the initial values came from the reference [13].
Numerical results for test problems from the CUTEst library
Name (Dim)  Method  Iter/Ng/CPU 

ARGLINA (200)  Dai_Kou  14/28/1.673e − 02 
PRP+  13/25/2.309e − 02  
ARGLINB (200)  Dai_Kou  22 /43/2.577e − 02 
PRP+  47/93/6.121e − 02  
ARGLINC (200)  Dai_Kou  22/43/2.420e − 02 
PRP+  47/92/6.144e − 02  
BDQRTIC (500)  Dai_Kou  118/264/3.731e − 02 
PRP+  181/317/6.208e − 02  
BOX (10,000)  Dai_Kou  30/100/1.662e − 01 
PRP+  56/104/2.615e − 01  
BROWNAL (200)  Dai_Kou  22/42/1.004e − 02 
PRP+  //  
BROWNALE (200)  Dai_Kou  1/1/9.500e − 05 
PRP+  1/1/1.070e − 04  
BRYBND (5,000)  Dai_Kou  24/34/3.827e − 02 
PRP+  32/62/9.025e − 02  
CHAINWOO (4,000)  Dai_Kou  223/361/2.337e − 01 
PRP+  271/480/4.458e − 01  
CHNROSNB (50)  Dai_Kou  344/548/3.404e − 02 
PRP+  564/952/8.028e − 02  
CRAGGLVY (5,000)  Dai_Kou  142/273/2.638e − 01 
PRP+  //  
COSINE (1,000)  Dai_Kou  9/22/6.495e − 03 
PRP+  14/25/1.433e − 02  
CURLY10 (10,000)  Dai_Kou  // 
PRP+  20,040/39,984/6.169e + 01  
CURLY20 (10,000)  Dai_Kou  // 
PRP+  27,216/54,259/1.278e + 02  
DIXMAANA (3,000)  Dai_Kou  10/12/5.625e − 03 
PRP+  16/27/2.274e − 02  
DIXMAANB (3,000)  Dai_Kou  10/12/5.704e − 03 
PRP+  11/15/1.145e − 02  
DIXMAANC (3,000)  Dai_Kou  12/15/6.271e − 03 
PRP+  14/21/1.697e − 02  
DIXMAAND (3,000)  Dai_Kou  14/17/1.011e − 02 
PRP+  16/24/1.547e − 02  
DIXMAANE (3,000)  Dai_Kou  85/123/4.520e − 02 
PRP+  80/152/8.792e − 02  
DIXMAANF (3,000)  Dai_Kou  31/42/2.522e − 02 
PRP+  30/41/4.214e − 02  
DIXMAANG (3,000)  Dai_Kou  29/40/2.873e − 02 
PRP+  27/35/2.557e − 02  
DIXMAANH (3,000)  Dai_Kou  28/37/1.468e − 02 
PRP+  26/34/2.635e − 02  
DIXMAANI (3,000)  Dai_Kou  124/186/6.319e − 02 
PRP+  124/239/1.124e − 01  
DIXMAANJ (3,000)  Dai_Kou  36/52/2.502e − 02 
PRP+  31/43/3.019e − 02  
DIXMAANK (3,000)  Dai_Kou  34/48/2.063e − 02 
PRP+  28/37/2.864e − 02  
DIXMAANL (3,000)  Dai_Kou  29/40/1.661e − 02 
PRP+  30/40/3.369e − 02  
DIXMAANM (3,000)  Dai_Kou  104/154/6.135e − 02 
PRP+  157/305/1.407e − 01  
DIXMAANN (3,000)  Dai_Kou  63/93/3.813e − 02 
PRP+  98/164/8.303e − 02  
DIXMAANO (3,000)  Dai_Kou  59/86/2.737e − 02 
PRP+  80/130/7.730e − 02  
DIXMAANP (3,000)  Dai_Kou  56/77/3.176e − 02 
PRP+  72/111/6.704e − 02  
DIXON3DQ (10,000)  Dai_Kou  620/945/5.557e − 01 
PRP+  1,467/2,933/2.524e + 00  
DMN15103LS (99)  Dai_Kou  119/206/1.417e + 00 
PRP+  39/106/1.053e + 00  
DMN15333LS (99)  Dai_Kou  80/171/1.143e + 00 
PRP+  //  
DQDRTIC (5,000)  Dai_Kou  53/100/6.594e − 02 
PRP+  76/151/1.327e − 01  
DQRTIC (5,000)  Dai_Kou  18/31/1.109e − 02 
PRP+  25/25/2.123e − 02  
EDENSCH (1,000)  Dai_Kou  28/43/1.159e − 02 
PRP+  31/51/1.590e − 02  
EG2 (1,000)  Dai_Kou  19/37/9.933e − 03 
PRP+  32/58/2.803e − 02  
EIGENALS (2,550)  Dai_Kou  24,758/37,853/2.181e + 02 
PRP+  21,640/41,892/3.618e + 02  
ENGVAL1 (1,000)  Dai_Kou  25/35/6.147e − 03 
PRP+  20/28/1.253e − 02  
ERRINROS (50)  Dai_Kou  111/171 /1.860e − 02 
PRP+  25,995/48,312/3.756e + 00  
ERRINRSM (50)  Dai_Kou  419/805/4.634e − 02 
PRP+  //  
EXTROSNB (1,000)  Dai_Kou  652/1,063/1.300e − 01 
PRP+  906/1,611/2.639e − 01  
FLETBV3M (5,000)  Dai_Kou  115/263/4.331e − 01 
PRP+  33/61/1.482e − 01  
FLETCBV2 (5,000)  Dai_Kou  1/1/1.099e − 03 
PRP+  1/1/1.283e − 03  
FMINSRF2 (5,625)  Dai_Kou  251/386/2.966e − 01 
PRP+  338/567/6.821e − 01  
FREUROTH (5,000)  Dai_Kou  191/331 /2.437e − 01 
PRP+  75/133/1.523e − 01  
GENHUMPS (5,000)  Dai_Kou  9,378/20,870/3.155e + 01 
PRP+  10,235/17,320/3.504e + 01  
GENROSE (1,000)  Dai_Kou  3,054/4,706/7.083e − 01 
PRP+  4,947/8,388/1.792e + 00  
HYDC20LS (99)  Dai_Kou  2,541/3,952/4.016e − 01 
PRP+  //  
INDEF (5,000)  Dai_Kou  // 
PRP+  //  
INDEFM (1,000)  Dai_Kou  // 
PRP+  628/1,271/5.722e − 01  
JIMACK (3,549)  Dai_Kou  716/1,098/4.231e + 01 
PRP+  401/725/4.284e + 01  
LIARWHD (5,000)  Dai_Kou  50/150/8.031e − 02 
PRP+  124/223/1.945e − 01  
MANCINO (100)  Dai_Kou  8/17/5.880e − 02 
PRP+  31/59/2.788e − 01  
MODBEALE (10,000)  Dai_Kou  371/738/1.879e + 00 
PRP+  //  
MOREBV (5,000)  Dai_Kou  1/1/5.170e − 04 
PRP+  1/1/7.230e − 04  
MSQRTALS (1,024)  Dai_Kou  749/1,148/1.534e + 00 
PRP+  520/969/1.854e + 00  
MSQRTBLS (1,024)  Dai_Kou  783/1,196/1.639e + 00 
PRP+  681/1279/2.391e + 00  
NCB20 (5,010)  Dai_Kou  365/688/1.466e + 00 
PRP+  148/248/8.941e − 01  
NCB20B (5,000)  Dai_Kou  98/172/3.661e − 01 
PRP+  77/131/4.434e − 01  
NONCVXU2 (5,000)  Dai_Kou  1,159/1,751/1.945e + 00 
PRP+  4,582/8,610/1.396e + 01  
NONCVXUN (5,000)  Dai_Kou  1,247/1,887/2.110e + 00 
PRP+  9,929/18,942/3.063e + 01  
NONDIA (5,000)  Dai_Kou  13/23/1.189e − 02 
PRP+  54/103/8.099e − 02  
NONDQUAR (5,000)  Dai_Kou  66/129/5.082e − 02 
PRP+  139/202/1.238e − 01  
OSCIGRAD (10,000)  Dai_Kou  31/44/5.616e − 02 
PRP+  //  
OSCIPATH (500)  Dai_Kou  30/78/6.678e − 03 
PRP+  //  
PENALTY1 (1,000)  Dai_Kou  18/28/4.520e − 03 
PRP+  //  
PENALTY2 (200)  Dai_Kou  112/164 /2.145e − 02 
PRP+  173/304/5.560e − 02  
PENALTY3 (200)  Dai_Kou  // 
PRP+  //  
POWELLSG (5,000)  Dai_Kou  118/225/7.709e − 02 
PRP+  147/260/1.233e − 01  
POWER (10,000)  Dai_Kou  22/25/1.965e − 02 
PRP+  //  
QUARTC (5,000)  Dai_Kou  18/31/9.852e − 03 
PRP+  25/25/2.080e − 02  
SCHMVETT (5,000)  Dai_Kou  38/68/1.145e − 01 
PRP+  33/63/1.478e − 01  
SENSORS (100)  Dai_Kou  // 
PRP+  32/65/4.099e − 01  
SINQUAD (5,000)  Dai_Kou  117/270/2.988e − 01 
PRP+  182/342/5.408e − 01  
SPARSINE (5,000)  Dai_Kou  875/1348/1.708e + 00 
PRP+  //  
SPARSQUR (10,000)  Dai_Kou  21/22/4.845e − 02 
PRP+  16/16/6.262e − 02  
SPMSRTLS (4,999)  Dai_Kou  136/219/1.742e − 01 
PRP+  161/278/3.338e − 01  
SROSENBR (5,000)  Dai_Kou  26/63/2.904e − 02 
PRP+  33/57/4.532e − 02  
SSBRYBND (5,000)  Dai_Kou  6,337/9,751/9.184e + 00 
PRP+  //  
SSCOSINE (5,000)  Dai_Kou  // 
PRP+  //  
TESTQUAD (5,000)  Dai_Kou  5,068/7,734/1.948e + 00 
PRP+  1,624/3,247/9.661e − 01  
TOINTGOR (50)  Dai_Kou  131/195/1.998e − 02 
PRP+  105/180/2.060e − 02  
TOINTGSS (5,000)  Dai_Kou  18/37/2.997e − 02 
PRP+  14/27/2.830e − 02  
TOINTPSP (50)  Dai_Kou  142/268/2.158e − 02 
PRP+  115/194/2.190e − 02  
TOINTQOR (50)  Dai_Kou  43/64/7.463e − 03 
PRP+  41/81/9.627e − 03  
TQUARTIC (5,000)  Dai_Kou  35/103/4.848e − 02 
PRP+  68/120/7.646e − 02  
TRIDIA (5,000)  Dai_Kou  1,633/2,491/7.701e − 01 
PRP+  628/1,255/5.693e − 01  
VARDIM (200)  Dai_Kou  18/18/1.765e − 03 
PRP+  //  
VAREIGVL (50)  Dai_Kou  19/29/4.227e − 03 
PRP+  23/39/6.727e − 03  
WOODS (4,000)  Dai_Kou  36/67/3.083e − 02 
PRP+  22/28/2.143e − 02 
Numerical results for some boundary value problems
Name (Dim)  Method  Iter/Ng/CPU 

Function2 (10,000)  Dai_Kou  12/27/1.266e − 02 
PRP+  12/23/1.529e − 02  
Function6 (10,000)  Dai_Kou  1/1/5.010e − 04 
PRP+  1/1/4.399e − 04  
Function8 (10,000)  Dai_Kou  12/16/4.678e − 02 
PRP+  10/17/7.151e − 02  
Function12 (10,000)  Dai_Kou  10/21/1.206e − 02 
PRP+  10/19/1.227e − 02  
Function13 (10,000)  Dai_Kou  222/330/2.044e − 01 
PRP+  346/691/5.704e − 01  
Function14 (10,000)  Dai_Kou  12/17/4.554e − 02 
PRP+  9/11/4.912e − 02  
Function18 (10,000)  Dai_Kou  1/1/8.588e − 04 
PRP+  1/1/7.632e − 04  
Function19 (10,000)  Dai_Kou  9/14/1.084e − 02 
PRP+  8/12/1.551e − 02  
Function20 (10,000)  Dai_Kou  1/1/7.464e − 04 
PRP+  1/1/9.391e − 04  
Function21 (10,000)  Dai_Kou  75/81/5.441e − 02 
PRP+  //  
Function22 (10,000)  Dai_Kou  13/21/1.300e − 02 
PRP+  12/21/1.580e − 02  
Function24 (10,000)  Dai_Kou  5/7/7.387e + 00 
PRP+  6/10/1.609e + 01  
Function25 (10,000)  Dai_Kou  12/22/2.008e − 02 
PRP+  16/26/4.658e − 02  
Function26 (10,000)  Dai_Kou  258/387/1.890e − 01 
PRP+  345/689/4.391e − 01  
Function27 (10,000)  Dai_Kou  143/212/1.285e − 01 
PRP+  171/341/2.837e − 01  
Function29 (10,000)  Dai_Kou  2,211/3,355/6.638e + 00 
PRP+  8,150/16,299/4.633e + 01  
Function31 (10,000)  Dai_Kou  1/1/5.388e − 04 
PRP+  1/1/9.083e − 04 
5 Conclusions
Declarations
Acknowledgements
The authors are very grateful to the associate editor and reviewers for their valuable suggestions which have greatly improved the paper. This work was partially supported by the National Natural Science Foundation of China (No. 11471102) and the Key Basic Research Foundation of the Higher Education Institutions of Henan Province (No. 16A110012).
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
 Fletcher, R, Reeves, C: Function minimization by conjugate gradients. Comput. J. 7, 149154 (1964) MathSciNetView ArticleMATHGoogle Scholar
 Hestenes, MR, Stiefel, EL: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(5), 409436 (1952) MathSciNetView ArticleMATHGoogle Scholar
 Polak, E, Ribière, G: Note sur la méthodes de directions conjuguées. Rev. Fr. Inform. Rech. Opér., Sér Rouge 16, 3543 (1969) MATHGoogle Scholar
 Polyak, BT: The conjugate gradient methods in extreme problems. USSR Comput. Math. Math. Phys. 9, 94112 (1969) View ArticleMATHGoogle Scholar
 Dai, YH, Yuan, Y: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177182 (1999) MathSciNetView ArticleMATHGoogle Scholar
 Liu, Y, Storey, C: Efficient generalized conjugate gradient algorithms. Part 1: theory. J. Optim. Theory Appl. 69, 129137 (1991) MathSciNetView ArticleMATHGoogle Scholar
 Dai, YH, Kou, CX: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23, 296320 (2013) MathSciNetView ArticleMATHGoogle Scholar
 Hager, WW, Zhang, HC: Algorithm 851: CG__DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Softw. 32, 113137 (2006) View ArticleMATHGoogle Scholar
 Hager, WW, Zhang, HC: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170192 (2005) MathSciNetView ArticleMATHGoogle Scholar
 Zhang, L, Li, JL: A new globalization technique for nonlinear conjugate gradient methods for nonconvex minimization. Appl. Math. Comput. 217, 1029510304 (2011) MathSciNetMATHGoogle Scholar
 Nakamura, W, Narushima, Y, Yabe, H: Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization. J. Ind. Manag. Optim. 9, 595619 (2013) MathSciNetView ArticleMATHGoogle Scholar
 Spedicato, E, Huang, Z: Numerical experience with Newtonlike methods for nonlinear algebraic systems. Computing 58, 6989 (1997) MathSciNetView ArticleMATHGoogle Scholar
 Dong, YD: A practical PR+ conjugate gradient method only using gradient. Appl. Math. Comput. 219, 20412052 (2012) MathSciNetMATHGoogle Scholar
 Powell, MJD: Convergence properties of algorithms for nonlinear optimization. SIAM Rev. 28, 487500 (1986) MathSciNetView ArticleMATHGoogle Scholar
 Dong, YD: The BFGS method only using gradient. Preprint. http://www.optimizationonline.org/DB_FILE/2016/07/5560.pdf
 Dai, YH, Liao, LZ: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43, 87101 (2001) MathSciNetView ArticleMATHGoogle Scholar
 Lewis, AS, Overton, ML: Nonsmooth optimization via quasiNewton methods. Math. Program. 141(12), 135163 (2013) MathSciNetView ArticleMATHGoogle Scholar
 Zoutendijk, G: Nonlinear programming, computational methods. In: Abadie, J (ed.) Integer and Nonlinear Programming, pp. 3786. NorthHolland, Amsterdam (1970) Google Scholar
 Bongartz, I, Conn, AR, Gould, N, Toint, PL: CUTE: constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21, 123160 (1995) View ArticleMATHGoogle Scholar
 Gould, NIM, Orban, D, Toint, PL: CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373394 (2003) View ArticleMATHGoogle Scholar
 Ortega, JM, Rheinboldt, WC: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970) MATHGoogle Scholar
 Dolan, ED, Morè, JJ: Benchmarking optimization software with performance profiles. Math. Program. 91, 201213 (2002) MathSciNetView ArticleMATHGoogle Scholar
 Solodov, MV, Svaiter, BF: A globally convergent inexact Newton method for systems of monotone equations. In: Fukushima, M, Qi, L (eds.) Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, pp. 355369. Kluwer Academic, Dordrecht (1999) Google Scholar
 Yan, QR, Peng, XZ, Li, DH: A globally convergent derivativefree method for solving largescale nonlinear monotone equations. J. Comput. Appl. Math. 234, 649657 (2010) MathSciNetView ArticleMATHGoogle Scholar
 Li, QN, Li, DH: A class of derivativefree methods for largescale nonlinear monotone equations. IMA J. Numer. Anal. 31, 16251635 (2011) MathSciNetView ArticleMATHGoogle Scholar