A New Descent Nonlinear Conjugate Gradient Method for Unconstrained Optimization

In this paper, a new nonlinear conjugate gradient method is proposed for large-scale unconstrained optimization. The sufficient descent property holds without any line searches. We use some steplength technique which ensures the Zoutendijk condition to be held, this method is proved to be globally convergent. Finally, we improve it, and do further analysis.


Introduction
The nonlinear conjugate gradient method is designed to solve the following unconstrained optimization problem: min ( ), f x n x   where is a smooth nonlinear function, and the gradient of f at x is denoted by ( ) g x .The iterative formula of the conjugate gradient methods is given by where k is a steplength which is computed by carrying out some line search, is the search direction defined by where k  is a scalar, k g denotes ( ) k g x .There are some well-known formulas for k  , which are given as follows: , (Polak-Ribiere-Polyak [1]), (1.3)    stands for the Euclidean norm of vectors.
In addition, the sufficient descent condition is defined as follows: where , has often been used in the literature to analyze the global convergence of conjugate gradient methods with inexact line searches.0 c  Generally, the PRP method was much better than the FR method judging from the numerical calculation.When the objective function was convex, Polak and Ribie`re proved that the PRP method with the exact line search was globally convergent.But Powell showed that there existed nonconvex functions on which the PRP method did not converge globally.He suggested that k  should not be less than zero.Under the sufficient descent condition, Gilbert and Nocedal proved that the modified PRP method was globally convergent with the Wolfe-Powell line search.
Recently, G. Yu [3] proposed a modified FR (MFR) formula such as where 1 (0, ), .In fact, the term 2 in the denominator of (1.6) played an important role in enhancing descent.It essentially controled the relative weight placed on conjugacy versus descent.Along this way, G. Yu [3] proposed a new nonlinear conjugate gradient formula such as in which 1   .It possessed the sufficient descent property for any line search, and had an advancement that the directions would approach to the steepest descent directions while the steplength was small.They also proved the algorithm which possed the global convergence property with the weak Wolfe-Powell.Z. Wei [4] proposed a new nonlinear conjugate gradient formula such as In [4], a new conjugate gradient formula *   , the new algorithm calls MN algorithm, where in which 1   .We add some parameters for ( )  is any given positive number, calls VMN algorithm.
In the next section, we present the global convergence of MN algorithm and establish some good properties for which.The global convergence results of VMN algorithm are given in Section 3. Finally, we have a conclusion section.

The Global Convergence of MN Algorithm
Firstly, we can prove ( ) The following theorem shows that MN algorithm possesses the sufficient descent property for any line search.
Theorem 2.1.Consider any method (1.1) and (1.2), where ( )   , then we have Otherwise, from the definition of k ( ) and then we have .
 , we can deduce that (2.1) holds for all . 1 k  In order to establish the global convergence result for MN algorithm, we will impose the following assumptions.
Assumption A.
1) The level set 2) In some neighborhood of , N  f is differentiable and its gradient g is Lipschitz continuous, that is to say, there exists a constant such that 0 By using the Assumption A, we can deduce that there exists B and such that 0 The following important result is obtained by Zoutendijk [5].
Lemma 2.2.Suppose that Assumption A holds.Consider any iteration method of the form (1.1) and (1.2), and is obtained by the Wolfe line search (1.4).Then Then we will analyse the global convergence property of MN algorithm.
Gilbert and Nocedal [6] introduced the following Property A which pertains to the PRP method under the sufficient descent condition.Now we will show that this Property A pertains to the new method.
Property A. Consider a method of form (1.1) and (1.2).Suppose that 0 .
We say that the method has Property A, if for all , there exist constants The following lemma shows that the new method has the Property A.
By (1.9) and (2.5) we have By the Assumption A ( 2) and (2.2) hold, if The proof is finished.If (2.5) holds and the methods have Property A, then the small steplength should not be too many.The following lemma shows this property.The proofs of Lemmas 2.4 and 2.5 had been given in [7].By the above three lemmas, it is easy to obtain the following convergence result.

The Global Convergence of VMN Algorithm
In this section we will add some parameters of ( )

MN k
  so that it is generalization, then we have VMN algorithm  is any given positive number.
The proof is similar as the one of Theorem 2.1 in Section 2.

Numerical Experiments
In this section, we carry out some numerical experiments.The MN algorithm has been tested on some problems from [8].The results are summarized in Table 1.For the test problem, No. is the number of the test problem in [8], 0 x is the initial point, k x is the final point, is the number of times of iteration for the problem.
k Table 1 shows the performance of the MN algorithm relative to the iteration.It is easily to see that, for all the problems, the algorithm is very efficient.The results for each problem are accurate, and with less number of times of iteration.

Conclusions
In this paper, we have proposed a new nonlinear conjugate gradient method-MN algorithm.The sufficient descent property holds without any line searches, and the algorithm satisfys Property A. We also have proved, employing some steplength technique which ensures the Zoutendijk condition to be held, this method is globally convergent.Judging from the numerical experiments in Table 1, compared to most other algorithms, MN algorithm has higher precision and less number of times of iteration.Finally, we have proposed VMN algorithm, it also have the sufficient descent property and Property A, and it is global convergence under weak Wolfe-Powell line search.

and 1 
was an any given positive constant.They proved that for any line search, (1.6) satisfied the condition (1 the search directions for unconstrained optimization problems.It was discussed some general convergence results for the proposed formula with some line searches such as the exact line search, the Wolfe-Powell line search and the Grippo-Lucidi line search.The given formula , and had the similar form with above, in this paper, we propose a new modified scalar formula ( ) and 1

Lemma 2 . 4 .
Suppose that Assumption A and (1.5) hold.Let   k x and   k d be generated by (1.1) and (1.2) in which k satisfies the Wolfe-Powell line search, k t  has Property A. If (2.5) holds, then, for any 0

Lemma 2 . 5 .
Suppose that Assumption A and (1.5) hold.Let   k x be generated by (1.1) and (1.2), k satisfies the Wolfe-Powell line search, and

Theorem 2 . 6 .
Suppose that Assumption A holds.Let

Theorem 3 . 1 .
Suppose that Assumption A holds.Let Let   k x be generated by (1.1) and (1.2), k satisfies the Wolfe-Powell line search, and can get .And the algorithm possesses the sufficient descent property, in which