^{1}

^{1}

^{2}

In this paper, we provide and analyze a new scaled conjugate gradient method and its performance, based on the modified secant equation of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method and on a new modified nonmonotone line search technique. The method incorporates the modified BFGS secant equation in an effort to include the second order information of the objective function. The new secant equation has both gradient and function value information, and its update formula inherits the positive definiteness of Hessian approximation for general convex function. In order to improve the likelihood of finding a global optimal solution, we introduce a new modified nonmonotone line search technique. It is shown that, for nonsmooth convex problems, the proposed algorithm is globally convergent. Numerical results show that this new scaled conjugate gradient algorithm is promising and efficient for solving not only convex but also some large scale nonsmooth nonconvex problems in the sense of the Dolan-Moré performance profiles.

The conjugate gradient method (CG) and Quasi-Newton method are two major popular iterative methods for solving smooth unconstrained optimization problems, and Broyden-Fletcher-Goldfarb-Shanno (BFGS) method is one of the most efficient quasi-Newton methods for solving small and medium sized unconstrained optimization problems [

x k + 1 = x k + α k d k , (1)

where α k is a step size and d k is a search direction. For continuously differentiable function h : R n → R , the minimization problem:

min x ∈ R n h ( x ) (2)

has been well studied for several decades. Conjugate gradient method is among the preferable methods for solving problem (2) with search direction d k given by

d k = ( − ∇ h k + β k d k − 1 if k ≥ 1 , − ∇ h k if k = 0 , (3)

where ∇ h k is the gradient of an objective function h ( x ) at k iterate and β k is a scalar describing the attributes of the CG methods.

Some well-known formulas for the scalar β k are the Hestenes-Stiefel (HS) [

β k H S = ∇ h k T y k d k − 1 T y k , β k P R P = ∇ h k T y k ‖ ∇ h k − 1 ‖ 2 ,

β k F R = ‖ ∇ h k ‖ 2 ‖ ∇ h k − 1 ‖ 2 , β k D Y = ‖ ∇ h k ‖ 2 d k − 1 T y k ,

where y k = ∇ h ( x k ) − ∇ h ( x k − 1 ) and ‖ ⋅ ‖ denotes the Euclidean norm. Due to their simplicity and low memory requirement, CG methods are more effective and desirable for large scale unconstrained smooth problems [

( h ( x k + α k d k ) ≤ h ( x k ) + ς α k ∇ h k T d k , ∇ h ( x k + α k d k ) T d k ≥ ρ ∇ h k T d k , (4)

and the strong Wolfe line search:

( h ( x k + α k d k ) ≤ h ( x k ) + ς α k ∇ h k T d k , | ∇ h ( x k + α k d k ) T d k | ≤ ρ | ∇ h k T d k | , (5)

where 0 < ς < ρ < 1 . CG methods use relatively little memory for large scale problems and require no numerical linear algebra, so each step is quite fast. However, they do not have second order information of the objective function, and typically converge much more slowly than Newton or quasi-Newton methods.

The quasi-Newton method is an iterative method with second order information of the objective function, and BFGS is the effective quasi-Newton method with the search direction

d k = − B k ∇ h k , (6)

where B k is an approximation of the Hessian matrix of h at x k . The update formula for B k is defined by

B k + 1 = B k − B k s k s k T s k T B k s k + y k y k T y k T s k , (7)

where s k is defined as s k = x k + 1 − x k , and the Hessian approximation B k + 1 of (7) satisfies the standard secant equation

B k + 1 s k = y k , (8)

if y k T s k > 0 , which is known as the curvature condition. The BFGS method has very interesting properties and remains one of the most respectable quasi-Newton methods for unconstrained optimization [

d k = ( − θ k ∇ h k + β k d k − 1 if k ≥ 1 , − ∇ h 0 if k = 0 , (9)

where

θ k = 1 + β k d k T ∇ h k ‖ ∇ h k ‖ 2 , (10)

β k = ∇ h k T y k − 1 ( 1 − τ ) ‖ ∇ h k − 1 ‖ 2 + τ d k − 1 T y k − 1 , (11)

and τ ∈ [ 0,1 ] . It is not difficult to notice that the denominator of (11) is the convex combination of the denominator of the conjugate parameters in HS and PRP conjugate gradient methods. The choice of spectral parameter given (10) ensures the sufficient descent property of the search direction without dependence of line search. The convergence property of their method analyzed under a new modified nonmonotone line search with some mild conditions. However, this spectral CG method has only first order information, and excludes second order information. When the number of dimension is large, the CG methods are more effective compared to the BFGS methods in term of the CPU-time but in term of the number of iterations and the number of function evaluations, the BFGS methods are better. In order to incorporate the remarkable properties of the CG and BFGS methods and to overcome their drawbacks, many hybrid of CG and BFGS methods are introduced for unconstrained smooth optimization [

Motivated by the work of Ou and Zhou [

The paper is organized as follows. In the next section, we consider a nonsmooth convex problem and review their basic results. In Section 3, we propose a new scaled CG algorithm that incorporates the BFGS secant equation which has both function value and gradient information of the objective function via the smoothing regularization. Using the new modified nonmonotone line search technique, we prove the global convergence of our new algorithm for nonsmooth convex problems. Numerical results and related comparisons are reported in Section 4. Finally, Section 5 concludes our work.

In this section, we consider the unconstrained optimization problem

min x ∈ R n f ( x ) , (12)

where f : R n → R is a possibly nonsmooth convex function. This problem is equivalent to the following problem

min x ∈ R n F ( x ) , (13)

where F : R n → R is the Moreau-Yosida regularization of f [

F ( x ) = min z ∈ R n { f ( z ) + 1 2 λ ‖ z − x ‖ 2 } , (14)

where λ is a positive parameter. The function F is a finite-valued, continuously differentiable convex function even though the function f is nondifferentiable (see [

F ( x ) = f ( p ( x ) ) + 1 2 λ ‖ p ( x ) − x ‖ 2 . (15)

Moreover, the gradient of F is globally Lipschitz continuous, i.e.,

‖ g ( x ) − g ( y ) ‖ ≤ 1 λ ‖ x − y ‖ , ∀ x , y ∈ R n , (16)

where

g ( x ) = ∇ F ( x ) = x − p ( x ) λ . (17)

The point x ∈ R n is an optimal solution to (12) if and only if g ( x ) = 0 (see [

Several methods have been proposed to solve (13) by incorporating bundle methods and quasi-Newton methods ideas [

f ( p α ( x , ε ) ) + 1 2 λ ‖ p α ( x , ε ) − x ‖ 2 ≤ F ( x ) + ε . (18)

Therefore, we can approximate F ( x ) and g ( x ) by

F α ( x , ε ) = f ( p α ( x , ε ) ) + 1 2 λ ‖ p α ( x , ε ) − x ‖ 2 , (19)

and

g α ( x , ε ) = x − p α ( x , ε ) λ , (20)

respectively. Implementable algorithms to define such a p α ( x , ε ) for nonsmooth convex model can be seen in [

Proposition 1. Let p α ( x , ε ) be a vector that satisfies (18), and let F α ( x , ε ) and g α ( x , ε ) be defined by (19) and (20), respectively. Then we obtain

F ( x ) ≤ F α ( x , ε ) ≤ F ( x ) + ε , (21)

‖ p α ( x , ε ) − p ( x ) ‖ ≤ 2 λ ε , (22)

and

‖ g α ( x , ε ) − g ( x ) ‖ ≤ 2 ε / λ . (23)

Proposition 1 shows that the approximations of F α ( x , ε ) and g α ( x , ε ) can be made arbitrarily close to the exact values of F ( x ) and g ( x ) respectively.

In this section, we introduce the new scaled CG search direction that incorporates the modified BFGS secant equation, and then describe the new algorithm for solving nonsmooth problems. We make use of a modified nonmonotone line search technique introduced by [

d k + 1 = ( − g α ( x k + 1 , ε k + 1 ) + β k + 1 d k if k ≥ 1 , − g α ( x k + 1 , ε k + 1 ) if k = 0 , (24)

where ε is an appropriately chosen positive number. Ou and Zhou [

d k + 1 = ( − Q ˜ k + 1 g α ( x k + 1 , ε k + 1 ) if k ≥ 1 , − g α ( x k + 1 , ε k + 1 ) if k = 0 , (25)

where Q ˜ k + 1 ∈ R n × n is defined

Q ˜ k + 1 = θ ˜ k + 1 I − θ ˜ k + 1 w k s k T + s k w k T w k T s k + [ 1 + θ ˜ k + 1 w k T w k w k T s k ] s k s k T w k T s k , (26)

with

θ ˜ k + 1 = s k T s k w k T s k ,

where

w k = y k ∗ + t k s k . (27)

The vector y k ∗ and t k in (27) are defined as

y k ∗ = g α ( x k + 1 , ε k + 1 ) − g α ( x k , ε k ) (28)

and

t k = t + max { s k T y k ‖ s k ‖ 2 , 0 } ( t > 0 ) . (29)

It is easy to observe that the (27) has only gradient value information. In order to have both gradient and function value information, we replace (27) and (29) by

w k ∗ = y k ∗ + max { t k ∗ , 0 } s k , (30)

and

t k ∗ = 6 [ F ( x k ) − F ( x k + α k d k ) ] + 3 ( g α ( x k + α k d k , ε k + 1 ) + g α ( x k , ε k ) ) T s k ‖ s k ‖ 2 , (31)

respectively. Thus, the BFGS method with the secant equation

B k + 1 s k = w k ∗ , (32)

and the update formula

B k + 1 = B k − B k s k s k T s k T B k s k + w k ∗ w k ∗ T w k ∗ T s k , (33)

has both gradient and function value information, and the matrix B k + 1 inherits the positive definiteness of B k for generally convex functions. Using the secant Equation (32), we propose the new search direction is defined by

d k + 1 = ( − θ ¨ k + 1 g α ( x k + 1 , ε k + 1 ) + β k + 1 d k − ϑ k + 1 w k ∗ , if k ≥ 1, − g α ( x k + 1 , ε k + 1 ) , if k = 0, (34)

where

θ ¨ k + 1 = 2 − d k T g α ( x k + 1 , ε k + 1 ) ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 ( g α ( x k + 1 , ε k + 1 ) T w k ∗ ‖ d k ‖ ‖ w k ∗ ‖ ) , (35)

β k + 1 = g α ( x k + 1 , ε k + 1 ) T w k ∗ ‖ d k ‖ ‖ w k ∗ ‖ + | d k T y k ∗ | , (36)

and

ϑ k + 1 = d k T g α ( x k + 1 , ε k + 1 ) ‖ d k ‖ ‖ w k ∗ ‖ . (37)

Now, based on the above search direction, we describe our new scaled CG algorithm with a modified nonmonotone line search for solving problem (13) as follows.

Algorithm 1

Step 0. Given ϵ > 0 , β ∈ ( 0 , 1 ) , ε ∈ ( 0 , 1 ) , σ ∈ ( 0 , 1 ) , and a point x 0 ∈ R n . Set d 0 = − g α ( x 0 , ε 0 ) and k : = 0 .

Step 1. If ‖ g α ( x k , ε k ) ‖ < ϵ , then stop, else go to the next step.

Step 2. Compute the search direction d k by using (34)-(37).

Step 3. Set trial step size α k = 1 .

Step 4. Set x k + 1 = x k + α k d k and choose a scalar ε k + 1 such that 0 < ε k + 1 < ε k .

Step 5. Let μ ∈ ( 0,1 ] , M ≥ 1 is a positive integer, define m ( k ) = min { k + 1 , M } , and choose

μ k i ≥ μ , i = 0 , 1 , 2 , ⋯ , m ( k ) − 1 , ∑ i = 0 m ( k ) − 1 μ k i = 1.

Let α k ≥ 0 be bounded above and satisfy:

F α ( x k + α k d k , ε k + 1 ) ≤ max [ F α ( x k , ε k ) , ∑ i = 0 m ( k ) − 1 μ k i F α ( x k − i , ε k − i ) ] σ α k g α ( x k , ε k ) T d k . (38)

If (38) does not holds, define α k = β α k and go to step 5.

Step 6. Set K := k + 1 and go to step 1.

It can be observed that the line search technique in step 5 of Algorithm 1 is a nonmonotone line search technique with some modifications.

Convergence AnalysisIn this subsection, we establish the global convergence of our method for nonsmooth convex problem (12). To prove the global convergence of Algorithm 1, the following Lemmas are needed.

Lemma 1. Assume that the search direction d k is generated by Algorithm 1, then for all k ≥ 0 , we have

g α ( x k + 1 , ε k + 1 ) T d k + 1 ≤ − ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 , (39)

and

‖ d k + 1 ‖ ≤ 5 ‖ g α ( x k + 1 , ε k + 1 ) ‖ . (40)

Proof. If k = 0 , then

g α ( x 0 , ε 0 ) T d 0 = − g α ( x 0 , ε 0 ) T g α ( x 0 , ε 0 ) = − ‖ g α ( x 0 , ε 0 ) ‖ 2 ,

and

‖ d 0 ‖ = ‖ − g α ( x 0 , ε 0 ) ‖ ≤ ‖ g α ( x 0 , ε 0 ) ‖ ≤ 5 ‖ g α ( x 0 , ε 0 ) ‖ .

Let k ≥ 1 , then from (34) we have

g α ( x k + 1 , ε k + 1 ) T d k + 1 = − θ ¨ k + 1 ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 + β k + 1 g α ( x k + 1 , ε k + 1 ) T d k − ϑ k + 1 g α ( x k + 1 , ε k + 1 ) T w k ∗ ≤ − θ ¨ k + 1 ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 + g α ( x k + 1 , ε k + 1 ) T w k ∗ ‖ d k ‖ ‖ w k ∗ ‖ + | d k T y k ∗ | g α ( x k + 1 , ε k + 1 ) T d k − d k T g α ( x k + 1 , ε k + 1 ) ‖ d k ‖ ‖ w k ∗ ‖ g α ( x k + 1 , ε k + 1 ) T w k ∗ = − 2 ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 + d k T g α ( x k + 1 , ε k + 1 ) ‖ d k ‖ ‖ w k ∗ ‖ g α ( x k + 1 , ε k + 1 ) T w k ∗

+ g α ( x k + 1 , ε k + 1 ) T w k ∗ ‖ d k ‖ ‖ w k ∗ ‖ + | d k T y k ∗ | g α ( x k + 1 , ε k + 1 ) T d k − d k T g α ( x k + 1 , ε k + 1 ) ‖ d k ‖ ‖ w k ∗ ‖ g α ( x k + 1 , ε k + 1 ) T w k ∗ = − 2 ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 + g α ( x k + 1 , ε k + 1 ) T w k ∗ ‖ d k ‖ ‖ w k ∗ ‖ + | d k T y k ∗ | g α ( x k + 1 , ε k + 1 ) T d k ≤ − 2 ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 + ( ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 ‖ d k ‖ ‖ w k ∗ ‖ ) ‖ d k ‖ ‖ w k ∗ ‖ = − ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 .

Once more, (34) yields that

‖ d k + 1 ‖ = ‖ − θ ¨ k + 1 g α ( x k + 1 , ε k + 1 ) + β k + 1 d k − ϑ k + 1 w k ∗ ‖ = ‖ − θ ¨ k + 1 g α ( x k + 1 , ε k + 1 ) + g α ( x k + 1 , ε k + 1 ) T w k ∗ ‖ d k ‖ ‖ w k ∗ ‖ + | d k T y k ∗ | d k − d k T g α ( x k + 1 , ε k + 1 ) ‖ d k ‖ ‖ w k ∗ ‖ w k ∗ ‖ ≤ ‖ θ ¨ k + 1 g α ( x k + 1 , ε k + 1 ) ‖ + 2 ‖ g α ( x k + 1 , ε k + 1 ) ‖ ≤ 4 ‖ g α ( x k + 1 , ε k + 1 ) ‖ + ( ‖ d k ‖ ‖ w k ∗ ‖ ‖ g α ( x k + 1 , ε k + 1 ) ‖ 2 ) ‖ g α ( x k + 1 , ε k + 1 ) ‖ 3 ‖ d k ‖ ‖ w k ∗ ‖ ≤ 5 ‖ g α ( x k + 1 , ε k + 1 ) ‖ .

Thus, the proof is completed.

Lemma 1 shows that the search direction d k developed in (34)-(37) leads to the most sufficiently descent direction and it belongs to a trust region.

Lemma 2. Let the step size α k satisfy (38), then there exist β > 0 satisfy a

α k ≥ min { 1, ( 1 − σ ) β L | g α ( x k , ε k ) T d k | ‖ d k ‖ 2 } . (41)

Proof. If α k = 1 satisfies the formula (38), then the proof is completed. Otherwise, there exist β such that

F α ( x k + α k β d k , ε k + 1 ) > max { F α ( x k , ε k ) , ∑ i = 0 m ( k ) − 1 μ k i F α ( x k − i , ε k − i ) } + σ α k β g α ( x k , ε k ) T d k > F α ( x k , ε k ) + σ α k β g α ( x k , ε k ) T d k .

Thus,

F α ( x k + α k β d k , ε k + 1 ) − F α ( x k , ε k ) > σ α k β g α ( x k , ε k ) T d k . (42)

Using mean value theorem, we have

F α ( x k + α d k , ε k + 1 ) − F α ( x k , ε k ) = ∫ 0 α ( g α ( x k + t d k , ε k + 1 ) − g α ( x k , ε k ) ) T d k d t + α g α ( x k , ε k ) T d k ≤ 1 2 L α 2 ‖ d k ‖ 2 + α g α ( x k , ε k ) T d k .

Combining the above inequality with (42), we have

α k ≥ min { 1, ( 1 − σ ) β L | g α ( x k , ε k ) T d k | ‖ d k ‖ 2 } .

Thus, the proof is completed.

Lemma 3. Assume that the sequence { x k } is generated by Algorithm 1, then we have

F α ( x k , ε k ) ≤ F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k − 2 α i g α ( x i , ε i ) T d i + σ α k − 1 g α ( x k − 1 , ε k − 1 ) T d k − 1 ≤ F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k − 1 α r g α ( x i , ε i ) T d i .

Proof. We prove this lemma by induction. For k = 1 , by (38) and μ ≤ 1 , we have

F α ( x 1 , ε 1 ) ≤ F α ( x 0 , ε 0 ) + σ α 0 g α ( x 0 , ε 0 ) T d 0 ≤ F α ( x 0 , ε 0 ) + μ σ α 0 g α ( x 0 , ε 0 ) d 0

Assume the equation holds for 1 , 2 , ⋯ , k , and we need to show for k + 1 . To show the condition, we have considered two cases.

Case 1:

max [ F α ( x k , ε k ) , ∑ i = 0 m ( k ) − 1 μ k i F α ( x k − i , ε k − i ) ] = F α ( x k , ε k ) .

Then, from (38), we have

F α ( x k + 1 , ε k + 1 ) = F α ( x k + α k d k , ε k + 1 ) ≤ F α ( x k , ε k ) + σ α k g α ( x k , ε k ) T d k ≤ F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k − 1 α i g α ( x i , ε i ) T d i + σ α k g α ( x k , ε k ) T d k ≤ F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k α i g α ( x i , ε i ) T d i .

Case 2:

max [ F α ( x k , ε k ) , ∑ i = 0 m ( k ) − 1 μ k i F α ( x k − i , ε k − i ) ] = ∑ i = 0 m ( k ) − 1 μ k i F α ( x k − i , ε k − i ) ,

let n = min [ k , m − 1 ] . Then, again from (38),

F α ( x k + 1 , ε k + 1 ) = F α ( x k + α k d k , ε k + 1 ) ≤ ∑ j = 0 n μ k j F α ( x k − j , ε k − j ) + σ α k g α ( x k , ε k ) T d k ≤ ∑ j = 0 n μ k j [ F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k − j − 2 α i g α ( x i , ε i ) T d i + σ α k − j − 1 g α ( x k − j − 1 , ε k − j − 1 ) T d k − j − 1 ] + σ α k g α ( x k , ε k ) T d k .

Thus, by imposing

( 1,2, ⋯ , n ) × ( 1,2, ⋯ , k − n − 2 ) ⊂ { ( j , i ) : 0 ≤ j ≤ n ,0 ≤ i ≤ k − n − 2 } ,

and

∑ j = 0 n μ k j = 1 , μ k j ≥ μ ,

we have

F α ( x k + 1 , ε k + 1 ) ≤ F α ( x 0 , ε 0 ) + μ ∑ i = 0 k − n − 2 ( ∑ j = 0 n μ k j ) α i g α ( x i , ε i ) T d i + σ ∑ j = 0 n μ k j α k − j − 1 g α ( x k − j − 1 , ε k − j − 1 ) T d k − j − 1 + σ α k g α ( x k , ε k ) T d k

≤ F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k − n − 2 α i g α ( x i , ε i ) T d i + μ σ ∑ i = k − j − 1 k − 1 α i g α ( x i , ε i ) T d i + σ α k g α ( x k , ε k ) T d k = F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k − 1 α i g α ( x i , ε i ) T d i + σ α k g α ( x k , ε k ) T d k ≤ F α ( x 0 , ε 0 ) + μ σ ∑ i = 0 k α i g α ( x i , ε i ) T d i .

Theorem 1. Assume that the sequences { x k } and { d k } are generated by Algorithm 1. Let F is bounded below on the level set L 0 = { x ∈ R n | F ( x ) ≤ F ( x 0 ) } and

lim k → ∞ ε k = 0.

Then

lim k → ∞ g α ( x k , ε k ) T d k = 0. (43)

Proof. Suppose that (43) is not true. Then there exist constants γ > 0 and k 0 such that

g α ( x k , ε k ) T d k ≤ − γ , ∀ k > k 0 . (44)

From Lemma 3, we have

F α ( x 0 , ε 0 ) − F α ( x k , ε k ) ≥ − μ σ ∑ i = 0 k − 1 α r g α ( x i , ε i ) T d i . (45)

By (40), (41) and (44), we have

F α ( x 0 , ε 0 ) − F α ( x k , ε k ) ≥ − μ σ ∑ i = 0 k − 1 α i g α ( x i , ε i ) T d i ≥ μ σ γ ∑ i = 0 k − 1 α i ≥ μ σ γ ∑ i = 0 k − 1 min { 1 , ( 1 − σ ) β L | g α ( x k , ε k ) T d k | ‖ d k ‖ 2 } ≥ μ σ γ ∑ i = 0 k − 1 min { 1 , ( 1 − σ ) β 25 L } .

Letting k → ∞ , we have

μ σ γ ∑ k = 0 ∞ min { 1 , ( 1 − σ ) β 25 L } ≤ ∑ k = 0 ∞ F α ( x 0 , ε 0 ) − F α ( x k , ε k ) ,

and this contradicts our assumption on F. Hence the theorem is proved.

Theorem 2. Let the conditions in Lemma 1 and Theorem 1 hold, then Algorithm 1 converges for nonsmooth problem (12).

Proof. From Lemma 1 and Theorem 1, we have

0 ≥ lim k → ∞ ( − ‖ g α ( x k , ε k ) ‖ 2 ) ≥ lim k → ∞ g α ( x k , ε k ) T d k = 0.

Then,

lim k → ∞ ‖ g α ( x k , ε k ) ‖ = 0. (46)

Thus, (23) and convergence of sequence { ε k } yield

0 ≤ lim k → ∞ ‖ g α ( x k , ε k ) − g ( x k ) ‖ ≤ lim k → ∞ 2 ε / λ = 0.

Hence,

lim k → ∞ ‖ g ( x k ) ‖ = 0. (47)

Let x ∗ be an accumulation point of { x k } . Then there exists a subsequence { x k } K satisfying

lim k ∈ K , k → ∞ x k = x ∗ . (48)

Thus, (17), (43) and (47) yield x ∗ = p ( x ∗ ) . Therefore x ∗ is an optimal solution of nonsmooth problem (12).

In this section, we present some numerical experiments to examine the efficiency of Algorithm 1 for some large scale nonsmooth academic test problems which are introduced in [

Problem 1

f ( x ) = max 1 ≤ i ≤ n x i 2

x i ( 1 ) = i for i = 1 , ⋯ , n / 2 and

x i ( 1 ) = − i for i = n / 2 + 1 , ⋯ , n

f ( x ∗ ) = 0.

Problem 2

f ( x ) = max 1 ≤ i ≤ n | ∑ i = 1 n x j i + j − 1 |

x i ( 1 ) = i for i = 1, ⋯ , n .

f ( x ∗ ) = 0.

Problem 3

f ( x ) = ∑ i = 1 n − 1 max { − x i − x i + 1 , − x i − x i + 1 + ( x i 2 + x i + 1 2 − 1 ) }

x i ( 1 ) = − 0.5 for i = 1, ⋯ , n ;

f ( x ∗ ) = − 2 ( n − 1 ) .

Problem 4

f ( x ) = ∑ i = 1 n − 1 max { x i 4 + x i + 1 2 , ( 2 − x i ) 2 + ( 2 − x i + 1 ) 2 , 2 e − x i + x i + 1 }

x i ( 1 ) = 2 for i = 1, ⋯ , n ;

f ( x ∗ ) = 2 ( n − 1 ) .

Problem 5

f ( x ) = max { ∑ i = 1 n − 1 ( x i 4 + x i + 1 2 ) , ∑ i = 1 n − 1 ( ( 2 − x i ) 2 + ( 2 − x i + 1 ) 2 ) , ∑ i = 1 n − 1 ( 2 e − x i + x i + 1 ) }

x i ( 1 ) = 2 for i = 1, ⋯ , n ;

f ( x ∗ ) = 2 ( n − 1 ) .

Problem 6

f ( x ) = max 1 ≤ i ≤ n { g ( − ∑ i = 1 n x i ) , g ( x i ) } ,

where g ( y ) = ln ( | y | + 1 ) ;

x i ( 1 ) = 1 for i = 1, ⋯ , n ;

f ( x ∗ ) = 0.

Problem 7

f ( x ) = ∑ i = 1 n − 1 ( | x i | x i + 1 2 + 1 + | x i + 1 | x i 2 + 1 )

x i ( 1 ) = − 1 when mod ( i , 2 ) = 1 , ( i = 1 , ⋯ , n ) and

x i ( 1 ) = 1 when mod ( i , 2 ) = 0 , ( i = 1 , ⋯ , n ) ;

f ( x ∗ ) = 0.

Problem 8

f ( x ) = ∑ i = 1 n − 1 ( − x i + 2 ( x i 2 + x i + 1 2 − 1 ) + 1.75 | x i 2 + x i + 1 2 − 1 | )

x i ( 1 ) = − 1 for i = 1, ⋯ , n ;

f ( x ∗ ) = v a r i e s .

Problem 9

f ( x ) = max { ∑ i = 1 n − 1 ( x i 2 + ( x i + 1 − 1 ) 2 + x i + 1 − 1 ) , ∑ i = 1 n − 1 ( − x i 2 − ( x i + 1 − 1 ) 2 + x i + 1 + 1 ) }

x i ( 1 ) = − 1.5 when mod ( i , 2 ) = 1 , ( i = 1 , ⋯ , n ) and

x i ( 1 ) = 2.0 when mod ( i , 2 ) = 0 , ( i = 1 , ⋯ , n ) ;

f ( x ∗ ) = 0.

Problem 10

f ( x ) = ∑ i = 1 n − 1 max { x i 2 + ( x i + 1 − 1 ) 2 + x i + 1 − 1 , − x i 2 − ( x i + 1 − 1 ) 2 + x i + 1 + 1 }

x i ( 1 ) = − 1.5 when mod ( i , 2 ) = 1 , ( i = 1 , ⋯ , n ) and

x i ( 1 ) = 2.0 when mod ( i , 2 ) = 0 , ( i = 1 , ⋯ , n ) ;

f ( x ∗ ) = 0.

The problems 1 - 5 are convex functions, and the others are nonconvex functions. We test the above problems with the dimension of n = 1000 , n = 3000 , n = 5000 , n = 6000 , n = 10000 , n = 12000 , n = 20000 , n = 50000 , n = 60000 and n = 100000 . For convenience sake, we denote Algorithm 1 by scaled conjugate gradient method based on modified secant equation of BFGS method (SCG-MBFGS), and in order to demonstrate validity of our algorithm, we also list the results of other three algorithms MPRP in [

Dim: the dimensions of problem.

NI: the total number of iterations.

NF: the number of function evaluations.

TIME: the CPU time in seconds.

f ( x ) : the value of f ( x ) at the final iteration.

From the numerical results in

No. | Dim | Algorithm 3.1 | MHS | MPRP | MSBFGS−CG |
---|---|---|---|---|---|

NI/NF/f(x)/TIME | NI/NF/f(x)/TIME | NI/NF/f(x)/TIME | NI/NF/f(x)/TIME | ||

1 | 1000 | 213/4351/6.9117E−8/0.3034E+0 | 225/4710/6.9354E−8/0.4358E+0 | 225/4710/6.7102E−6/0.6947E+0 | 186/1601/2.6568E−9/0.5977E+0 |

3000 | 219/4509/1.9754E−8/0.2356E+0 | 238/5273/6.6915E−8/1.1157E+0 | 228/5196/6.5327E−6/1.6902E+0 | 219/2310/2.2089E−9/1.3871E+0 | |

5000 | 224/4597/5.7735E−7/1.6248E+0 | 246/5284/8.1495E−7/1.2650E+0 | 250/5253/9.0184E−7/1.9379E+0 | 242/2725/1.2183E−9/1.6962E+0 | |

6000 | 227/4608/6.81679E−8/1.1756E+0 | 253/5298/6.7261E−8/1.2323E+0 | 252/5308/5.8563E−6/2.0417E+0 | 248/2849/1.8972E−8/2.0233E+0 | |

10,000 | 230/4719/3.0882E−6/1.9051E+0 | 264/5387/7.3580E−6/1.8361E+0 | 261/5466/3.4089E−8/2.2684E+0 | 253/2996/3.4045E−9/2.1527E+0 | |

12,000 | 234/4841/8.5011E−8/2.1084E+0 | 264/5529/6.5042E−8/2.4321E+0 | 266/5573/9.0925E−7/2.3537E+0 | 261/3328/4.9085E−9/2.2886E+0 | |

20,000 | 243/5150/7.7681E−8/2.43061E+0 | 275/5830/6.9187E−8/2.6893E+0 | 270/5769/5.68982E−8/2.8049E+0 | 275/4573/6.1823E−10/2.6981E+0 | |

50,000 | 268/5539/1.0736E−8/2.9909E+0 | 289/6002/7.6945E−7/2.8937E+0 | 286/5991/8.0015E−6/3.0539E+0 | 283/5597/7.0687E−8/2.9298E+0 | |

60,000 | 277/5913/5.9244E−8/3.1618E+0 | 294/6125/6.4520E−8/2.9931E+0 | 295/6110/9.6412E−7/3.2291E+0 | 288/6018/1.7851E−8/3.1037E+0 | |

100,000 | 309/6884/8.6471E−8/4.1954E+0 | 324/7018/6.7620E−8/3.0068E+0 | 311/6983/5.1947E−7/4.7095E+0 | 311/6836/6.6109E−8/3.6814E+0 | |

2 | 1000 | 58/1507/8.0315E−8/5.3728E+1 | 96/1605/8.0248E−9/5.4181E+1 | 91/1482/6.1664E−9/2.9639E+1 | 94/1301/4.0582E−9/4.1905E+1 |

3000 | 66/1682/6.7354E−9/5.3848E+2 | 110/1917/6.9902E−9/5.8991E+2 | 105/11760/8.8715E−6/1.7785E+2 | 103/1420/3.0085E−9/2.5608E+2 | |

5000 | 77/1920/5.9738E−7/6.8072E+2 | 115/1963/7.1280E−5/8.8693E+2 | 111/1938/5.0135E−7/7.8101E+2 | 119/1499/2.6731E−9/9.6649E+2 | |

6000 | 84/2007/8.3699E−9/2.3108E+3 | 118/2103/8.45247E−9/2.5384E+3 | 112/1966/6.3507E−7/1.5338E+3 | 129/1573/1.9629E−9/1.9867E+3 | |

10,000 | 93/2235/5.6492E−6/2.9875E+3 | 126/2302/6.0153E−7/5.5084E+3 | 120/2127/4.8937E−8/3.3185E+3 | 141/1901/4.8003E−9/4.7503E+3 | |

12,000 | 96/2276/9.3619E−9/1.0518E+4 | 127/2310/9.5284E−9/1.1372E+4 | 123/2162/7.0430E−5/1.0882E+4 | 147/2243/6.1285E−8/1.2026E+4 | |

20,000 | 108/2419/8.9409E−9/1.6564E+4 | 165/2676/7.9950E−9/1.7582E+4 | 130/2296/2.1027E−8/1.0947E+4 | 155/2489/3.0318E−8/1.3732E+4 | |

50,000 | 119/2693/9.0057E−6/1.9714E+4 | 185/2709/5.8276E−8/1.8972E+4 | 141/2604/4.8826E−5/1.4608E+4 | 168/2707/5.6590E−7/1.5809E+4 | |

60,000 | 121/2701/9.9354E−9/2.0018E+4 | 215/2786/6.9338E−9/2.0358E+4 | 148/2651/3.0798E−5/1.5101E+4 | 173/2842/3.7206E−7/1.8104E+4 | |

100,000 | 127/2820/8.9262E−9/2.1883E+4 | 240/3008/6.9393E−9/2.2960E+4 | 195/2907/8.7203E−8/3.1048E+4 | 189/2803/6.6907E−8/2.1061E+4 | |

3 | 1000 | 16/79/6.2899E−9/5.0301E−2 | 37/114/7.2687E−9/4.8344E−2 | 37/114/7.2687E−9/0.3916E−1 | 37/110/2.3278E−9/0.2861E−1 |

3000 | 23/82/5.6856E−7/3.1106E−2 | 37/123/5.3594E−9/4.4758E−2 | 37/116/8.5786E−9/0.56813E−1 | 37/113/4.0082E−9/0.4193E−1 | |

5000 | 28/89/8.3625E−8/4.4800E−2 | 39/123/6.0082E−6/6.1701E−2 | 39/120/9.0931E−9/0.6191E−1 | 39/116/5.9987E−9/0.5518E−1 | |

6000 | 31/90/6.8937E−9/5.4075E−2 | 40/123/5.4561E−9/7.6563E−2 | 39/121/8.4838E−9/0.8434E−1 | 39/117/6.1096E−9/0.6991E−1 | |

10,000 | 39/110/5.9901E−7/1.107E−1 | 40/125/4.9830E−8/1.1071E−1 | 40/123/9.0941E−9/0.5089E+0 | 40/121/1.6943E−9/0.4805E+0 | |

12,000 | 40/113/1.5721E−9/1.125E−1 | 40/125/1.6369E−9/1.3819E−1 | 40/125/8.6969E−9/0.7394E+0 | 40/123/1.8803E−9/0.6097E+0 | |

20,000 | 44/135/6.0317E−9/2.011E−1 | 50/157/6.5211E−9/1.5450E−1 | 48/151/7.1062E−9/0.6928E+0 | 41/149/3.9088E−9/0.6681E+0 | |

50,000 | 47/158/7.0052E−9/8.062E−1 | 50/160/5.9735E−6/6.1943E−1 | 55/153/6.1343E−9/0.7906E+0 | 49/161/6.1874E−9/0.6714E+0 | |

60,000 | 49/161/6.0976E−9/8.3069E−1 | 50/165/5.1158E−9/7.4703E−1 | 59/158/6.1999E−9/0.8005E+0 | 53/161/6.607E−9/0.6968E+0 | |

100,000 | 55/180/6.1437E−9/1.169E+0 | 60/174/6.0371E−9/1.0152E+0 | 60/166/5.1093E−9/1.5108E+0 | 62/173/7.0773E−9/1.3005E+0 | |

4 | 1000 | 5/74/2.8725E+3/0.0416E+0 | 7/89/2.5699E+3/0.3053E−1 | 7/89/1.9980E+3/0.1489E−1 | 7/89/1.8835E+3/0.1069E−1 |

3000 | 5/74/1.9435E+4/0.0522E+0 | 7/89/2.3654E+4/0.4348E−1 | 7/89/1.7105E+3/0.1863E−1 | 7/89/1.7098E+3/0.1306E−1 | |

5000 | 5/74/1.5184E+5/0.0637E+0 | 7/89/1.6941E+4/0.4463E−1 | 7/89/1.7329E+4/0.2814E+0 | 7/89/1.8026E+3/0.2804E+0 | |

6000 | 5/85/1.5352E+4/0.0696E+0 | 7/89/1.5433E+4/0.48906E−1 | 7/89/1.1998E+4/0.3819E+0 | 7/89/1.4620E+4/0.2997E+0 | |

10,000 | 6/89/1.2674E+4/0.1009E+0 | 7/89/2.0593E+4/0.8813E−1 | 7/89/1.8801E+5/0.4097E+0 | 7/89/1.3806E+4/0.3991E+0 |

12,000 | 6/89/2.99787E+4/0.1047E+0 | 7/89/3.0868E+4/0.9556E−1 | 7/89/2.3998E+4/0.5523E+0 | 7/89/2.2855E+4/0.4895E+0 | |
---|---|---|---|---|---|

20,000 | 6/89/2.9304E+4/0.4095E+0 | 7/89/8.3594E+4/0.3845E+0 | 7/89/1.6858E+5/0.9945E+0 | 7/89/1.5008E+4/0.8028E+0 | |

50,000 | 7/89/1.9998E+5/0.4569E+0 | 7/89/5.0973E+4/0.4611E+0 | 7/89/2.0017E+4/1.2931E+0 | 7/89/1.8801E+4/1.1561E+0 | |

60,000 | 7/89/2.0953E+5/0.9996E+0 | 7/89/1.5435E+5/0.6831E+0 | 7/89/1.1999E+5/1.3883E+0 | 7/89/1.4096E+5/1.3468E+0 | |

100,000 | 7/89/1.9468E+5/3.5798E+0 | 7/89/2.04359E+5/3.4358E+0 | 7/89/2.0071E+5/6.1947E+0 | 7/89/2.1188E+6/6.1609E+0 | |

5 | 1000 | 5/22/1.9899E+3/0.6409E−2 | 7/86/2.5699E+3/0.7187E−3 | 7/85/1.9980E+3/0.6281E−1 | 7/86/1.1063E+4/0.5437E−1 |

3000 | 7/29/2.3548E+3/0.1138E−1 | 17/95/1.9353E+4/0.2854E−2 | 16/95/2.7126E+3/0.7897E−1 | 16/94/2.0019E+4/0.7669E−1 | |

5000 | 14/42/3.0083E+3/0.1307E−1 | 21/114/2.0891E+4/0.6341E−2 | 21/114/1.8462E+4/0.8633E−1 | 21/114/1.1192E+4/0.6675E−1 | |

6000 | 15/58/1.6253E+4/0.15625E−1 | 21/114/1.5433E+4/0.6912E−2 | 21/114/1.1999E+4/0.9487E−1 | 21/114/1.0930E+4/0.9036E−1 | |

10,000 | 15/87/2.5984E+3/0.1171E+0 | 21/114/4.8327E+5/0.1088E+0 | 21/114/1.5827E+4/0.1984E+0 | 21/114/2.3072E+4/0.1772E+0 | |

12,000 | 16/92/3.0879E+4/0.1299E+0 | 21/114/3.0868E+4/0.1108E+0 | 21/114/2.3998E+4/0.2789E+0 | 21/114/2.1984E+4/0.2683E+0 | |

20,000 | 19/96/1.9348E+5/0.1515E+0 | 21/114/2.9306E+4/0.1935E+0 | 21/114/1.7102E+5/0.3195E+0 | 21/114/1.5120E+5/0.2956E+0 | |

50,000 | 22/101/2.1184E+4/0.4091E+0 | 21/114/1.8256E+5/0.3638E+0 | 21/114/1.5293E+5/0.4093E+0 | 21/114/2.0968E+5/0.3885E+0 | |

60,000 | 22/115/1.9356E+5/0.6657E+0 | 21/114/1.5434E+5/0.52975E+0 | 21/114/1.1999E+5/0.5383E+0 | 21/114/2.3001E+5/0.4896E+0 | |

100,000 | 25/117/2.0009E+5/1.3158E+0 | 21/114/1.6972E+5/1.4358E+0 | 21/114/1.7582E+5/4.9145E+0 | 21/114/2.2293E+5/3.6984E+0 | |

6 | 1000 | 59/1047/6.1866E−9/2.8017E−1 | 78/1047/6.3785E−9/2.0109E−1 | 77/1026/6.8037E−9/1.0741E−1 | 71/891/3.6735E−9/0.1009E+0 |

3000 | 63/1216/6.2814E−9/2.9565E−1 | 85/1168/5.6373E−9/2.4364E−1 | 81/1153/7.0961E−9/5.9417E−1 | 77/921/6.8083E−9/0.5067E+0 | |

5000 | 66/1290/8.5604E−9/1.1307E+0 | 91/1308/7.9629E−8/1.0254E+0 | 90/1281/7.8404E−9/1.3245E+0 | 82/937/7.9864E−9/1.2838E+0 | |

6000 | 68/1374/6.9799E−9/1.1864E+0 | 92/1323/8.2692E−9/1.0613E+0 | 91/1316/5.5483E−9/1.5115E+0 | 86/998/7.8405E−9/1.3644E+0 | |

10,000 | 88/1457/8.0915E−8/2.1735E+0 | 94/1430/6.6018E−7/2.1506E+0 | 96/1401/9.9366E−9/2.9348E+0 | 88/1208/5.7163E−9/2.8861E+0 | |

12,000 | 95/1492/6.8749E−9/2.2673E+0 | 99/1452/5.6143E−9/2.2001E+0 | 97/1440/8.5456E−9/3.1554E+0 | 93/1417/9.9355E−9/3.1047E+0 | |

20,000 | 106/1504/4.5481E−9/3.5061E+0 | 107/1671/6.9926E−9/3.3804E+0 | 104/1597/6.7138E−9/5.1947E+0 | 105/1531/6.8017E−9/5.0093E+0 | |

50,000 | 110/1691/5.2964E−7/1.2993E+1 | 110/1688/8.0153E−6/1.2054E+1 | 110/1665/6.1343E−9/1.1832E+1 | 112/1682/7.7084E−9/1.1570E+1 | |

60,000 | 112/1702/6.7549E−9/1.3608E+1 | 112/1707/6.4697E−9/1.2916E+1 | 112/1698/7.5022E−9/1.3527E+1 | 112/1704/5.9938E−9/1.2643E+1 | |

100,000 | 117/1808/6.9354E−8/8.968E+1 | 119/1793/7.0642E−9/1.0054E+2 | 116/1731/6.7139E−9/9.4368E+1 | 120/1750/8.8802E−9/9.1146E+1 | |

7 | 1000 | 30/81/6.7682E−9/1.4821E−1 | 38/117/7.2687E−9/1.2581E−1 | 38/117/7.2687E−9/0.7074E+0 | 35/114/1.5021E−9/0.6621E+0 |

3000 | 36/85/5.8186E−9/1.0918E−1 | 39/121/6.5165E−9/1.7375E−1 | 39/121/9.2894E−9/0.8198E+0 | 37/116/3.8037E−9/0.6997E+0 | |

5000 | 38/94/7.1437E−8/2.2432E−1 | 41/123/6.2047E−7/5.3942E−1 | 40/123/9.0932E−9/1.8426E+0 | 39/120/5.2352E−9/1.6801E+0 | |

6000 | 39/102/5.9864E−9/2.7029E−1 | 41/126/5.4561E−9/7.1584E−1 | 40/123/9.8411E−8/4.6316E+0 | 40/123/7.0942E−9/4.3308E+0 | |

10,000 | 41/117/9.9318E−5/1.9885E+0 | 46/129/6.1533E−8/1.0827E+0 | 41/125/1.8188E−8/5.9017E+0 | 41/126/2.1136E−9/5.8892E+0 | |

12,000 | 44/122/8.2994E−9/2.0108E+0 | 48/131/8.1848E−9/1.3739E+0 | 46/130/9.1246E−9/8.4288E+0 | 46/129/1.8277E−9/7.8591E+0 | |

20,000 | 46/142/6.9354E−8/2.1563E+0 | 54/162/6.9354E−8/5.5384E+0 | 49/142/7.8906E−9/9.9472E+0 | 47/138/5.0472E−9/9.0988E+0 | |

50,000 | 51/163/6.8141E−8/9.9182E+0 | 67/185/7.0255E−7/8.2977E+0 | 55/153/9.0948E−8/1.5013E+1 | 58/160/6.5509E−9/1.6395E+0 | |

60,000 | 51/169/6.9354E−8/1.0056E+1 | 72/191/6.9354E−8/9.7955E+0 | 57/160/1.7923E−8/2.1545E+1 | 60/164/1.4301E−9/1.9896E+1 | |

100,000 | 64/1905/6.9354E−8/1.4891E+1 | 87/2012/6.9354E−8/1.4358E+1 | 66/1936/1.8862E−8/7.4791E+1 | 71/1942/2.0446E−9/7.0104E+1 | |

8 | 1000 | 31/77/−2.9735E+2/1.5502E−2 | 36/114/−2.4974E+2/3.2875E−2 | 37/114/−2.4975E+4/3.9808E−2 | 38/116/−5.3979E+3/3.5074E−2 |

3000 | 34/84/−2.1068E+2/2.9607E−2 | 39/118/−1.8852E+2/3.7518E−2 | 38/115/−2.0965E+4/7.17093E−2 | 39/119/−2.4796E+4/4.6310E−2 | |

5000 | 37/88/−1.7664E+3/3.3509E−2 | 39/122/−1.5326E+3/6.3006E−2 | 39/120/−1.24995E+5/1.9137E−1 | 41/123/−3.2153E+4/1.7892E−1 |

6000 | 37/94/−1.4998E+3/3.8201E−2 | 39/123/−1.4998E+3/7.9009E−2 | 39/120/−1.2488E+5/2.0213E−1 | 41/126/−1.9248E+4/1.8068E−1 | |
---|---|---|---|---|---|

10,000 | 39/101/−1.5683E+3/1.0021E−1 | 40/126/−2.7053E+3/1.1906E−1 | 40/123/−2.4997E+5/3.0749E−1 | 43/129/−2.0127E+4/2.9976E−1 | |

12,000 | 40/107/−2.9998E+3/1.0131E−1 | 40/126/−2.9998E+3/1.5488E−1 | 40/123/−1.4887E+5/4.7906E−1 | 43/131/−2.4998E+4/3.7175E−1 | |

20,000 | 40/113/−2.9231E+3/4.1561E−1 | 41/138/−2.9409E+3/3.4008E−1 | 41/126/−2.5827E+6/9.0468E−1 | 46/137/−2.5037E+5/6.5936E−1 | |

50,000 | 40/120/−2.0583E+3/7.0931E−1 | 42/154/−1.8099E+3/5.9141E−1 | 43/157/−1.24997E+6/9.3709E−1 | 47/165/−1.7036E+5/7.9384E−1 | |

60,000 | 41/137/−1.9357E+3/8.5164E−1 | 42/160/−1.4999E+3/6.8471E−1 | 57/161/−1.24995E+6/9.9741E−1 | 54/174/−1.4881E+5/8.8056E−1 | |

100,000 | 41/152/−1.5094E+3/1.0171E+0 | 46/197/−2.9354E+3/9.8405E−1 | 71/196/−1.499E+5/9.4011E−1 | 66/205/−1.6068E+5/9.0985E−1 | |

9 | 1000 | 36/47/9.0397E−9/4.5592E−2 | 39/174/9.04688E−9/3.265E−2 | 37/114/5.4897E−9/4.6481E−2 | 34/101/7.0463E−9/3.9807E−2 |

3000 | 39/55/6.5101E−9/4.9901E−2 | 39/176/6.8510E−9/3.375E−2 | 38/117/5.4993E−9/6.1903E−2 | 36/106/5.5439E−9/4.364E−2 | |

5000 | 39/61/8.2853E−9/8.8256E−2 | 41/181/7.0387E−9/7.9083E−2 | 39/120/6.8294E−9/9.9665E−2 | 36/113/3.8145E−9/5.1750E−2 | |

6000 | 40/67/5.9491E−9/9.6753E−2 | 42/183/6.7514E−9/9.4875E−2 | 39/121/6.9655E−9/1.6048E−1 | 39/114/5.6824E−9/1.2194E−1 | |

10,000 | 40/88/6.5687E−8/1.7263E−1 | 43/185/8.6588E−8/1.6537E−1 | 40/123/6.8253E−9/1.6047E−1 | 39/120/4.3654E−9/1.4481E−1 | |

12,000 | 42/96/6.8993E−9/1.806E−1 | 43/186/6.7466E−9/1.5463E−1 | 42/139/6.7574E−9/1.8889E−1 | 42/127/5.8553E−9/1.6617E−1 | |

20,000 | 46/112/6.9506E−9/2.9899E−1 | 53/195/6.9307E−9/2.03582−1 | 47/145/6.7908E−9/4.7286E−1 | 45/139/6.1085E−9/3.6828E−1 | |

50,000 | 53/137/9.0249E−8/5.5128E−1 | 58/204/6.1284E−9/6.1639E−1 | 56/157/8.5275E−9/8.0077E−1 | 56/151/7.0083E−9/7.6994E−1 | |

60,000 | 55/146/7.9419E−9/8.9897E−1 | 59/220/8.4274E−9/8.5912E−1 | 57/168/9.6814E−9/8.3438E−1 | 56/166/8.8087E−9/8.5238E−1 | |

100,000 | 66/203/8.5929E−9/1.0168E+0 | 74/242/7.9908E−9/1.0008E+0 | 69/214/8.7106E−9/1.047E+0 | 73/211/7.8997E−9/1.0183E+0 | |

10 | 1000 | 31/84/6.7887E−9/5.3011E−2 | 39/120/6.8185E−9/3.2375E−2 | 39/120/6.8184E−9/4.9791E−2 | 37/112/6.0424E−9/3.3661E−2 |

3000 | 32/91/6.8185E−9/4.0807E−2 | 41/124/7.1982E−9/3.7025E−2 | 40/123/6.9058E−9/8.1673E−2 | 38/117/6.0885E−9/6.0898E−2 | |

5000 | 34/98/8.4897E−9/6.7185E−2 | 42/127/6.4897E−9/5.0688E−2 | 41/126/8.5258E−9/3.2518E−1 | 39/121/1.8205E−9/2.8856E−1 | |

6000 | 34/102/5.1906E−9/8.8093E−2 | 42/129/5.1156E−9/6.3125E−2 | 41/126/8.6835E−9/4.5537E−1 | 40/123/1.5825E−9/3.8677E−1 | |

10,000 | 37/118/8.6826E−9/1.4077E−1 | 43/130/5.0362E−9/1.0055E−1 | 42/129/8.5262E−9/5.0619E−1 | 42/125/2.6473E−9/4.6409E−1 | |

12,000 | 40/120/6.0162E−9/1.7186E−1 | 43/132/5.1157E−9/1.3838E−1 | 42/130/6.2721E−9/6.1254E−1 | 42/126/3.5262E−9/5.2537E−1 | |

20,000 | 46/139/5.9398E−9/4.0125E−1 | 51/164/6.8235E−9/2.7099E−1 | 50/149/6.7894E−9/9.4011E−1 | 53/139/5.8502E−9/8.8816E−1 | |

50,000 | 74/188/5.2798E−9/9.0155E−1 | 84/216/6.8873E−9/7.9928E−1 | 87/222/5.3291E−9/1.0004E+0 | 78/197/6.0074E−9/8.9038E−1 | |

60,000 | 76/197/7.0307E−9/9.9906E−1 | 87/220/6.3948E−9/8.8975E−1 | 87/226/5.3563E−9/1.1664E+0 | 88/204/5.5493E−9/1.1107E+0 | |

100,000 | 84/225/6.3793E−9/1.0626E+0 | 97/244/6.4077E−9/1.0037E+0 | 99/255/5.0979E−9/1.4198E+0 | 94/222/4.9906E−9/1.2684E+0 |

SCG-MBFGS is superior or competitive to the other three methods in solving the given problems in terms of number of iteration, number of function evaluations and CPU time. Furthermore, to directly illustrate the performances of our method, we employed the tool provided by Dolan and Moré [

From

In this paper, we propose a new scaled conjugate gradient method which incorporates a modified secant equation of BFGS method. This modified secant equation contains both function value and gradient information of the objective function, and its Hessian approximation update generates positive definite matrix. Under a modified nonmonotone line search and some mild conditions, the strong global convergence of the proposed method is analyzed for nonsmooth convex problems. The search direction of our new method generates sufficiently descent condition and belongs to a trust region. Compared with existing nonsmooth CG methods, the search direction of our approach is more descent direction. Numerical results and related comparisons show that the proposed method is effective for solving large scale nonsmooth optimization problems.

The authors would like to thank the reviewers and editor for their valuable comments which greatly improve our paper. This work is supported by the National Natural Science Foundation of China [Grant No. 11771003].

The authors declare no conflicts of interest regarding the publication of this paper.

Woldu, T.G., Zhang, H.B. and Fissuh, Y.H. (2020) A Scaled Conjugate Gradient Method Based on New BFGS Secant Equation with Modified Nonmonotone Line Search. American Journal of Computational Mathematics, 10, 1-22. https://doi.org/10.4236/ajcm.2020.101001