A New Nonmonotone Adaptive Trust Region Method

The trust region method plays an important role in solving optimization problems. In this paper, we propose a new nonmonotone adaptive trust region method for solving unconstrained optimization problems. Actually, we combine a popular nonmonotone technique with an adaptive trust region algorithm. The new ratio to adjusting the next trust region radius is different from the ratio in the traditional trust region methods. Under some appropriate conditions, we show that the new algorithm has good global convergence and superlinear convergence.


Introduction
In this paper, we consider the following unconstrained optimization problem: is a real-valued twice continuously differentiable function.
At each iteration, the strategy of choosing a trust region radius k ∆ is very crucial. In the standard trust region method, the following ratio to make a comparison between the objective function and the model is defined: In the case, if the ratio k r is close to 1, it is concluded that there is a good agreement between the objective function and the model over this step, so it is safe to increase the trust region radius for the next iteration. Otherwise, if k r is close to 0 or even negative, we must shrink the trust region radius. The strategies of determining and updating trust region radius affect the number of computational cost and convergence of the algorithm. There are many researchers who pay much attention to determining and updating the trust region radius [22]- [27]. In 1997, Sartenaer [22] proposed a new approach to determine a radius by monitoring an agreement between the model and the objective function along the direction k g − computed at the starting point. But the parameters of this procedure may be dependent on the problem that should be solved.
In 2005, Gould et al. [28] examine the sensitivity of trust-region algorithms on the parameters related to the step acceptance and update of the trust region, although, they did not discuss an initial trust-region radius. Motivated by a problem in neural network, in 2002, Zhang et al. [26] proposed a strategy to determine the trust region radius. Specifically, they solved the subproblem (1.2) with  [27] proposed an adaptive radius given by where ( ) 0,1 c ∈ , p is a non-negative integer and ˆk More recently, Shi and Guo [31] proposed an adaptive trust region: the vector k q satisfies the angle consdition: Theoretical analysis shows that the proposed trust region method has global convergence for first-order critical points, and preliminary numerical results show that the proposed method is effective for solving medium-scale unconstrained optimization problems. Kamandi et al. [32] give an improved version of the trust-region radius (1.4). They proposed a modification of k q : . It is straightforward that k q satisfies the condition (1.5). To avoid a very small trust region radius, the formula is defined: where 1 λ > , and k q is determined by (1.5). Then, the trust region radius is updated by and p is a nonnegative integer.
Due to the high efficiency of nonmonotone techniques, many researchers use the nonmonotone technique in the trust region algorithm framework. In 1986, Grippo et al. [33] put forward the nonmonotone line search technology for the first time. The stepsize k α satisfies condition  [34], Sun [35], Fu and Sun [36] presented various nonmonotone trust region methods. In 2004, Zhang and Hager [37] found that nonmonotone techniques (1.10) have some drawbacks.
For example, the numerical performances are seriously dependent on the choice of parameter M ; A good function value generated at any iteration may not be useful; For any given bound M on the memory, even an iterative method is generating R-linearly convergence for a strongly convex function, the iterates may not satisfy the condition (1.3) for k sufficiently large [38]. In order to cope with these disadvantages, Zhang and Hager [37] propose another nonmonotonic technique where the stepsize k α satisfies the following condition:  [39] introduced it into trust region method and developed a nonmonotone algorithm.
The numerical results indicate that the algorithm is robust and encouraging. In 2019, Xue et al. [40] propose a new improved nonmonotone adaptive trust region method for solving unconstrained optimization problems. From the perspective of theoretical analysis, it is shown that algorithm possesses global convergence and superlinear convergence under classical assumptions. Among the existing nonmonotone strategies, Gu and Mo [41] propose a simpler nonmonotone technique, and its computational complexity is greatly reduced. Therefore, based on the method in [40] and [41], we propose a new improved nonmonotone adaptive trust region method for solving unconstrained optimization problems. Under appropriate conditions, we analyze the global convergence and superlinear convergence of the algorithm.

The Structure of the New Algorithm
In this section, we talk about our algorithm for solving unconstrained optimization problems in detail. As we see, The nonmonotone technique proposed by Journal of Applied Mathematics and Physics Zhang and Hager [37] implies that each k C is a convex combination of the previous 1 k C − and k f , including the complex k η and k Q . In practice, it becomes an encumbrance to update k η and k Q at each iteration. Therefore, Gu and Mo [41] proposed another nonmonotone technique where the nonmonotone term is revised by: Then, the actual reduction of the objective function value is given by: the predicted reduction of the objective function value is given by: In order to determine whether the trial step is feasible and how to update the new trust region radius, we compute the modified ratio that is given by: We describe the new trust region region algorithm with adjustable radius as below: Algorithm 2.1. ANNATR (A new nonmonotone adaptive trust region method) Step 0. Step 2. Compute k q according to expression (1.6), k s by (1.7) and set 0 p = .
Step 3. Compute k ∆ by (1.8), solve subproblem (1.2) to find the trial step k d and compute k r by (2.4).
Set : 1 k k = + and go to Step 1.
In Algorithm 2.1, if k r v ≥ , it is called a successful iteration. The loop started from Step 3 to Step 4 is called the inner cycle.
The flowchart of our algorithm is provided here: Y. Zhang et al.

Convergence Analysis
In this paper, we consider the following assumptions that will be used to analyze the convergence properties and the superlinear convergence rate of the below new algorithm: (H1) The level set for all k N ∈ .
Proof. See [32] for reference.  From (3.4), (3.5) and (3.6), we have (3.9) Proof. The proof this lemma is quite as the same as [40], for the completeness of this work, we prove it again here. By contradiction, assume that the inner loop By the assumption that the inner cycle cycles infinity and (1.8), we obtain that implies that the right-hand side of the above equation (3.13) tends to zero. This means that for sufficiently large i, we get ( ) ( ) ( ) ( )

Conclusion
In this paper, we introduce the algorithm of a new nonmonotone adaptive trust region method for solving unconstrained optimization problems based on (1.8) and (2.1). The nonmonotone strategy is introduced into a new adaptive trust region. Maratos effects are avoided and the amount of calculation is reduced. Furthermore, it is obvious that the current objective function value k f is fully employed. With the help of nonmonotone technique and adaptive trust region radius, our algorithm can reduce the number of ineffective iterations so that we enhance the effectiveness of the algorithm. Under some standard and suitable assumptions, the global convergence and superlinear convergence of the new algorithm are analyzed theoretically. However, our algorithm still has some continuation and expansion, we can consider the following aspect: although this paper gives the theoretical proof of the proposed method as detailed as possible, it does not fully demonstrate the superiority of the new algorithm through numerical experiments, which will be the focus of further work.

Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.