Share This Article:

A Continuous Dynamical Systems Approach to Gauss-Newton Minimization

Abstract PP. 1-10
DOI: 10.4236/oalib.1101028    1,562 Downloads   1,835 Views  
Author(s)    Leave a comment

ABSTRACT

In this paper we show how the iterative Gauss-Newton method for minimizing a function can be reformulated as a solution to a continuous, autonomous dynamical system. We investigate the properties of the solutions to a one-parameter ODE initial value problem that involves the gradient and Hessian of the function. The equation incorporates an eigenvalue shift conditioner, which is a non-negative continuous function of the state. It enforces positive definiteness on a modified Hessian. Assuming the existence of a unique global minimum, the existence of a bounded connected sub-level set of the function and that the Hessian is non-zero in the interior of this set, our main results are: 1) existence of local solutions to the ODE initial value problem; 2) construction of a global solution by recursive extension of local solutions; 3) convergence of the global solution to the minimizing state for all initial values contained in the interior of the bounded level set; 4) eventual exact exponential decay of the gradient magnitude independent of the particular function and number of its variables. The results of a numerical experiment on the Rosenbrock Banana using a constant step-size 4th order Runge-Kutta method are presented and we point toward the direction of future research.

Conflicts of Interest

The authors declare no conflicts of interest.

Cite this paper

Danchick, R. (2014) A Continuous Dynamical Systems Approach to Gauss-Newton Minimization. Open Access Library Journal, 1, 1-10. doi: 10.4236/oalib.1101028.

References

[1] Liao, L.-Z., Qi, L.Q. and Tam, H.W. (2005) A Gradient-Based Continuous Method for Large-Scale Optimization Problems. Journal of Global Optimization, 31, 271-286.
http://dx.doi.org/10.1007/s10898-004-5700-1
[2] Danchick, R. (2006) Accurate Numerical Partials with Applications to Optimization. Applied Mathematics and Computation, 183, 551-559.
http://dx.doi.org/10.1016/j.amc.2006.05.083
[3] Ganesh, S.S. (2007) Lecture Notes on Ordinary Differential Equations. Annual Foundation School, IIT, Mumbai, 3-28 December.
[4] Boyd, S. (2008-2009) Lecture 12 Basic Lyapunov Theory, Electrical Engineering 363 Course Notes, Stanford University, Stanford, 10.
[5] Someijer, B.P. (1986) On the Economization of Explicit Runge-Kutta Methods. Applied Mathematics and Computation, 2, 57-69.
[6] Danchick, R. and Juncosa, M. (2006) Maximum Polynomial Degree Nordsieck-Gear (k, p) Methods: Existence, Stability, Consistency, Refinement, Convergence, and Computational Examples. Applied Mathematics and Computation, 182, 907-933.
http://dx.doi.org/10.1016/j.amc.2006.04.067

  
comments powered by Disqus

Copyright © 2019 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.