OALibJ> Vol.1 No.8, November 2014

Author(s)    Leave a comment

Roy Danchick

10336 Wilshire Boulevard, Unit 101, Los Angeles, USA.

In this paper we show how the iterative Gauss-Newton method for minimizing a function can be reformulated as a solution to a continuous, autonomous dynamical system. We investigate the properties of the solutions to a one-parameter ODE initial value problem that involves the gradient and Hessian of the function. The equation incorporates an eigenvalue shift conditioner, which is a non-negative continuous function of the state. It enforces positive definiteness on a modified Hessian. Assuming the existence of a unique global minimum, the existence of a bounded connected sub-level set of the function and that the Hessian is non-zero in the interior of this set, our main results are: 1) existence of local solutions to the ODE initial value problem; 2) construction of a global solution by recursive extension of local solutions; 3) convergence of the global solution to the minimizing state for all initial values contained in the interior of the bounded level set; 4) eventual exact exponential decay of the gradient magnitude independent of the particular function and number of its variables. The results of a numerical experiment on the Rosenbrock Banana using a constant step-size 4th order Runge-Kutta method are presented and we point toward the direction of future research.

Gauss-Newton Method, Unconstrained Minimization, Dynamical Systems, Ordinary Differential Equations

Danchick, R. (2014) A Continuous Dynamical Systems Approach to Gauss-Newton Minimization. Open Access Library Journal, 1, 1-10. doi: 10.4236/oalib.1101028.