^{1}

^{2}

^{*}

^{3}

^{1}

This paper proposes a system representation for unifying control design and numerical calculation in nonlinear optimal control problems with inequality constraints in terms of the symplectic structure. The symplectic structure is derived from Hamiltonian systems that are equivalent to Hamilton-Jacobi equations. In the representation, the constraints can be described as an input-state transformation of the system. Therefore, it can be seamlessly applied to the stable manifold method that is a precise numerical solver of the Hamilton-Jacobi equations. In conventional methods, e.g., the penalty method or the barrier method, it is difficult to systematically assign the weights of penalty functions that are used for realizing the constraints. In the proposed method, we can separate the adjustment of weights with respect to objective functions from that of penalty functions. Furthermore, the proposed method can extend the region of computable solutions in a state space. The validity of the method is shown by a numerical example of the optimal control of a vehicle model with steering limitations.

An optimal control is one of the most important strategies in control design. Optimal control problems for nonlinear systems can be formulated by Hamilton-Jacobi equations [

On the other hand, almost all actual control systems possess not only nonlinearity, but also constraints with respect to, e.g., inputs or state variables. Constrained optimal problems have attracted a lot of attentions, and many methods, e.g., the penalty method or the barrier method have been proposed for such systems [

This paper proposes the formulation for integrating inequality constraints in the nonlinear optimal regulator design using the stable manifold method. The formulation is derived from the input-output linearization technique [

Furthermore, our formulation has the following three advantages. First, a heuristic parameter tuning of weights is practically required for convergence of calculations in many cases. However, in conventional methods, the weights of error functions such as quadratic terms with respect to states or inputs and penalty functions are described as a single objective function. Thus, it is difficult to independently adjust each weight, because solutions in nonlinear optimizations are quite sensitive with respect to such a change. In our method, the constraint is described as a part of control systems by using the transformation between inputs and state variables; therefore, the weights can be separately adjusted. Second, in the barrier method (or the interior penalty method), it is not so easy to set initial states of numerical calculations in a constrained region in some cases. In the stable manifold method, optimal orbits are calculated from a stable point at the origin to the surrounding area of the origin in the inverse direction of time evolutions. Therefore, we can systematically search solutions intersecting a given initial condition. Third, cost functions with Lagrange multiplier are used for incorporating equality constraints in addition to penalty functions in the multiplier method. The method is based on a local optimality, i.e., the Karush-Kuhn-Tucker condition, and solutions are approximately calculated by an iterative calculation, e.g., the sequential quadratic programming method [

This paper is constructed as follows: Section 2 states basic definitions of nonlinear optimal control problems and provides the brief introduction of our method. In Section 3, we first present a formulation of nonlinear optimal control problems with constraints in terms of differentiable saturation functions. Next, an augmented system including the constraints is introduced from the formulation. Then, we discuss three topics: the implementation of the system with the stable manifold method, the optimality of the formulation, and the extension to multi-input and multi-constraint cases. Finally, in Section 4, a numerical control example for a vehicle model with steering limitations modeled by a saturation function is illustrated to show the validity of our method.

In this section, we make a brief summary of nonlinear optimal control problems and formalize a class of control systems with constraints described by inequalities.

Let us consider the following nonlinear system:

with an initial condition

Then, we consider the following constraint with respect to a state variable

where

Our method has been constructed with a mind to be applied to the stable manifold method [

Assumption 1. System (1) satisfies

1.

2. The approximate linear system of (1) is stabilizable.

We first recall the standard setting of nonlinear optimal control problems.

Problem 1. Find a control input u in (1) minimizing the objective function

where the weights Q and R are diagonal matrices such that

The Hamiltonian H in the variational calculus of the above problem can be introduced by using Lagrange multiplier

That is, Problem 1 can be rewritten as the problem of minimizing H. Then, from the stationary condition with respect to u, we obtain the optimal feedback as follows:

By substituting (6) into (5) and using dynamic programming, we get the following Hamilton-Jacobi equation:

If we can solve (7) with respect to x and p, then the solution

The stable manifold method, which is the main tool in this paper, is an integral recursion formula for calculating solutions of the Hamiltonian systems that are transformed from the Hamilton-Jacobi Equation (7) that are directly difficult to solve (see Appendix).

The penalty method and the barrier method are extended optimization methods for treating constraints. In these methods, the penalty function

are respectively added to the cost function (4), where S is the domain within the constraint,

This section briefly summarizes the proposed method for applying the stable manifold method to nonlinear optimal problems with inequality constraints. The main idea of the method is to transform the problem as a standard problem for unconstrained systems. That is, we derive an input transformation that acts as the constraints with respect to a state

Let us consider the original control system

where

and

The input of

Example 1. Let us consider the nonlinear system

under the constraint

On the other hand, by substituting the relation

where

Furthermore, we can reduce this expression by eliminating

This system actually includes the inequality constraint. Moreover, the system (16) is an input affine first order nonlinear system, and it satisfies Assumption 1. Hence, we can apply the stable manifold method to the nonlinear optimal control problem of (16) instead of (13) with the constraint. In this case, the objective function is given as

where Q and R are some weight matrices, and we have defined the state variables

In this section, we formally state the formulation of the previously discussed basic concept for constrained systems, and then we prove the optimality of the formulation. Finally, we show that the formulation is generalized for multi-input-output systems with higher relative degrees.

For simplification, we first introduce the standard formulation for single-input-output systems (1) with the constraint (3) from [

Definition 1. Let y be the output of the system (1). We call

and we denote

In this paper, we consider the following function describing inequality constraints.

Definition 2. Let

1.

2. bijective,

3.

4. there exists a constant

We call such an

Proposition 1. The state

for any

Proof. If (3) holds, there exists a unique z satisfying (19), because

Proposition 2. Consider a k-th differentiable saturation function

where

Proof. By integrating the both side of

where

In the same way of the integration for each l, we can see that (20) is equal to (19). ,

Thus, we use the relation (20) instead of the inequality constraint (3) in the nonlinear optimal control design.

Remark 1. In Definition 2, the function

In this section, we clarify the representation of the connected system

Proposition 3. Let

where

Proof. By applying a time differentiation to (19), we get

For

By regarding the first and second terms in (25) as

Proposition 4. Consider the system (1) with the state

where we assume that the initial condition is

Proof. From the direct calculation of (23) for

Hence, we can determine u by using the second equality. ,

In Proposition 4, the subsystem with respect to z corresponds with

Corollary 1. The system (26) is equivalent to the following system under the assumption that the constraint in (19) holds:

where we have defined

Let us consider designing an optimal feedback (6) for the system (30) by using the stable manifold method. To apply the stable manifold method, we must check whether Assumption 1 holds. The first assumption obviously holds.

Lemma 1. Consider the constrained system (30). Then,

Proof. In the original system (1), we assumed that

Consequently, the following condition is obtained from the above fact.

Theorem 5. Consider the nonlinear optimal regulator design of the system (2) that satisfies Assumption 1 with the constraint (3) in terms of the augmented system (30). The stable manifold method can be applied to the system if the linearized system of the augmented system (30) is stabilizable.

Proof. By Corollary 1 and Lemma 1, we can prove the applicability of the stable manifold method for the system. A stable manifold is defined by the adjoint variable

Furthermore, we can derive the following Hamiltonian system from the above representation in (30) for the constrained system.

Lemma 2. Equation (7) for the system (30) can be transformed into the equivalent Hamiltonian system

One of the most important advantages of this implementation of the stable manifold method is the following symplectic property of the numerical scheme.

Proposition 6. The numerical precision on the optimality of controllers obtained from the stable manifold method for the constrained optimal problem can be checked by the condition whether the Hamiltonian

Proof. The Hamilton-Jacobi equation is defined by

This section shows that the formulation using the augmented system (30) is reasonable in the sense of optimal problems.

Our purpose is to find u that is subject to minimize the cost function J in Problem 1. However, the costs with respect to

Problem 2. Find a control input u in (1) minimizing the objective function

for the augmented system (30), where the weights

The purpose of the term

Theorem 7. If the weight

if the i-th components

an approximation of that of Problem 1 with arbitrary accuracy.

To prove the above theorem, we prepare the following facts.

Lemma 3. Consider

Proof. According to Definition 2,

and

Lemma 4. If the i-th components

where the sketch of the function W is illustrated in

and

Proof. Let

where we used the relation in Lemma 3. Therefore, Equation (34) can be rewritten as

because

Proof of Theorem 1. From Lemma 4, we can obtain the correspondence between (4) and (34) under the assumptions. On the other hand, in the barrier method, if there exist an optimal solution

In this section, we describe the basic idea of extending the previous discussion on single input systems (i.e.,

As the simplest case, if inputs are isolated with each other in their input-output relations, i.e., they do not have a common output, multi-input system representation can be defined by independently applying the same way of the single input case to each input with constraint. On the other hand, there might exist many inputs

We first consider the following case of two inputs with different relative degrees for simplification.

Proposition 8. Consider the system (1). Let y be the output of the system with the relative degree

Proof. If

The above discussion can be easily extended to the case of multi inputs with different relative degrees.

Next, we consider the case when many inputs with the same relative degree are derived from a certain y by differentiations. In this case, u is not uniquely determined from (29), because

Definition 3. Let

and

as in (29). There exists some j satisfying

By eliminating

Proposition 9. Let

Proof. The input

As a result, in the multi constraint case, we only have to apply the above two extensions to each input, and the integration to the stable manifold method is basically the same construction of the single constraint case.

This section shows the numerical result of the nonlinear optimal control for a vehicle model to demonstrate of the validity of the proposed method.

Let us consider a 2-wheel vehicle model that is equivalent to a 4-wheel vehicle under the following assumptions: the characteristics of wheels are same, resistive forces except for the friction between tires and grounds are negligible, and the equilibrium point of the system is a state of a steady driving with a constant speed. The state equation of the model is given by

where the state variables

where the subscript i means the front wheel if

Let us design an optimal regulator for the vehicle model with the inequality constraint as an angle limitation of the front wheels. The purpose is to stabilize the state variables

as

where

with

We solved the nonlinear optimal control problem for the system (48) with the constraint (47) by the stable manifold method. The result was compared that of the penalty method with the penalty function

reduced the difference between the both methods as far as possible by using the following small weights with respect to inputs:

On the other hand,

Finally,

This paper presented a control system representation that seamlessly can be applied to a precise numerical solver of Hamilton-Jacobi equations called the stable manifold method in the optimal regulator design for nonlinear systems with inequality constraints. The representation was derived from the slack variable method and the input-output linearization technique without any approximation of solutions and any system reduction. We clarified the following four facts: how to integrate the representation to stable manifold method, the weight adjustments of error and penalty functions which can be separated in this formulation, the optimality of the formulation, and the extension of the representation to multi-input and multi-constraint cases. Finally, the validity of the method was shown by the numerical example of a vehicle model with steering limitations modeled by a saturation function.

The extensions of our method to time-variable or state-dependent constraints, practical numerical simulations using more detailed vehicle models, and discussions on the robustness of the controller obtained from our method are possible future works.

We thank the Editor and the referee for their comments. This work was supported by JSPS Grants-in-Aid for Scientific Research (C) No. 26420415, and JSPS Grants-in-Aid for Challenging Exploratory Research No. 26630197. N. Sakamoto was supported by Nanzan University Pache Research Subsidy I-A-2 for the 2015 academic year.

YoshikiAbe,GouNishida,NoboruSakamoto,YutakaYamamoto, (2015) Symplectic Numerical Approach for Nonlinear Optimal Control of Systems with Inequality Constraints. International Journal of Modern Nonlinear Theory and Application,04,234-248. doi: 10.4236/ijmnta.2015.44018

The stable manifold method calculates a solution of the Hamilton-Jacobi Equation (7) as the stable manifold of the Hamiltonian system (33).

The following information is required in the calculation of the stable manifold method:

i) Calculate a symmetric matrix P such that

that is the linearized relation of the Hamilton-Jacobi equation at the origin, where the matrices

ii) Calculate a matrix S that is a solution of Lyapunov equation

iii) Transform the original system into

by the coordinate transformation

The stable manifold of (33) can be calculated by the following iteration:

i) Calculate sequences of functions,

for some parameter

ii) By the iterative calculation of (53), extend a solution along an initial vector

iii) If an obtained solution from the iteration passes through a desired initial state of control systems, then the iteration is finished. If not, back to ii).