Abstract

We study some nonlinear optimal control problems under state constraint. We construct extremal flows by differential-algebraic equations to solve an optimal control problem subject to mixed control-state constraint. Then we present an approximation approach to the state constraint optimal control problem.

Keywords

State Constraint Optimal Control, Global Optimization, Extremal Flow, Differential-Algebraic Equation

Share and Cite:

1. Introduction

In general, it is hard to obtain an analytic solution for nonlinear optimal control problem under state constraint. To solve a constraint optimal control problem, it is rather common to use a direct discretization approach to exact solution for the problem [1] [2]. Mathematically, the direct discretization methods should be supported by the investigation of convergence properties for the solutions of discretized problems approximating to the solution of the continuous problem. One usually expects a desired error between a numerical value and the optimal objective value of the original problem. However, for many engineering applications, the direct discretization methods may be useful to deal with some constrained optimal control problems efficiently, but so far for these methods one still expects more researches on theoretical foundation of convergence results [2]. The purpose of this paper is to provide a convergence result for a nonlinear optimal control problem under state constraint.

In this paper, according to the traditional optimal control theory, an admissible control is measurable and bounded on the interval $\left[0,T\right]$ such that the ordinary differential equation in the control problem has a unique solution.

We consider three nonlinear optimal control problems under state constraint as follows.

(1.1)

where the cost function $P\left(x\right)$ is continuously differentiable on $R^n$ and $Q\left(x\right)$ is a convex function on $R^n$ . For this problem the matrix functions $f\left(x\right),g\left(x\right)$ on $R^n$ are smooth and the vector $a\in R^n$ satisfying $Q\left(a\right)<0$ is given in the control system in (1.1).

(1.2)

where the cost function $P\left(x\right)$ is continuously differentiable on $R^n$ and $Q\left(x\right)$ is a convex function on $R^n$ . For this problem the matrix functions $f\left(x\right),g\left(x\right)$ on $R^n$ are smooth and the vector $a\in R^n$ satisfying $Q\left(a\right)<0$ is given in the control system in (1.2).

(1.3)

where the parameter $\beta >0$ is given and the cost function $P\left(x\right)$ is continuously differentiable on $R^n$ and $Q\left(x\right)$ is a convex function on $R^n$ . For this problem the matrix functions $f\left(x\right),g\left(x\right)$ on $R^n$ are smooth and the vector $a\in R^n$ satisfying $Q\left(a\right)<0$ is given in the control system in (1.3).

Main assumption: In this paper, we assume that the sets of admissible control of all problems concerned in this paper are not empty.

Remark 1.1. Through out the paper by optimal value of the problem we mean the infimum of the cost functional, i.e. $\inf J\left(u\right)$ . On the other hand, noting that $P\left(x\right)$ is continuously differentiable and that the matrix functions $f\left(x\right),g\left(x\right)$ on $R^n$ are smooth, by functional analysis we see that the cost functional

$J\left(u\right)=P\left(x\left(T\right)\right)+{\displaystyle {\int }_0^T\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right)\text{d}t}$ is continuous on admissible control space.

The rest of the paper is organized as follows. In Section 2, to deal with the problem , we present a partial differential equation by rewriting the Hamilton-Jacobi-Bellman equation. Then we create an extremal flow by a differential-algebraic equation to compute the optimal value of the problem . We prove a conver-gence theorem for an approximation approach to the optimal value of the problem by a series of optimal values of the problem with different parameters in Section 3. In Section 4, we provide a convergence result for the optimal value of the problem by proving that the problem and the problem have the same optimal value. We give a proof of Theorem 2.1 in Section 5 and a conclusion in Section 6.

2. A Study on Optimal Control Problem Subject to Mixed Control-State Constraint

In this section, we deal with the optimal control problem by a partial differential equation. In the following, the positive number $\beta$ is fixed. For given $x\in R^n$ , define a set

$S\left(x\right)=\left\{u:u^{\text{T}}u<\frac{-Q\left(x\right)}{\beta }\right\}.$ (2.1)

Note that if $Q\left(x\right)\ge 0$ then $S\left(x\right)=\varnothing$ . In the following we assume that

$S\left(x\right)\ne \varnothing .$ (2.2)

We consider the Hamilton-Jacobi-Bellman equation as follows:

$v_t\left(t,x\right)+v_x^{\text{T}}\left(t,x\right)f\left(x\right)+\underset{u\in S\left(x\right)}{\text{min}}\left\{v_x^{\text{T}}\left(t,x\right)g\left(x\right)u+\frac{1}{2}{u}^{\text{T}}u\right\}=0,\left(T,x\right)=P\left(x\right).$ (2.3)

For given $\left(\lambda ,x\right)\in R^m\times R^n$ with $Q\left(x\right)<0$ , define a function

$H\left(\lambda ,x\right):=\underset{u\in S\left(x\right)}{\text{min}}\left\{{\lambda }^{\text{T}}u+\frac{1}{2}{u}^{\text{T}}u\right\},$ (2.4)

then for $t\in \left[0,T\right]$ and $\lambda =g^{\text{T}}\left(x\right)v_x\left(t,x\right)$ , we have

$H\left(g^{\text{T}}\left(x\right)v_x\left(t,x\right),x\right)=\underset{u\in S\left(x\right)}{\text{min}}\left\{v_x^{\text{T}}\left(t,x\right)g\left(x\right)u+\frac{1}{2}{u}^{\text{T}}u\right\}.$ (2.5)

By (2.5), we can rewrite the Hamilton-Jacobi-Bellman equation in (2.3) with global optimization to obtain the following partial differential equation [3]:

$v_t\left(t,x\right)+v_x^{\text{T}}\left(t,x\right)f\left(x\right)+H\left(g^{\text{T}}\left(x\right)v_x\left(t,x\right),x\right)=0,\left(T,x\right)=P\left(x\right).$ (2.6)

We will solve the optimal control problem in (1.3) by the partial dif-ferential equation in (2.6).

Given a pair $\left(x,u\right)\in R^n\times R^m$ , satisfying $u\in S\left(x\right)$ , i.e. $u^{\text{T}}u<\frac{-Q\left(x\right)}{\beta }$ . For $\frac{-Q\left(x\right)}{\beta }>0$ , in the following let $\widehat{u}$ denote the global minimizer of ${\min }_{u\in S\left(x\right)}\left\{{\lambda }^{\text{T}}u+\frac{1}{2}{u}^{\text{T}}u\right\}$ , i.e. $H\left(\lambda ,x\right)={\lambda }^{\text{T}}\widehat{u}+\frac{1}{2}{\widehat{u}}^{\text{T}}\widehat{u}={\text{min}}_{u\in S\left(x\right)}\left\{{\lambda }^{\text{T}}u+\frac{1}{2}{u}^{\text{T}}u\right\}$. We need the following lemma to study the expression of $H\left(g^{\text{T}}\left(x\right)v_x\left(t,x\right),x\right)$ . For given $r>0$ and $\lambda \in R^m$ , we then define auxiliary function $h\left(\lambda ,r\right)={\text{min}}_{{\Vert u\Vert }^2<r}\left\{{\lambda }^{\text{T}}u+\frac{1}{2}{u}^{\text{T}}u\right\}$ . Again let $\widehat{u}$ denote the global minimizer of ${\text{min}}_{{\Vert u\Vert }^2<r}\left\{{\lambda }^{\text{T}}u+\frac{1}{2}{u}^{\text{T}}u\right\}$ , i.e. $h\left(\lambda ,x\right)={\lambda }^{\text{T}}\widehat{u}+\frac{1}{2}{\widehat{u}}^{\text{T}}\widehat{u}={\text{min}}_{{\Vert u\Vert }^2<r}\left\{{\lambda }^{\text{T}}u+\frac{1}{2}{u}^{\text{T}}u\right\}$. We see that, given $x\in R^n$ satisfying $\frac{-Q\left(x\right)}{\beta }>0$ , for $r=\frac{-Q\left(x\right)}{\beta }$ , $\lambda =g^{\text{T}}\left(x\right)v_x\left(t,x\right)$ , we have $H\left(g^{\text{T}}\left(x\right)v_x\left(t,x\right),x\right)=h\left(\lambda ,r\right)$ . By primary optimi-zation theory we have the following lemma.

Lemma 2.1. Given $r>0$ and $\lambda \in R^m$ . If ${\Vert \lambda \Vert }^2<r$ , then $\widehat{u}=-\lambda ,h\left(\lambda ,r\right)=\frac{-{\Vert \lambda \Vert }^2}{2}$ . On the other hand, if ${\Vert \lambda \Vert }^2\ge r$ , then $\widehat{u}=\sqrt{r}\left(\frac{-\lambda }{\Vert \lambda \Vert }\right)$ , $\left(\lambda ,r\right)=\frac{r}{2}-\sqrt{r}\Vert \lambda \Vert$ .

For given $\left(t,x\right)\in \left[0,T\right]\times R^n$ such that $\frac{-Q\left(x\right)}{\beta }>0$ , denoting $\frac{-Q\left(x\right)}{\beta }$ by $r$ and denoting $g^{\text{T}}\left(x\right)v_x\left(t,x\right)$ by $\lambda$ , by Lemma 2.1, we see that, if ${\Vert g^{\text{T}}\left(x\right)v_x\left(t,x\right)\Vert }^2<\frac{-Q\left(x\right)}{\beta }$ , then

$\widehat{u}=\phi \left(t,x\right):=-g^{\text{T}}\left(x\right)v_x\left(t,x\right),H\left(g^{\text{T}}\left(x\right)v_x\left(t,x\right),x\right)=\frac{-1}{2}{\Vert g^{\text{T}}\left(x\right)v_x\left(t,x\right)\Vert }^2,$ (2.7)

and if ${\Vert g^{\text{T}}\left(x\right)v_x\left(t,x\right)\Vert }^2\ge \frac{-Q\left(x\right)}{\beta }$ ,then

$\begin{array}{l}\widehat{u}=\phi \left(t,x\right):=\sqrt{\frac{-Q\left(x\right)}{\beta }}\left(\frac{-g^{\text{T}}\left(x\right)v_x\left(t,x\right)}{\Vert g^{\text{T}}\left(x\right)v_x\left(t,x\right)\Vert }\right),\\ H\left(g^{\text{T}}\left(x\right)v_x\left(t,x\right),x\right)=\frac{-Q\left(x\right)}{2\beta }-\sqrt{\frac{-Q\left(x\right)}{\beta }}\Vert g^{\text{T}}\left(x\right)v_x\left(t,x\right)\Vert .\end{array}$ (2.8)

Remark 2.1. By Lemma 2.1 we see that $H\left(\lambda ,x\right)$ is continuous with respect to $\left(\lambda ,x\right)$ . We can get a viscosity solution of the partial differential equation in (2.6) [4]-[6]. Then the Hamilton-Jacobi-Bellman equation in (2.3) can be solved for a numerical solution [7].

Definition 2.1. For a solution $v\left(t,x\right)$ of the partial differential equation in (2.6), we call $\left(\widehat{x}(\cdot ),\widehat{u}(\cdot )\right)$ an extremal flow related to $v\left(t,x\right)$ if it is a solution of the following differential-algebraic equation:

$\dot{\widehat{x}}\left(t\right)=f\left(\widehat{x}\left(t\right)\right)+g\left(\widehat{x}\left(t\right)\right)\widehat{u}\left(t\right),\widehat{x}\left(0\right)=a\in R^n,$ (2.9)

$\begin{array}{l}{v}_x^{\text{T}}\left(t,\widehat{x}\left(t\right)\right)g\left(\widehat{x}\left(t\right)\right)\widehat{u}\left(t\right)+\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\\ =H\left(g^{\text{T}}\left(\widehat{x}\left(t\right)\right)v_x\left(t,\widehat{x}\left(t\right)\right),\widehat{x}\left(t\right)\right),t\in \left[0,T\right].\end{array}$ (2.10)

By the same way in [7], we can prove the following theorem.

Theorem 2.1. Let $v\left(t,x\right)$ be a solution of the partial differential equation in (2.6) and $\left(\widehat{x}(\cdot ),\widehat{u}(\cdot )\right)$ be an extremal flow defined by (2.9), (2.10). Then, $\widehat{u}(\cdot )$ is an optimal control of the problem , and

$v\left(0,a\right)=P\left(\widehat{x}\left(T\right)\right)+{\displaystyle {\int }_0^T\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\text{d}t}$ is the optimal value of the problem .

Theorem 2.2. If the continuously differentiable function $v\left(t,x\right)$ is a solution of the partial differential equation in (2.6) on $\left[0,\infty \right)\times \left\{x:-Q\left(x\right)>0\right\}$ and $\phi \left(t,x\right)$ is the function defined in (2.7), (2.8), then $u=\phi \left(t,x\right)$ is an optimal feedback control of the problem .

Proof: Since $v_x\left(t,x\right)$ is continuous, by (2.7), (2.8), we see that $\phi \left(t,x\right)$ is continuous on $\left[0,T\right]\times \left\{x:-Q\left(x\right)>0\right\}$ . By classical theory of ordinary differential equation we see that the equation

$\dot{x}\left(t\right)=f\left(x\left(t\right)\right)+g\left(x\left(t\right)\right)\phi \left(t,x\left(t\right)\right),\widehat{x}\left(0\right)=a\in R^n$ (2.11)

has a solution on $\left[0,T\right]\times \left\{x:-Q\left(x\right)>0\right\}$ . Let the solution of the ODE in (2.11) be denoted by $\widehat{x}\left(t\right)$ and let $\phi \left(t,\widehat{x}\left(t\right)\right)$ be denoted by $\widehat{u}\left(t\right)$ . By lemma 2.1 and (2.7),(2.8) we see that

$\begin{array}{l}{v}_x^{\text{T}}\left(t,\widehat{x}\left(t\right)\right)g\left(\widehat{x}\left(t\right)\right)\widehat{u}\left(t\right)+\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\\ =H\left(g^{\text{T}}\left(\widehat{x}\left(t\right)\right)v_x\left(t,\widehat{x}\left(t\right)\right),\widehat{x}\left(t\right)\right),t\in \left[0,T\right].\end{array}$ (2.12)

Noting (2.11), (2.12), by Definition 2.1, the pair $\left(\widehat{x}(\cdot ),\widehat{u}(\cdot )\right)$ is an extremal flow related to $v\left(t,x\right)$ . It follows from Theorem 2.1 that $u=\phi \left(t,x\right)$ is an optimal feedback control of the problem .

3. An Approximation Approach to the Optimal Value of Problem

In this section we show a convergent result for an approximation approach to the optimal value of problem which is restated as follows.

(3.1)

where the cost function $P\left(x\right)$ is continuously differentiable on $R^n$ and $Q\left(x\right)$ is a convex function on $R^n$ . In this problem the matrix functions $f\left(x\right),g\left(x\right)$ on $R^n$ are continuously differentiable and the vector $a\in R^n$ such that $Q\left(a\right)<0$ are given in the control system in (3.1).

In the following, for a given positive number $\beta$ , the optimal value of problem is denoted by $V_{\beta }$ and the optimal value of problem is denoted by $\widehat{V}$ .

Lemma 3.1. (i). For each given number $\beta >0$ , $V_{\beta }\ge \widehat{V}$ . (ii). If ${\beta }_1\ge {\beta }_2>0$ , then $V_{{\beta }_1}\ge V_{{\beta }_2}$ .

Proof: Firstly, let $\left(x(\cdot ),u(\cdot )\right)$ be an admissible pair of the problem . Note that the functions $f\left(x\right),g\left(x\right)$ and the vector $a$ appearing in and are the same. It follows from the fact $Q\left(x\left(t\right)\right)+\beta u^{\text{T}}\left(t\right)u\left(t\right)<0$ , $t\in \left[0,T\right]$ that $Q\left(x\left(t\right)\right)<0$ , $t\in \left[0,T\right]$ . Thus $\left(x(\cdot ),u(\cdot )\right)$ is also an admissible pair of the problem . Consequently, $V_{\beta }\ge \widehat{V}$ .

Secondly, let $\left(x(\cdot ),u(\cdot )\right)$ be an admissible pair of the problem with the parameter ${\beta }_1$ . Note that functions $f\left(x\right),g\left(x\right)$ and the vector $c,a$ appearing in do not depend on different parameter $\beta$ . Noting ${\beta }_1\ge {\beta }_2>0$ , it follows from the fact $Q\left(x\left(t\right)\right)+{\beta }_1{u}^{\text{T}}\left(t\right)u\left(t\right)<0$ , $t\in \left[0,T\right]$ that $Q\left(x\left(t\right)\right)+{\beta }_2{u}^{\text{T}}\left(t\right)u\left(t\right)<0$ , $t\in \left[0,T\right]$ . Thus $\left(x(\cdot ),u(\cdot )\right)$ is also an admissible pair of the problem with the parameter ${\beta }_2$ . Consequently, $V_{{\beta }_1}\ge V_{{\beta }_2}$ . The lemma is proved.

Theorem 3.1. For given $ϵ>0$ there exists a positive number $\beta$ such that

$\left|V_{\beta }-\widehat{V}\right|<ϵ.$ (3.2)

Proof: Given $ϵ>0$ . Let $\left(\overline{x},\overline{u}\right)$ be an admissible pair of the problem such that

$\widehat{V}\le J\left(\overline{u}\right)<\widehat{V}+ϵ,$ (3.3)

noting that $\widehat{V}$ is the infimum of $J\left(u\right)$ for the problem .

Noting that $\overline{x}\left(t\right)$ is continuous and $c^{\text{T}}\overline{x}\left(t\right)<0,t\in \left[0,T\right]$, we see that there is a $\delta >0$ such that

$Q\left(\overline{x}\left(t\right)\right)<-\delta ,t\in \left[0,T\right].$ (3.4)

Noting that the admissible control is bounded on $\left[0,T\right]$ , there is a number $M>0$ such that

$\overline{u}{\left(t\right)}^{\text{T}}\overline{u}\left(t\right)\le M,t\in \left[0,T\right].$ (3.5)

By (3.4), (3.5), there is a $\beta >0$ , such that

$Q\left(\overline{x}\left(t\right)\right)+\beta \overline{u}{\left(t\right)}^T\overline{u}\left(t\right)\le -\delta +\beta M<0,t\in \left[0,T\right].$ (3.6)

Thus $\left(\overline{x},\overline{u}\right)$ is an admissible pair of both problem and problem with the parameter $\beta$ . Then we have

$J\left(\overline{u}\right)\ge V_{\beta }$ (3.7)

By Lemma 3.1 and (3.3),(3.7) we have

$0\le V_{\beta }-\widehat{V}<V_{\beta }-\left(J\left(\overline{u}\right)-ϵ\right)\le J\left(\overline{u}\right)-\left(J\left(\overline{u}\right)-ϵ\right)=ϵ.$ (3.8)

Therefore (3.2) is true and the theorem has been proved.

Corollary 3.1. Let ${\beta }_n,n=1,2,\cdots$ be a decrease sequence of positive numbers satisfying ${\beta }_n\to 0$ when $n\to +\infty$ . Then

$\underset{n\to +\infty }{\lim }{V}_{{\beta }_n}=\widehat{V}.$ (3.9)

Proof: By Lemma 3.1, noting (3.3), (3.6) in the proof of Theorem 3.1, for each $n\left(=1,2,\cdots \right)$ , there is an admissible pair $\left(x_n,u_n\right)$ of the problem satisfying

$\widehat{V}\le J\left(u_n\right)<\widehat{V}+\frac{1}{n},$ (3.10)

noting that $\widehat{V}$ is the infimum of $J\left(u\right)$ for the problem .

Noting that $x_n(.)$ is continuous and $c^{\text{T}}{x}_n\left(t\right)<0,t\in \left[0,T\right]$ , we see that there is a ${\delta }_n>0$ such that

$Q\left(x_n\left(t\right)\right)<-{\delta }_n<0,t\in \left[0,T\right],$ (3.11)

and noting that the admissible control is bounded on $\left[0,T\right]$ , there is a number $M_n>0$ such that

$u_n{\left(t\right)}^{\text{T}}{u}_n\left(t\right)\le M_n,t\in \left[0,T\right].$ (3.12)

By (3.11), (3.12), there is a ${\beta }_n>0$ , such that

$Q\left(x_n\left(t\right)\right)+{\beta }_{\beta }{u}_n{\left(t\right)}^{\text{T}}{u}_n\left(t\right)\le -{\delta }_n+{\beta }_n{M}_n<0,t\in \left[0,T\right].$ (3.13)

Thus $\left(x_n,u_n\right)$ is an admissible pair of both problem and problem with the parameter ${\beta }_n$ . Then we have

$J\left(u_n\right)\ge V_{{\beta }_n}.$ (3.14)

This process (3.10)-(3.14) begins from $n=1$ . But by Lemma 3.1 we see that if a positive number $\beta$ is got to satisfy (3.13) then for every positive number ${\beta }^{\prime }$ less than $\beta$ the process (3.10)-(3.14) still works. For $n=1$ , we choose $0<{\beta }_1<1$ as in (3.10)-(3.14). After the step $n\left(\ge 1\right)$ has been done, in the next

step we choose $0<{\beta }_{n+1}\le \frac{{\beta }_n}{n+1}\left(<{\beta }_n\right)$ such that

$Q\left(x_{n+1}\left(t\right)\right)+{\beta }_{n+1}{u}_{n+1}{\left(t\right)}^{\text{T}}{u}_{n+1}\left(t\right)\le -{\delta }_{n+1}+{\beta }_{n+1}{M}_{n+1}<0,t\in \left[0,T\right].$ (3.15)

Then we see that in this way the positive sequence $\left\{{\beta }_n\right\}$ is strictly decreasing and tends to zero when $n\to \infty$ . By the deductive process the same as (3.8) in the proof of Theorem 3.1, or by Lemma 3.1 and (3.10), (3.14), we have for each $n=1,2,\cdots$ ,

$0\le V_{{\beta }_n}-\widehat{V}<J\left(u_n\right)-\left(J\left(u_n\right)-\frac{1}{n}\right)=\frac{1}{n}.$ (3.15)

Therefore we have

$\underset{n\to +\infty }{\text{lim}}{V}_{{\beta }_n}=\widehat{V},$ (3.16)

with the positive sequence $\left\{{\beta }_n\right\}$ being strictly decreasing and tending to zero when $n\to \infty$ . The Corollary 3.1 has been proved.

4. On the Optimal Value of Problem .

In this section we deal with the problem :

(4.1)

In the following, the optimal value of the problem is denoted by $V^{\text{*}}$ . Recall that in Section 4 the optimal value of the problem is denoted by $\widehat{V}$ . In the following lemma, noting that $c^{\text{T}}a<0$ , we define two sets of admissible control as follows:

$D_1:=\left\{u(.):Q\left(x_u\left(t\right)\right)\le 0,t\in \left[0,T\right]\right\},$

$D_2:=\left\{u(.):Q\left(x_u\left(t\right)\right)<0,t\in \left[0,T\right]\right\}.$

Lemma 4.1. Under the notations above, we have

$\widehat{V}\ge V^{\text{*}}.$ (4.2)

Proof: Let $x_u(.)$ be the solution of the linear equation

$\dot{x}\left(t\right)=f\left(x\left(t\right)\right)+g\left(x\left(t\right)\right)u\left(t\right),x\left(0\right)=a\left(\in R^n\right)$

corresponding to an admissible control $u(.)$ . It is clear that $D_2\subset D_1$ . Con-sequently,

$\widehat{V}\ge V^{\text{*}}.$ (4.4)

The lemma is proved.

In the following lemma we should recall that, in the first section of this paper, we have assumed that the admissible control set $D_2=\left\{u(.):c^{\text{T}}{x}_u\left(t\right)<0,t\in \left[0,T\right]\right\}$ is not empty.

Lemma 4.2. Let $\overline{u}(.)\in D_2=\left\{u(.):Q\left(x_u\left(t\right)\right)<0,t\in \left[0,T\right]\right\}$ . Then for any admis-sible control $\tilde{u}(.)$ such that $Q\left(x_u\left(t\right)\right)\le 0$ , $t\in \left[0,T\right]$ , we have, for $\forall \alpha \in \left(0,1\right]$ ,

$\alpha \overline{u}(.)+\left(1-\alpha \right)\tilde{u}(.)\in D_2=\left\{u(.):Q\left(x_u\left(t\right)\right)<0,t\in \left[0,T\right]\right\}.$ (4.5)

Proof: Let $\overline{x}(.)$ and $\tilde{x}(.)$ be the trajectories of the linear system $\dot{x}\left(t\right)=Ax\left(t\right)+Bu\left(t\right)$ , $x\left(0\right)=a\in R^n$ corresponding to $\overline{u}(.)$ and $\tilde{u}(.)$ respec-tively. Noting that $Q\left(x\right)$ is convex, we have, for $t\in \left[0,T\right]$ ,

$Q\left(\alpha \overline{x}\left(t\right)+\left(1-\alpha \right)\tilde{x}\left(t\right)\right)=\alpha Q\left(\overline{x}\left(t\right)\right)+\left(1-\alpha \right)Q\left(\tilde{x}\left(t\right)\right)<0,$ (4.6)

also noting that $Q\left(\overline{x}\left(t\right)\right)<0$ and $Q\left(\tilde{x}\left(t\right)\right)\le 0$ , $\alpha \in \left(0,1\right]$ . The lemma is proved.

Theorem 4.1. Let the notations of $\widehat{V},V^{\text{*}}$ be as in Lemma 4.1. Then

$V^{\text{*}}=\widehat{V}.$ (4.7)

Proof: Let $\overline{u}(.)\in D_2=\left\{u(.):c^{\text{T}}{x}_u\left(t\right)<0,t\in \left[0,T\right]\right\}$. We show (4.7) by induc-tion. In the initial step, for given $\delta >0$ , we have an admissible control $\widehat{u}(.)$ satisfying $c^{\text{T}}{x}_{\widehat{u}}\left(t\right)\le 0$, $t\in \left[0,T\right]$ and

$V^{\text{*}}\le J\left(\widehat{u}\right)<V^{\text{*}}+\delta ,$ (4.8)

noting that $V^{\text{*}}$ is the infimum of $J\left(u\right)$ for the problem .

Recalling Remark 1.1, noting that each admissible control is bounded, the func-tion $P\left(x\right)$ in the cost functional for the concerned problems is continuously differentiable and the cost functional $J(.)$ is continuous on the control space. By Lemma 4.2, there exists a number $\alpha >0$ , such that the control

$u_{\alpha }(.)=\alpha \overline{u}(.)+\left(1-\alpha \right)\widehat{u}(.)=\widehat{u}(.)+\alpha \left(\overline{u}(.)-\widehat{u}(.)\right)$ (4.9)

satisfying

$V^{\text{*}}\le J\left(u_{\alpha }\right)<V^{\text{*}}+\delta$ (4.10)

and

$Q\left(x_{\alpha }\left(t\right)\right)<0,t\in \left[0,T\right].$ (4.11)

Thus $u_{\alpha }$ is also an admissible control for the problem . Then by Lemma 4.1 and (4.10) we have

$V^{\text{*}}\le \widehat{V}\le J\left(u_{\alpha }\right)<V^{\text{*}}+\delta ,$ (4.12)

consequently,

$V^{\text{*}}\le \widehat{V}<V^{\text{*}}+\delta .$ (4.13)

Next the same as in the previous step we have an admissible control which is denoted by $\tilde{u}(.)$ satisfying $c^{\text{T}}{x}_{\tilde{u}}\left(t\right)\le 0$, $t\in \left[0,T\right]$ such that

$V^{\text{*}}\le J\left(\tilde{u}\right)<V^{\text{*}}+\frac{\delta }{2},$

noting that $V^{\text{*}}$ is the infimum of $J\left(u\right)$ for the problem . As in the previous step above, we have a number $\tilde{\alpha }>0$ , such that the control

$u_{\tilde{\alpha }}(.)=\tilde{\alpha }\overline{u}(.)+\left(1-\tilde{\alpha }\right)\tilde{u}(.)=\tilde{u}(.)+\tilde{\alpha }\left(\overline{u}(.)-\tilde{u}(.)\right)$ (4.14)

satisfying

$V^{\text{*}}\le J\left(u_{\tilde{\alpha }}\right)<V^{\text{*}}+\frac{\delta }{2}$ (4.15)

and

$Q\left(x_{\tilde{\alpha }}\left(t\right)\right)<0,t\in \left[0,T\right].$ (4.16)

Thus $u_{\tilde{\alpha }}$ is also an admissible control for the problem . Then by Lemma 4.1 and (4.15) we have

$V^{\text{*}}\le \widehat{V}\le J\left(u_{\tilde{\alpha }}\right)<V^{\text{*}}+\frac{\delta }{2},$ (4.17)

consequently,

$V^{\text{*}}\le \widehat{V}<V^{\text{*}}+\frac{\delta }{2}.$ (4.18)

Similar to the process from (4.13) to (4.18), by induction, in this way, for $n=0,1,2,\cdots$ , when $n=0$ , it is in the initial step, we have $V^{\text{*}}\le \widehat{V}<V^{\text{*}}+\delta$, and when staying in the $n+1$ -th step, we have,

$V^{\text{*}}\le \widehat{V}<V^{\text{*}}+2^{-\left(n+1\right)}\delta .$ (4.19)

Thus for each positive integer $N$ , we have

$0\le \widehat{V}-V^{\text{*}}<2^{-N}\delta .$

Let $N\to +\infty$ , we have

$V^{\text{*}}=\widehat{V}.$ (4.20)

Therefore (4.7) is true and the theorem has been proved. By Theorem 4.1 and Corollary 3.1, we have the following convergence result.

Corollary 4.1. Let ${\beta }_n,n=1,2,\cdots$ be a decrease sequence of positive numbers satisfying ${\beta }_n\to 0$ when $n\to +\infty$ . Then

$\underset{n\to +\infty }{\lim }{V}_{{\beta }_n}=V^{*}.$

5. A proof of Theorem 2.1

By (2.9), (2.10), we have, for $t\in \left[0,T\right]$ ,

$\begin{array}{c}\frac{\text{d}}{\text{d}t}v\left(t,\widehat{x}\left(t\right)\right)=v_t\left(t,\widehat{x}\left(t\right)\right)+v_x^{\text{T}}\left(t,\widehat{x}\left(t\right)\right)f\left(\widehat{x}\left(t\right)\right)+v_x^{\text{T}}\left(t,\widehat{x}\left(t\right)\right)g\left(\widehat{x}\left(t\right)\right)\widehat{u}\left(t\right)\\ =v_t\left(t,\widehat{x}\left(t\right)\right)+v_x^{\text{T}}\left(t,\widehat{x}\left(t\right)\right)f\left(\widehat{x}\left(t\right)\right)\\ +H\left(g^{\text{T}}\left(\widehat{x}\left(t\right)\right)v_x\left(t,\widehat{x}\left(t\right)\right),\widehat{x}\left(t\right)\right)v_x\left(t,\widehat{x}\left(t\right)\right)-\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\\ =-\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right).\end{array}$ (5.1)

Integrating the above equality with respect to $t$ from 0 to $T$ , noting that $v\left(T,\widehat{x}\left(T\right)\right)=P\left(\widehat{x}\left(T\right)\right)$ , $\widehat{x}\left(0\right)=a$ , we have

${\displaystyle {\int }_0^T-\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\text{d}t}={\displaystyle {\int }_0^T\frac{\text{d}}{\text{d}t}v\left(t,\widehat{x}\left(t\right)\right)\text{d}t}=P\left(\widehat{x}\left(T\right)\right)-v\left(0,a\right)$ (5.2)

and

$v\left(0,a\right)=P\left(\widehat{x}\left(T\right)\right)+{\displaystyle {\int }_0^T\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\text{d}t}.$ (5.3)

Now let $\left(x(\cdot ),u(\cdot )\right)$ be an arbitrary admissible pair of the control system in the problem . We have, for $t\in \left[0,T\right]$ ,

$c^{\text{T}}x\left(t\right)+\beta u^{\text{T}}\left(t\right)u\left(t\right)<0,$ (5.4)

which implies $u\left(t\right)\in S\left(x\left(t\right)\right)$ . Thus, by (2.5) with $\lambda =g^{\text{T}}\left(x\left(t\right)\right)v_x\left(t,x\left(t\right)\right)$ for a $t\in \left[0,T\right]$ , we have

$H\left(g^{\text{T}}\left(x\left(t\right)\right)v_x\left(t,x\left(t\right)\right),x\left(t\right)\right)\le v_x^{\text{T}}\left(t,x\left(t\right)\right)g\left(x\left(t\right)\right)u\left(t\right)+\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right).$ (5.5)

Then for each $t\in \left[0,T\right]$ , by the partial deferential equation in (2.6), also noting that $\left(x(\cdot ),u(\cdot )\right)$ is an arbitrary admissible pair of the control system in the problem , we have, by (5.5),

$\begin{array}{c}0=v_t\left(t,x\left(t\right)\right)+v_x^{\text{T}}\left(t,x\left(t\right)\right)f\left(x\left(t\right)\right)+H\left(g^{\text{T}}\left(x\left(t\right)\right)v_x\left(t,x\left(t\right)\right),x\left(t\right)\right)\\ \le v_t\left(t,x\left(t\right)\right)+v_x^{\text{T}}\left(t,x\left(t\right)\right)f\left(x\left(t\right)\right)+v_x^{\text{T}}\left(t,x\left(t\right)\right)g\left(x\left(t\right)\right)u\left(t\right)+\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right)\\ =v_t\left(t,x\left(t\right)\right)+v_x^{\text{T}}\left(t,x\left(t\right)\right)\frac{\text{d}x\left(t\right)}{\text{d}t}+\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right)\\ =\frac{\text{d}}{\text{d}t}v\left(t,x\left(t\right)\right)+\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right).\end{array}$ (5.6)

Integrating the above inequality over $\left[0,T\right]$ , noting $v\left(T,x\left(T\right)\right)=P\left(x\left(T\right)\right)$ , $x\left(0\right)=a$ , by (5.6), we have

$\begin{array}{c}0\le {\displaystyle {\int }_0^T\left[\frac{\text{d}}{\text{d}t}v\left(t,x\left(t\right)\right)+\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right)\right]\text{d}t}\\ =P\left(x\left(T\right)\right)-v\left(0,a\right)+{\displaystyle {\int }_0^T\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right)\text{d}t}.\end{array}$ (5.7)

By (5.3), (5.7), we have

$P\left(\widehat{x}\left(T\right)\right)+{\displaystyle {\int }_0^T\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\text{d}t}=v\left(0,a\right)\le P\left(x\left(T\right)\right)+{\displaystyle {\int }_0^T\frac{1}{2}{u}^{\text{T}}\left(t\right)u\left(t\right)\text{d}t}.$ (5.8)

By (5.8), we see that $\widehat{u}(\cdot )$ is an optimal control for the problem and

$v\left(0,a\right)=P\left(\widehat{x}\left(T\right)\right)+{\displaystyle {\int }_0^T\frac{1}{2}{\widehat{u}}^{\text{T}}\left(t\right)\widehat{u}\left(t\right)\text{d}t}$ is the optimal value of the problem .

The theorem has been proved.

6. Conclusion

In this paper, we study nonlinear optimal control problem under state constraint. Firstly we deal with a nonlinear optimal control problem subject to mixed control-state constraint. We try to create a partial differential equation by Hamilton-Jacobi-Bellman equation with global optimization. Then we provide a convergence result for an approximation approach to the optimal value of a nonlinear optimal control problem under state constraint.

Conflicts of Interest

The author declares no conflicts of interest.

Conflicts of Interest

The author declares no conflicts of interest.

References