Necessary conditions are proved for certain problems of optimal control of diffusions where hard end constraints occur. The main results apply to several dimensional problems, where some of the state equations involve Brownian motions, but not the equations corresponding to states being hard restricted at the terminal time. The necessary conditions are stated in terms of weak variations. Two versions of necessary conditions are given, one version involving solutions of variational equations, the other one involving first order adjoint equations.

Optimal Control Diffusions Hard Terminal State Restrictions
1. Introduction

The purpose of the present paper is to prove necessary conditions for the optimal control of certain types of control problems involving diffusions where hard end constraints on solutions occur. The books  and  contain introductions to the topic of optimal control of diffusions. Various types of maximum principles have been proved for problems of control of diffusions in case of no or soft terminal state restrictions, see e.g.    and  . Maximum principles for problem with hard terminal restrictions are proved for certain types of continuous time controlled diffusion problems in  . In that paper only the drift term contained controls. In this paper controls are allowed to enter also the diffusion term. Singular controls are not discussed, below we merely consider problems where the controls appearing may be said to be absolutely continuous with respect to Lebesgue measure. The restriction to such controls makes it harder to operate with hard terminal state restrictions. In fact, the case discussed in detail in this paper is the case where hard terminal restrictions are only placed on states governed by differential equations not containing a Brownian motion. The necessary condititions are stated in terms of “weak” variations, not strong ones (needleshaped ones). Brownian motions will only appear in differential equations of states unconstrained at the terminal time. The constrained states are, however, influenced by other states directly influenced by Brownian motions, so, below, necessary conditions are stated and proved for such problems. Because the states are stochastic, the state space is infinite dimensional, so to obtain necessary conditions, one must impose a condition amounting to demanding sufficient variability of the first order variations in the problem.

2. The Control Problem and the Statement of the Necessary Condition

Let and let be a given point in the Euclidean space, let be a projection from onto, , such that and let U be a closed convex subset of a Euclidean space. The symbol is used for the Euclidean norm in any Euclidean space, including, and, applied to matrices, it is the linear operator norm. Furnish the interval with the Lebesgue measure. Let be a given filtered probability space, (i.e. for, the’s are sub-s-algebras of the given s-algebra of subsets of, if, P is a probability measure on), and assume that is complete, that contains all the P-null sets in and that is right continuous. Let, , be the set of Lebesgue -measurable functions for which and let be the subset of essentially bounded functions in. Related to, let be a vector the components of which (denoted) are independent one-dimensional Brownian motions all adapted to, such that is independent of for all, , and is normally distributed with mean 0 and covariances, the identity matrix. In applications where the’s are entities that can be observed, it is natural to take as the natural filtration generated by the’s. There are given functions and , from into. Let be the set of functions taking values in U, such that, for each t, when restricted to, is Borel -measurable (i.e. progressively measurable).

Let be the -norm both on and on.

The following conditions are called the Basic Assumptions.

A1 The functions and have derivatives and with respect to, that are continuous in.

A2 As a function of t, the functions f and and their derivatives have one-sided limits with respect to t.

Write for the -matrix whose columns are, let () be the matrix with entries (), and write. Also, write for the indicator function of the set C.

The following assumptions are called the Global Assumptions.

B1 and are uniformly continuous in, uniformly in t.

B2 For some constant M, for some given in, for all,

B3 A constant exists such that for all,

B4

Let be the set of progressively measurable functions in,. From now on we require that all control functions belong to. The strong (unique) solution―continuous in t―of the equation

is denoted.

Let (a fixed,) such that, let denote scalar product, and consider the problem

subject to

Below, will be an optimal control in this problem, assumed to exist. Write.

We have collected a few definitions that are going to be used in the sequel.

For, let

where. Let

Finally, let

In the subsequent necessary conditions, the following local linear controllability condition (15) is needed. There exist numbers, and, and a progressively measurable function such that for all for which

,

A more “concrete” condition implying (15) is presented in Remark 3.

Theorem 1. Assume that is optimal in problem (5),(6), that Assumptions A and B hold (the Basic and Global Assumptions) and that (15) is satisfied. Then the following necessary conditions hold: For some linear functional on, bounded on, and some number, for all,

where is the solution of

with. Moreover1,. ,

Remark 1. If (15) holds for, then. Moreover, when, (21) below holds and is (also) left continuous, then both and is a continuous linear functional on ((15) then holds for). ,

Remark 2. Let, and let be the resolvent of the equation

, so, the identity map in, and. Let, . For, , , is continuous in -norm and hence can be represented

by an -function, progressively measurable and right continuous in t, a.s. and in (even continuous in this manner if is continuous). Note that if on then (16) implies.

Let be the natural filtration generated by the’s, augmented by the P-null sets in. The function, now taken to be a row vector, then satisfies the following condition: On, there exist -valued, progressively measurable functions, , , all row

vectors, continuous in t, such that, for all, and such that

and such that, for all, a.s.

In this case, we have that for all, for a.e. t, a.s.

When, is left continuous, and (21) below holds, the following additional properties hold:, , the -limit (and a.s. limit) exists, , and finally

Remark 3 The condition (15), with, is implied by the following conditions: and for some, for all,

The -norm (or the norm) used in the formulation of the condition (15) can have quite strong consequences. Assume for example that, and, for some, that for all. Then for some, for all, if and (and balls in -spaces,), but if U is strictly smaller than it is neither possible to obtain this inclusion nor (15). When the situation may be different. Consider in particular the case where the original terminal constraint is, say,. Using auxiliary controls and auxiliary states z and y, the system can be rewritten as a system where the end constraints are, , by adding new state equations, , , , and by changing the first state equations to read, ,. Define. We assume now, so the control set in the rewritten system is now. The control is optimal in the rewritten system. Assume that for some for all. Then, for some, for all, , for , a ball in. Then for all, for all so for all ,

,

a ball in. Hence, for all, , where (Necessary conditions for

systems with inequality constraints are stated in Remark 5 below, the local linear controllability condition just obtained suffices for the controllability condition in Remark 5 to apply to the original system).

3. Proof of Theorem 1

The proof consists of three lemmas and six proof steps A.-F., and relies on an “abstract” maximum principle, Corollary I in Appendix.

Without loss of generality, from now on let and let. Note that. For any, let be the solution of

By general existence results and (1) and (2), , and, see (4), do exist, both unique, strong solutions, continuous in t (both column vectors).

A) Growth properties.

When, by (1), and, similarly,. By (2), (4), and

(57) in Appendix, Lemma A, (a consequence of Gronwall’s inequality), with, , , , , , , , we have, for some constant independent of (only dependent on), that

Let. Using (2), and (57) in Appendix, Lemma A, (with, , , , , , , , for some constant D independent of and u (only dependent on), we get

(the last inequality by (2)).

Let. As explained below, for some, we have

The inequality (25) follows from (57) in Appendix, Lemma A, together with (2), for, , , (), and

Similarly2, for,

In (25) and (26), is independent of (it only depends on).

We need to prove that

This follows from (24) and (2), because, in a shorthand notation, for all t,

We also need:

This follows from (26) and (2), because, in a shorthand notation, for, , for all t,

Note finally that, when, , then

see Remark L in Appendix.

B) Local controllability of the linear perturbations.

Note that, by (2), in a shorthand notation,

when,. Then, from (15) and (29), when,

where “co” can be omitted due to the concavity of U. To see this, apply Remark W in Appendix.

Next, let the number satisfy, (for, see (25)), and let . Let , so. When , then, hence. Then, for all, for all t

To see this, note that by (25), for all t,

Then, using again Remark W in Appendix and (31), (29), (32), for all,

C) Relations between exact and linear perturbations.

Let be given elements in. Let be arbitrarily given. Let us first prove that for small enough, for any, for, for all t,

Write,. Define, in a shorthand notation,

For any, there exists a, such that

by Lemmas B and C in Appendix.

Consider now

By (56) in Appendix, Lemma A, for, for some only dependent on, for all t

To see this, in Lemma A let, , , , , , . To obtain (34), put.

Next, given and assume that Let us prove that for any, for small enough, for any, for, we have that

Let

Now,

Because and, then Lemma J in Appendix gives that when. Next, let,. Using and, we get, which when, by (36). Hence, (37) follows.

D) Continuity of at.

Let. Define. Let us first prove that

Now, in a shorthand notation,

Using the notation in Lemma A in Appendix, we write where , ,

(, ,).

Then, by (57),

where

for

and

for

Consider e.g the term . Now, , , and when , Hence, by Remark T in Appendix, when. Similarly, the terms containing converge to zero when. So uniformly in t when.

Next, assume that,. We want to prove that then satisfies

where, in a shorthand notation,

Now,

Let us show that the four terms in square brackets have -norms when. Now, when. Hence, by Lemma J in Appendix, the second and fourth square bracket in when (recall that and). Moreover the first and third square bracket have -norms smaller than and, respectively. So the -norms of both these terms when, recall that when, see (38). So (39) has been proved.

E) Dense subsets of.

Let (for this equality see Appendix L). Note that for each, for some,. Let. For each, let be the projection of on the line containing 0 and. Let if has the opposite direction of or if, let if is longer than and has the same direction as, and if is smaller than and has the same direction as, let. Evidently, for some measurable, , and and (also depends on, the notation hides this fact). Evidently, and is progressively measurable. Now, for any, let . We have that is -dense in because if, for any, for some, (, and hence, , belong

to the linear space). Then , and so for (is convex).

Note finally that if, then, for, so, in fact

F) Final proof steps.

Define

Using Jensens inequality, for an arbitrary function we get. It follows that

For, define and . Define

Furthermore, let be the subset of consisting of all element such that and such that, (limit in -norm,). It is easily seen that elements of the type, , , precisely make up the

set. To see this, let be such a function, and, using Jensen’s inequality three times (and for any, that ), note that for any interval,

so, in particular,. This yields also, for, that

so, see (43). Moreover, by (44), similarly,

so is a -limit of. Hence, . Note also that

for all (holding “the more” for).

Finally, if, then, for

where,

and where, progressively measurable. (So the linear map satisfies.)

Let be the linear map from (see (43)) into defined by and note that,. In (45) we have just proved that has norm for the norms and (or for, as we shall express it).

Now, let, let cl(2) be closure in in -norm, let, let , and let be a -ball in .

Now, for, we have, where

Hence, by (33), using and, for, (for and, used in a moment, see definitions subsequent to (31)), we get

where

(note that is continuous in, as shown above). Let,. Then

Using Remark W, , and the fact that the first integrand has a -norm by (32) (hence the first integral has a -norm, we get from (49) that

Observe that, given, (), by for and (37), for any given, for small enough, for any,

And, by (39) and, for (), then

To obtain the conclusion in Theorem 1, we will now apply Corollary I in Appendix, and for this end, we make the following identifications:, (for see below), , , , the norm on equal to, , , , , , , , , ,. By (34) and (51) it follows that the property (61) is satisfied for for (by (40)), and (62) is trivially satisfied by concavity of U, both (61) and (62) in the manner required in Corollary I. By (50), for, for, so for, ,), where, defined subsequent to (31). By and (27) and (24), and are continuous. By (26), (28), and, is continuous in, for any, and by (38) and (52), for, is continuous at. The required boundedness of is satisfied because of (25). Evidently, is complete, see Appendix, Remark M. Hence all conditions in Corollary I are satisfied. Thus, for some, some -continuous linear functional on, , for all, .

By Hahn-Banach’s theorem, has an -continuous linear extension to all, also denoted.

Proof of Remark 1. Let. Note that (50) and gives. If, the necessary condition

(16) implies and hence , a contradiction. So.

From (16), in a shorthand notation, we get, for, and even for, that

Fix any, such that By (21) and, there exists a, such that , . So , where, , so () and.

By (25), for some constant K, , so the absolute value of the scalar products on the right hand side of (53) are smaller than (,) and

, respectively, the last number being small than . Hence, by (53), for ,. Replacing by, by the same argument, if, we get. In fact, we have for all,.

Thus, on this subspace of, is -continuous. Hence, by Hahn-Banach’s theorem (see  ) and a representation theorem, for some in, , for all (in fact on ).

Note that for, , which

means that is -continuous on and has a unique -continuous extension to the -closure of, which equals (and includes) in case of left continuity of.

Proof of Remark 2.

Let,. Let () and let and be the adjoints of and of.

It is easily proved (using Lemma A in Appendix and for) that for, for some constant D, see Remark P in Appendix. Hence, for any given, by the -continuity of on, and hence of, there exists a -function on such that for any -function, we have. The function is progressively measurable and right continuous in, both a.s. and in, see arguments in Remark Q in Appendix.

Assume now, (21) and left continuity of, which implies -continuity of on all (by extension). In this case the argument for -continuity of for in Remark Q in Appendix extends to. Let be a representation of and let. Consider now

where. In this case, we can evidently put and we have

In case of, (21), and left continuity of, by -continuity of, the two definitions of coincide, because for both definitions we have for all , t arbitrary in.

Let now be the natural filtration generated by the’s , augmented by the P-null sets in, and let.

Then, consider the pair of equations

By Theorem 2.2, p. 349 in  , there is a unique progressively measurable collection , continuous in, satisfying these equations (or more precisely, a.s. the integrated version of these equations), , , with equal to -a.s. for each t (see Remark O in Appendix). The uniqueness says that if two pairs, () and satisfy the pair of equations, then for all and for a.e.. We can let when () and obtain such functions defined on each. For, by the fact that a.s. and uniqueness, we have that for a.e a.s. Then, evidently, there exists a unique pair on satisfying (54), with a.s. equal to for any, , for all, continuous.

Let us show (19). For any, if on, so in this case (a consequence of (16)).

Define and, , and similarly. It turns out that

. From this and it follows that for all, for a.e., a.s., see Remark S. In fact, this holds for a.e. because was arbitrary. An informal proof, using Ito’s formula, shows the last equality:

So

Remark 4. (Exact attainability). In Theorem 1, drop the assumption that is optimal (and the optimization problem), so () is simply a pair satisfying

(4) and a.s. Then, for each, , (cl2 and int = interior both corresponding to

and the space), for all, for some number and some control, a.s. ,

Proof of Remark 4. Let,. Corollary G in Appendix will be applied. Let, , , the norm on Y equal to, , , , , , , , , ,. Recall that, when. By (51) it follows that the property (61) is satisfied for for, and (62) is trivially satisfied. By (27) H is continuous, by (28), is continuous in A for any, and by (52), for, is continuous at.

Assume that and let. For any, it was shown earlier that there exists a such that, (, ,), so and. Thus, if , then . Hence all conditions in Corollary G are satisfied and the conclusion in Remark 4 follows. ,

Remark 5 If it is required that instead of, where C is a -closed convex set in, and if (15) holds for replaced by

, (for see (47)), then

(16) again holds, and with. In particular, this holds if a.s, , a.s., , given numbers. In fact, because automatically, the results here cover the case where the terminal constraints are a.s, , a.s.,. ,

Proof of Remark 5. A proof can be obtained (for) by replacing by and by, with replaced by, and replaced by, keeping and as before and letting,. (Then, , , , gives , i.e. ). The details are omitted.

Remark 6 (The case for some or all j,). Let be closure in as well as in. For, let

. When, and is the

natural filtration, augmented by null sets as before, the necessary condition of Theorem 1 can be obtained for defined and continuous on (with,), if, for some, for some, for some,

a) for some, , when, , where

If, then. The condition (A), with, is implied by

b): For some, for all, for any,

Letting, , and , it is here sufficient to operate with the

-norm instead of on, with. Note that, so (A) holds for replaced by. The set arising by replacing by in is

denoted. Define. The condition (A), so modified, implies that for some, ,

close to a ball in, so (close to). See Appendix, Remark V for the next to last inclusion and the implication (B) Þ (A).

Now, in the manner required in Corollary I, (34) Þ (61) (with, ,), (24) and (26) implies continuity of and, and (38) implies continuity of at.

4. Conclusion

In this paper, necessary conditions for optimal control of diffusions with hard end restrictions on the states have been obtained. The main case considered is the one where the states restricted at the terminal time correspond to differential equations not containing Brownian motions. Brownian motions only occur in differential equations for states unconstrained at the terminal time. A removal of this restriction is discussed in Remark 6.

Acknowledgements

The author is grateful to a referee for useful comments.

Cite this paper

Seierstad, A. (2018) Necessary Conditions for Optimal Control of Diffusions with Hard Terminal State Restrictions. Open Journal of Optimization, 7, 1-40. https://doi.org/10.4236/ojop.2018.71001

Appendix

The appendix contains, among other things, a number of wellknown results, included for the convenience of the reader. The first one concerns a result on comparison of solutions. Still.

Lemma A. Assume that (an -vector) and, (a matrix, with columns,) are Lipschitz continuous in with rank and progressively measurable in. Assume that six progressively measurable functions and exist (,-matrices), satisfying

and

where. We assume that the eight integrands belong to -spaces. Then, for some constant,

(applied to matrices the index j indicates columns), and for some constant,

and only dependent on.

Proof of (57). We shall use a shorthand notation. Using the algebraic inequality, then for some positive constant k,

The Burkholder-Davis-Gundy inequality yields, for a “universal” constant

, that.

Similar inequalities hold for the other terms involving. Hence (using also Jensen’s inequality) we get

Note that, by Gronwall’s inequality, for any functions, , if , and is increasing, then. Hence, for,

Using the fact that the square root of a sum of positive numbers is £ the sum of square roots of the numbers, we get

Note that, and that . Using this for the term containing, and a similar argument for the term containing, then (57) follows.

Proof of (56).

Using Ito’s isometry,

Then, again using and Jensen’s inequality, for some positive constant k,

Thus, for,

so (56) follows.

Simple results om Gâteaux derivatives appear in the next two lemmas.

Lemma B. Let be continuously differentiable in z for each t, and have one-sided limits with respect to t, and assume for all. For each, for each “direction”, and for each t, (), has, in the norm, a bounded linear Gâteaux derivative at