1. Introduction
The Matrix Riccati equation named after the mathematician Jacopo Francesco Riccati [1] , can be written as
(1)
where
,
,
,
and
.
The matrix Riccati equation arises in many branches of applied mathematics [2] including optimal control [3] , theory of stabilization, transport theory, quantum mechanics [4] [5] , physics, filtration of control systems, differential game [6] [7] , financial mathematics [8] , random processes, diffusion problems, non-unifornm transmission line and stochastic control. The Riccati equation can serve as an unifying link between linear quantum mechanics and other fields in physics, such as thermodynamics and cosmology, and new uses keep popping up as we discover new applications. Thus there is a need to extend previous findings to more complicated applications.
The scalar Riccati equation [9] defined as
is a particular case of the Matrix Riccati equation.
It is well-known that the change of variable
leads to the second
order linear differential equation [10] :
If
and
are constant functions then the scalar Riccati
equation can be solved analytically.
We can use the scalar Riccati equation as a blueprint for how we should move forward to solve analytically the matrix Riccati equation. I will be looking at adapting the method above to matrix Riccati equation which required some change to accommodate the matrix properties.
Control system and optimal control [3] [11] [12] is a fairly new field of mathematics, starting in the 1950s.
In a nutshell, Optimal Control extends the maximization/minimization process we learn in calculus to functions, or functionals. Where Calculus gave us a method of finding points
that maximize or minimize some function
, optimal control theory deals with way of finding a control for a dynamical system over a period of time such that an objective function is optimized.
A terrible but helpful metaphor is that if we have an equation that approximates the shape of a mountain, Calculus methods can tell us the location of its highest peak and deepest crag where Optimal Control tells us which ridge to hide in order to get to the peak the fastest and by expanding the least energy.
Matrix Riccati equations can be used to solve some optimal control problem, for instance the Linear Quadratic Regulator (LQR) problem. Recall that the general analytical solutions for the matrix Riccati equation and the algebraic matrix Riccati equation are not available. Only special cases are treated particularly for the scalar Riccati equation. This explains why it is very hard for the (LQR) to find explicit formula for the controls and then approximations of the controls are more frequent which leads to errors. The particular case where the solution of the matrix Riccati equation approaches a constant has been studied [13] . In this case, we deal with the steady-state version referred to as the algebraic Riccati equation. Analytical solutions for algebraic Riccati equation are also intractable. In this paper, a change of variable is proposed in order to turn the matrix Riccati equation into a second order linear matrix equation. This change of variable was mainly inspired by some of the work done with the scalar Riccati equation. This will allow to find exact values for the controls.
This article is divided into six chapters.
In Chapter 2, a change of variable to the Matrix Riccati equation is proposed that turns it to a second order linear matrix differential equation.
In Chapter 3, we look at the field of optimal control which is a branch of mathematics that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized.
In Chapter 4, we use the results from the two previous chapters to solve analytically an optimal control example.
Chapter 5 is dedicated to the conclusion.
2. A Solution to the Matrix Riccati Equation
Consider the matrix Riccati equation
where
,
,
,
and
.
For the sake of brevity, we will assume that
so that
and
.
Proposition 1. If A is invertible, then
(2)
where
represents the derivative with respect to t that is
.
Proof. Since
where I is the identity matrix
therefore
.
,
Theorem 1. If
and if A is invertible, then the matrix Riccati Equation (1) can be turned into the second order matrix linear differential equation.
(3)
using the change of variable
(4)
where U is invertible.
Proof.
Using (2) on U,
Since
then
Using (2) on A and after simplification, we obtain
,
Example 1. Choosing
,
,
,
After plugging into the equation, this gives us:
After solving for U and using the change of variable (4) we obtain
where
are constants given by the initial conditions.
The next theorem generalizes theorem 1.
Theorem 2. If
for all
and
then (2) can be turned into the equation:
(5)
The proof of theorem 2 is very similar to the proof of theorem 1.
3. Optimal Control Theory
The field of Optimal Control, as the name suggests, is a branch of mathematics that deals with analyzing a system to find solutions that cause it to behave optimally for the cost we are willing to pay. If a system is controllable given an initial state and some assumptions, then we can reach a desired state of the system by finding the appropriate control with minimum cost. Control is handled through a feedback into u that depends on the state of the system. The basic optimal control problem can be stated as follows: Given the system of differential equations along with an initial condition,
(6)
where
is the state of the system,
, and
is the control.
The goal is to find a control
over
which for any
, minimizes the cost function
To better frame the optimal control problem, let’s consider a simple example.
Example 2. Consider the circuit below that consists of resistor, inductor and a source (RL circuit).
The circuit in Figure 1 is very frequent in electronic device for filtering signals.
The behavior of the resistor is specified by a constant R called resistance.
The behavior of the inductor is specified by a constant L called inductance.
i is the current across drop the circuit. u is the control and represents the voltage across the source.
According to the Kirchoff’s law of current, the sum of the voltage drop across the circuit in Figure 1 equals the voltage across the source, therefore,
Let
be the initial value of the current that is
.
We deal with the system:
Suppose that we want to switch the current
from
to another value
at
that is
with a minimum cost.
The goal is expressed by the cost functional.
where c and
are positive constants and T is the fixed final time
.
The first integral is the state cost and the second integral is the control cost.
We also can assume that that u belongs to a set of admissible control
where
and
are constant real numbers.
Question: What values of
allow this switch with minimum cost?
General Statement of the Optimal Control Problem: The Pontryagin Principle
The basic optimal control problem
can be stated as follow: Given the system of differential equations along with an initial condition:
(7)
where
is the state of the system
and
is the input of the system
.
The goal is to find a control
over
that minimizes the cost functional
To solve an optimal control problem, we can use the Pontryagin principle which represents some necessary conditions the optimal control
and the optimal state
need to satisfy.
Theorem 3 (Pontryagin’s maximum principle) If
and
are
optimal for the problem (P), then there exist a function
and a
function H defined as:
that satisfy the three properties.
a)
for all control u at each time t.
b)
c)
represents the transpose of
and H is called the Hamiltonian.
The Pontryagin’s maximum principle yields to the controls that represent the candidates for the optimal controls. Those candidates need to be tested. The following theorem gives a sufficient condition for a candidate to be optimal.
Theorem 4. Let
be the set of admissible controls of u and X an open subset of
. If there exists a function
of class
such that the three statements below are true.
i) If
generates the solution
of (7) and
for all
, then
, for some
.
ii)
for all
for some
.
iii)
for all
and
.
Then the control
generating the solution
for all
with
, is optimal with respect to X.
The proof of the theorem 3 and 4 can be found in [3] .
Remark 1. The proof of theorem 4 suggests that the test function
can be chosen so that:
.
Remark 2. In case we deal with a nonautonomous system that is a system in the form
then we always can turn such system into an autonomous system.
We can define
where
then we deal with the following autonomous system
Also if the cost integrand depends on t that is
then we can rewrite the cost function: as
Now we can refine the maximum’s Pontryagin principle to the autonomous system
with cost function given by:
and then
.
4. The Linear-Quadratic Regulator (LQR)-Riccati Equation
We suppose that
then
then
(8)
with initial state
and the interval
is specified and
The cost to be minimized is:
where
,
,
.
The matrix Q is symmetric that is
.
The matrix R is symmetric and positive definite that is
if
.
The functions
and
are of class
.
We can use the Pontryagin minimum principle to find
.
The Hamiltinian H of the problem is given by:
(9)
(10)
The adjoint equations are:
which implies
Since Q and R are symmetric then
therefore
Moreover, since there is non-constraint on u, therefore
(11)
so from (10)
(12)
therefore
(13)
The goal is to express
in term of
. Let’s replace
in the system of Equation (8), we obtain
(14)
therefore, we get the following system with 2n variables
(15)
that has a unique solution
given an initial condition.
Using the matrix representation,
(16)
where
(17)
therefore
where
In particular
for all
.
Dividing
into blocks of
matrices
(18)
where
are
matrices
.
Therefore
Since
is unique then
must be invertible, therefore
Let
.
So
(19)
From (13),
.
Finding P(t)
Taking the derivative in both sides of the Equation (19), and using (14) we obtain
(20)
Using (15) and (20), we get the following equation:
(21)
This equation holds for all
.
So
is a solution of solution of the Matrix Riccati equation:
(22)
satisfying the initial condition
.
Theorem 5. The function
is the optimal solution at
and the minimum value of J is given by:
where
is the corresponding optimal solution of (7).
Proof. First notice that
is symmetric that is
. Let’s take the transpose in both side of the Equation (22), we obtain:
Since R and Q are symmetric then
which shows that
is also a solution of (22) satisfying
.
Since the solution of (22) along with initial condition
, is unique therefore
.
Now to show that
, we first can show that
Since P is symmetric then we can easily verify that
.
Therefore
Using (15), (19) and (22)
Since
then
.
After cancellation,
Since
then
Taking the integral from
to
in both side and multiplying by
, we
obtain
(23)
which shows that
.
To show that
is the optimal solution, we can verify the three conditions of the theorem 4.
From (23), we can choose a test function defined in
as
where
.
We can show that the test function satisfies the three conditions in theorem 4.
Since
then
then condition (i) is satisfied.
For (ii), let
so
Using (22)
After simplification
The gradient of
,
so
then
therefore
is a critical value for
.
The Hessian
which is positive definite, therefore
has a global minimum at
. It can be shown easily that
. This shows (ii).
Since
has a global minimum at
then
for all u and x therefore
this shows (iii).
,
5. Example
Consider the optimal control problem:
where
Find u that minimizes
.
is a solution of the Riccati equation:
Let’s make the change
. So
(24)
then
The solution of the equation is given by:
(25)
where
and
where
since
that is
then we conclude that
(26)
(27)
Combining the entries in the diagonal of
, we obtain
(28)
and
(29)
Since we have four equations with eight variables, we can choose
as the free variables and solve for
.
From (25) and (26),
From (27),
Since
.
From (28),
Therefore
where
or
The final expression is given by:
where I is the 2 × 2 identity.
This leads to two controls that satisfy the problem (P)
where
or
.
6. Conclusions
In this paper, we apply the results we obtain from the Matrix Riccati Equation to optimal control. We provide an explicit control for a specific example in control optimal, the Linear-Quadratic-Regulator. Notice that more complicated examples could be considered.
One of the extensions of these results in this paper would be to apply them in other branches of mathematics for instance non-uniform transmission line, stochastic control and mathematical finance.