Infinite Horizon LQ Zero-Sum Stochastic Differential Games with Markovian Jumps

This paper studies a class of continuous-time two person zero-sum stochastic differential games characterized by linear Itô’s differential equation with state-dependent noise and Markovian parameter jumps. Under the assumption of stochastic stabilizability, necessary and sufficient condition for the existence of the optimal control strategies is presented by means of a system of coupled algebraic Riccati equations via using the stochastic optimal control theory. Furthermore, the stochastic H∞ control problem for stochastic systems with Markovian jumps is discussed as an immediate application, and meanwhile, an illustrative example is presented.


Introduction
The stochastic control problems governed by Itô's differential equation have become a popular research topic in past decades.Recently, stochastic H ∞ control problem with state and control-dependent noise was considered [1,2].It has attracted much attention and has been widely applied to various fields.Particularly, the stochastic H 2 /H ∞ control with state-dependent noise has been addressed [3,4].Recently, linear quadratic differential games and their applications have been widely investigated in many literatures, and examples of differential games in economics and management science can be found e.g. in [5][6][7][8][9].These results are mainly based on the deterministic systems.However, to the best of our knowledge, few results have been obtained for stochastic differential games with Markovian jumps.
In this paper, the stochastic zero-sum games for linear quadratic systems governed by Itô's differential equations with state-dependent noise and Markovian jumps are addressed, Such class of systems has important applications in engineering practice since they can be used to represent random failure processes in manufacturing systems, electric power systems and so on, see [10][11][12][13][14][15][16][17][18].In particular, stability and robust stabilization for such perturbed systems were investigated extensively in [13,15,17].A bounded real lemma for Markovian jump stochastic systems was derived in [13].[11] studied the optimal filtering problem for such systems, while [12,14,16] addressed the issue of linear quadratic regulator.The goal of this paper is to develop the differential game theory for stochastic Itô systems with Markovian jumps, a necessary and sufficient condition is developed for the existence of optimal control strategies in terms of a coupled algebraic Riccati equations (AREs), which can be viewed as an extension of the existing results of [19].In the end, stochastic H ∞ control problem with Markovian jumps is given as our theoretical applications and an illustrative example is presented.
For convenience, we will make use of the following notations in this paper: A T : transpose of a matrix or vector A; A −1 : inverse of a matrix or vector A; A > 0 (A ≥ 0): positive definite (positive semidefinite) symmetric matrix A; χ A : indicator function of a set A; : space of all , ,n ; n : space of all n × n symmetric matrices; : space of all

Definitions and Preliminaries
Throughout this paper, let r  be a given filtered probability space where there exists a standard one dimensional Wiener process , and a right continuous homogeneous Markov chain with state space .We assume that is independent of and has the following transition probability: where for and i j  . F t stands for the smallest σ-algebra generated by process denote the space of all measureable functions , which is F t -measurable for every , and Consider the following linear stochastic controlled system with Markovian jumps where and are the state and control input, respectively.The coefficients It is well known that for any and , there exists a unique solution of (1) with initial condition . Next, we first introduce the definition of stochastic stabilizability which is an essential assumption in this paper.
Definition 1. System (2) or (A, B, C) is called stochastic stabilizable (in mean-square sense), if there exists a Now we give two lemmas which are important in our subsequent analysis.For system (2), by applying Itô's formula to   T x P i x , we immediately obtain the following result.

Problem Formulation
 be the set of the -valued, square integrable processes adapted with the σ-field generated by x t is the solution to the following linear stochastic differential equation with statedependent noise and Markovian parameter jumps In ( 5) and ( 6),   t i A r A  , etc. whenever t r i  .Now we consider the following zero-sum differential game problem.
Problem 1.Given a system described by ( 6), find Max Min , ; , Min Max , ; , , ; , , That is, there are two players for the differential game.Player 1 chooses control to minimize the objective J, while Player 2 chooses control to maximize J. Now we introduce a new type of coupled algebraic Riccati equations associated with the problem 1. with is called a system of coupled algebraic Riccati equations (AREs).
In the next section, we will give our main results of this paper.

Main Results
In this section, we will show that the solvability of the AREs ( 7) is sufficient and necessary for the existence of the optimal control strategies of problem 1.
, , , A B B C is stochastic stabilizable, problem 1 has a pair of solutions with respect to the initial     0 0, 0, x n    , where and are the following feedback strategies respectively, iff the AREs (7) admits a solution  be a solution of the AREs (7).According to lemma 1, we have , ; , By a series of simple computation together with (7), the cost function   1 2 0 , ; , J u u x i can be expressed as following is minimized by the control strategies and with the optimal value being be the optimal control strategies to problem 1. Implement and in (6), then


According to lemma 2 and the stochastic optimal cotrol theory, we can easily obtain the conclusion that the A n REs (7) admit a solution So this completes the proo s f of Theorem 1. Remark 1.It is interesting to ee the specialization of our results in the deterministic case (i.e.  . which can be viewed as an extended results of [12].Remark 2. From Theorem 1 we can see that the derie of di ay be solved by a standard numerical integration su ∞ Control olve irstly, vation of the optimal control strategies for this typ fferential games is transformed into deriving the solutions to coupled algebraic Riccati Equations ( 7), this conclusion be coincident with the results presented in [4], etc.
Remark 3.For the coupled algebraic Riccati Equations ( 7) m ch as LMI method [20], or iterative algorithm similar with the algorithm presented in [21].

Application to Stochastic H
Now, we apply the above developed theory to s some problems related to stochastic H ∞ control.F we statement the stochastic H ∞ control problem with Markovian jumps, then, we demonstrate the usefulness of the above developed theory in the study of stochastic H ∞ control.
Consider the following controlled system: with the cost functional where is a right continuous Markov process give ility space i.e. when ally, , the state trajectory of (9) with any in value  .rol pr Generally speaking, the H ∞ cont oblem described d ( 10) is to fi trol such that by (9) an nd a con As stated in [23], if we view   u t and in the stocontrol problem as two control strategi f players P 1 and P 2 from the viewpoint of theory, the H ∞ control problem can be converted into solving a stochastic game problem, while   * * , u v is in fact the saddle point of this game, e.g.
According to Theorem 1 discussed in S ction 4, the following results can be obtained straightly:  with e Theorem 2. For system described by (9), the stochastic   , solving (11) via using the algorithm proposed in [21], we have Therefore, the H control is given by while ;     stochastic differen and Markovian jum se rizon, sufficient and necessary conditions for the existence of the optimal control strategies have been obtained, which are expressed in a system of coupled algebraic Riccati equations.The results obtained in this paper extend the existing results of [19].Throughout this paper, we only have focused on the zero-sum LQ differential games for stochastic systems, while we believe that nonzero-sum LQ differential games still have essential applications, and further studies on such kind of case should be continued.

2 u 2 .
DefinitionThe following system of algebraic equations  is the vector of the exogenous di s.The following de parallel with the definition in[22].

Definition 3 .
Given disturbance attenuation level γ > 0, the state feedback str

v
Illustrative example: Consider system (9) with the coficients as follows: