On the Iterative Solution to H ∞ Control Problems

This paper addresses the problem for solving a Continuous-time Riccati equation with an indefinite sign of the quadratic term. Such an equation is closely related to the so called full information H∞ control of linear time-invariant system with external disturbance. Recently, a simultaneous policy update algorithm (SPUA) for solving H∞ control problems is proposed by Wu and Luo (Simultaneous policy update algorithms for learning the solution of linear continuous-time H∞ state feedback control, Information Sciences, 222, 472-485, 2013). However, the crucial point of their method is to find an initial point, which ensuring the convergence of the method. We will show one example where Wu and Luo’s method is not effective and it converges to an indefinite solution. Three effective methods for computing the stabilizing solution to the considered equation are investigated. Computer realizations of the presented methods are numerically compared on the computational platforms MATLAB and SCILAB.


Introduction
The continuous-time algebraic Riccati equations and their extensions have been investigated extensively in the literature.Recently, the H ∞ control problem was solved for linear time-invariant system [1]- [3] and for stochastic systems [4]- [7].
Wu and Luo [8] have commented the iterative solution of the following continuous-time algebraic Riccati equation ( ) ( ) Note that this equation has indefinite quadratic part.Assume there exists a positive semidefinite solution * X to (1) with property that real parts of eigenvalues of ( ) ( ) − are negative.Such type solution is called a stabilizing solution.
The H ∞ linear quadratic problems have been introduces by Basar and Bernhard [9] as a two-player zero sum gane.We consider a model for a a two-player zero-sum game, where the control function ( ) u t is a minimizing player (or a controller player) of the functional ( ) Knowing the stabilizing solution * X to (1) we define the following functions: ( ) And thus they form the equilibrium point of the two-player zero-sum game described by ( 2) and the functional ( ) . This fact is well known in the literature and it can be derived using the Pontryagin's Maximum Principle for example.Moreover, the stabilizing solution is very important solution to Equation (1).
So, why we need to study the iterative equations for computing the stabilizing equation to (1)?Many researchers have investigated Riccati Equation ( 1) and more specially how to compute his stabilizing solution.Lanzon et al. [1] have proposed two effective methods.The first method constructs two matrix sequences where the first sequence converges to the stabilizing solution.The second method avoids the second matrix sequence and defines one matrix sequence which directly approximates the stabilizing solution.Later, Wu and Luo [8] have studied the same equation and the proposed method in [1].They have commented that the second Lanzon's method (it is Algorithm 2 [8]) is not fully effective and by this reason they have introduced the new method described as Algorithm 4 in their paper [8].Here, we consider an example where these two algorithms will be compared.
Example 1.Let us we take the following matrix coefficients to (1) (using the MATLAB notations): [ ] 0.0665 8 0 0; 0 3.663 3.663 0; 6.86 0 13.736 13.736; 0.6 0 0 0 We execute Algorithm 2 [8] and Algorithm 4 [8] with the initial point 0 0 X = .Starting Algorithm 2 in MATLAB we obtain the following stabilizing solution to (1) after 4 main iterations and (the average number of iterations for the inner loop is 7): . The solution is indefinite and it is not the stabilizing solution.So, this example gives us the conclusion that the Algorithm 4 described in [8] compute only a solution to (1) and this solution is not always positive definite and this solution is not always stabilizing.
In this reason we confirm that the Lanzon's method [1] is an effective method for computing the stabilizing solution.His main essential feature is that the iterative process includes two iterative loops-the out loop and the inner loop.We extend the ideas described by Lanzon et al. [1] and Feng and Anderson [6] to propose iterative methods where one matrix sequence is constructed.Here we introduce additional two iterative methods which lead directly to the stabilizing solution.Our contribution is to apply two computational schemes for realization the first iterative equation.Moreover, the second iterative equation is a new method for computing the stabilizing solution to (1).We present a few examples for testing the introduced recurrence equations on the MATLAB and SCILAB computational platforms.
We write − is positive semidefinite for any two symmetric matrices X and Y.We use some properties of positive definite and positive semidefinite matrices.A matrix A is said to be asymptotically stable if all the eigenvalues of A lie in the open left half plane.

Iterative Methods for Stabilizing Solution to (1)
The first method is the Lanzon's method [1] and Algorithm 2 from [8].We present the main theorem with properties for constructing two matrix sequences of positive semidefinite matrices . The matrix sequences are constructed as follows.We take , with 0, 0,1, 2, We find ) (k Z as the stabilizing solution of the algebraic Riccati equation with definite quadratic part: where are stabilizing solutions for the sequence of algebraic Riccati Equations (4).We will prove that the second sequence is monotonically non-decreasing and converges to the unique stabilizing solution to set of Equation (1).We reformulate the convergence theorem introduced in [1] (Theorem 3) and we present it as sufficient conditions to existence the stabilizing solution to (1).
Theorem 1 Assume there exist symmetric matrices X and ≤ , and the pair ( ) ( ) 0 , A B is a stabilizable one.Then for the matrix sequences defined by (3), ( 4) are satisfied for 0,1, 2, The proof follows the proof of Theorem 3 from [1].
Theorem 1 presents sufficient conditions for the equation ( ) 0 R X = has a solution.Such type conditions are introduced here for the considered equation ( ) 0 R X = for the first time.Theorem 1 confirms the convergence properties of iterative method (3), (4).
Further on, we consider an alternative iteration process where one matrix sequence is constructed.Consider the behaviour of the controller player (u(t)).Assume the controller player knows the matrix ( ) − .He wants to find ( ) And the functional ( ) The corresponding Riccati equation regarding to ( ) .
Based on recurrence Equation ( 5) we derive the following new iteration: with 0 0 P = .We perform iteration (6) using two recurrence approaches.The first one is to solve Equation ( 6) as a Riccati equation.We call this approach "(6) + care".The second one is to solve Equation ( 6) applying the Lyapunov iteration: ( with 0 s Y = .The matrix sequence { } , 0,1, s Y s =  converges to ( ) k P .Iteration (7) defines the inner loop for iteration (6).We call the second approach "(6) + lyap".In fact, that is an extension of the Algorithm 2 [8].
The notation "(6) + care" means that the iteration ( 6) is solved as a Ricacti equation with unknown matrix 6) is computed as a solution to Riccati Equation ( 6).The notation "(6) + lyap" stands for the fact that the solution ( ) k P to Equation (6) is computed as a limit of the sequence { } s Y and each matrix s Y is a solution to iteration (7).Iteration (7) describes the inner loop for finding the matrix sequence which converges to the stabilizing solution to (1).It is easy to see that the recurrence Equation ( 6) is obtained from ( 4) when we substitute ( ) Next, we extricate the term and continue with some matrix manipulations.We derive We apply the following implementation for the latest recurrence equation: Our thoughts and algebraic manipulations for deriving recurrence Equation (8) show that it is equivalent to the main iterative process (3)-( 4).Thus iteration (9) constructs a new matrix sequence which converges to the stabilizing solution of (1).In order to execute iteration (9) we apply the following algorithm: 1) We take ( ) 0 0 X = , and ε as a small positive number.
2) We compute as a solution to the equation ) as a solution to the Lyapunov equation c) Algorithm stops when the inequality 4) The stabilizing solution is ( )

Numerical Experiments
We carry out experiments for solving a continuous-time algebraic Riccati equation with an indefinite quadratic term (1).We construct two matrix sequences is its stabilizing solution.In addition, we apply two iterations ( 6) and ( 9) for computing the stabilizing solution to (1), where one matrix sequence is established.We perform iteration (6) in two ways "(6) + care" and "(6) + lyap".We are solving Riccati recurrence Equations ( 4) and ( 6) with the MATLAB procedure care where the flops are 3 81n per one iteration.The MATLAB procedure lyab is applied for solving (7) and ( 9) and the flops are Error tolR ≤  is satisfied for some 0 k .That is a practical stopping criterion.However, iteration "(6) + lyap" defines two loops- external and inner.The inner loop stops when the inequality Error tolR ≤  holds and the average number avIt of iterations for all hundred runs of each size.In addition, the variable L avIt stands for the average number of iterations executed by (7) in order to obtain the stabilizing solution through ( 6) for all hundred runs of each size.For instance, the iteration "(6) + lyap" executes 4 It = main iterations and 7 L avIt = for Example 1.The column "CPU" presents the CPU time for execution the corresponding iterations.In our definitions the functions randn (p, k) and sprand (q, m, 0.3) return a p-by-k matrix of pseudorandom scalar values and a q-by-m sparse matrix respectively (for more information see the MATLAB description).
Example 2. We consider a family of examples in case 10, ,15 n =  , where the coefficient real matrices are given as follows: , , , A G B D and C were constructed using the MATLAB notations:   The application of all iterative methods shows that they achieve the same accuracy for different number of iterations.Our conclusions based on experiments are: 1) The execution the iterations (3), (4) and "(6) + care" takes almost the same CPU time (see the corresponding SCILAB implementations achieve the same accuracy and based on the fact it is an open source software we deduce the SCILAB is an useful tool for education to master and PhD students.The conclusions are indicated by implemented numerical simulations.

Conclusion
We have studied two iterative processes for finding the stabilizing solution to generalized Riccati Equations (2).We have made numerical experiments for computing this solution and we have compared the considered methods numerically.We have compared the results from the experiments in regard of number of iterations and CPU time for executing.Our numerical experiments confirm the effectiveness of proposed new method (9).It is introduced here and moreover numerical experiments show its efficiency.

γ
and the disturbance function ( ) v t is a maximizing player (or a disturbance player), where of the system: (6) and it is a Lyapunov iteration for computing ( ) k P .Thus, Equation (6) can be considered as a new iteration formula.This equation constructs a new matrix sequence for each example.The first matrix sequence is computed using the iterative process (3)-(4).Iteration (4) is a Riccati equation and ( ) k Z have carried out experiments in the open source software SCILAB http://www.scilab.org/scilab/about.It provides a computing environment for scientific applications.There are functions for solving linear and nonlinear matrix equations.We apply the "ricc" function for solving a continuous Riccati equation and "lyap" function for a linear Lyapunov equation.Our experiments are executed in MATLAB on a 2.20 GHz Intel (R) Core (TM) i7-4702MQ CPU computer.We use two variables tolR and tol for small positive numbers to control the accuracy of computations.We denote


is satisfied for some integer 0 s .For our purpose we have executed hundred runs of each value of n for two family of examples.The tables report the maximal number It of iterations for which the inequality , It Results from experiments in Example 2 are given in for all values of n.

Table 1
We consider a family of examples in case 15, , 20 n = , where the coefficient real matrices are given as follows: , , , A G B D and C were constructed using the MATLAB notations: