Contribution in Information Signal Processing for Solving State Space Nonlinear Estimation Problems

In this paper, comprehensive methods to apply several formulations of nonlinear estimators to integrated navigation problems are considered and developed. The problem of linear and nonlinear filters such as Kalman Filter (KF) and Extended Kalman Filter (EKF) is stated. Analog solution which is based on fisher information matrix propagation for linear and nonlinear filtering is also developed. Additionally, the idea of iterations is included through the update step both for Kalman filters and Information filters in order to improve accuracy. Through this development, two new formulations of High order Kalman filters and High order Information filters are presented. Finally, in order to compare these different nonlinear filters, special applications are analyzed by using the proposed techniques to estimate two well-known mathematical state space models, which are based on nonlinear time series used to apply these estimation algorithms. A criterion used for comparison is the root mean square error RMSE and several simulations under specific conditions are illustrated.


Introduction
Different kinds of filters exist and were developed in order to ensure high quality measurement in input-output systems and permit more accurate control system in several fields such as in Aerospace, for aircraft's navigation, ship, spacecraft, tracking etc. Kalman filter (KF) was firstly derived from using orthogonality principle and presented in [1][2][3].Generally so-called filter or/and estimator, is/are one of several techniques of estimation based on LMMSE (Linear Minimum Mean Square Error) [4].In 1970, Kalman and Bucy introduced extended Kalman filter for nonlinear estimation.Actually this kind of filter is called standard local filter and is based on approximation of nonlinear functions by Taylor series.The most common filter in the field of engineering and aerospace especially, is the extended Kalman filter.These local standard filters also contain the second order Kalman filter and the iterated filter.Other kinds of nonlinear filtering algorithms exist but are not treated in this paper [5].The most interesting and main idea introduced in this paper is to use the parallel solutions to Kalman filter and standard local filters which are based on the Fisher Information Matrix propagation [6,7].It is analog to Kalman technique but is more efficient and robust to several constraints.The main idea is to use the inverse matrix lemma to develop analog estimator to Kalman filter with less computational time.These filters are more efficient when the number of the input is more than the dimension of the state vector.In this paper, we introduced classic information filters for linear and nonlinear filtering problem.Behind this was explored in our solution, the efficiency includes iterations through the updated step of the different algorithms [8].Two new formulations are presented such as the iterated second order Kalman filter and the second order information filter followed by the second order Iterated Information filter.So, generally the step of initialization is the main important in nonlinear filtering and we propose to use the information filters to solve the problem of initialization.Of course, information filters present several other advantages when the state space model input is a combination of several sensors as in data fusion or multi-sensors fusion, it was proven that comparing with Kalman filter and extended Kalman filter both for linear and nonlinear case, the information filters are more easy to implement in real time application with multiple information combination [9][10][11][12].We apply these different nonlinear filters to dynamical state models such as references.It is expected that this work could serve in investigate integrated navigation system INS (Inertial navigation System)/GNSS (Global Navigation by Satellite System) problems, in order to show possible application in the field of aerospace.

Kalman Filter and Nonlinear Filtering
If the system is linear and the statistical distribution is Gaussian, then the Bayesian prediction and update equation can be solved analytically.The system is completely described by the Gaussian parameters such as mean and covariance and this filter is called the Kalman filter [13].As a discrete statistical recursive algorithm, Kalman filter provides an estimate of the state at time k given all observations up to time k and provides an optimal minimal mean squared error estimate of these states.
Process Model: A linear dynamic system in discrete time can be described by Kalman filter is usually called as the optimal filter in the case of linear assumption and white Gaussian noises both in state and in measurement equations.

Extended Kalman Filter
In most real applications the process and/or observation models are nonlinear and hence linear Kalman filter algorithm described above cannot be directly applied.To overcome this, a linearised Kalman filter or Extended Kalman Filter (EKF) can be applied which are estimators where the models are continuously linearized before applying the estimation techniques [14].However, in most practical navigation applications, nominal trajectory does not exist beforehand.The solution is to use the current estimated state from the filter at each time step k as the linearization reference from which the estimation procedure can proceed.Such algorithm is called extended Kalman filter.If the filter operates properly, the linearization error around the estimated solution can be maintained at a reasonably small value [15][16][17].However, if the filter is ill-conditioned due to modeling errors, incorrect tuning of the covariance matrices, or initialization error, then the estimation error will affect the linearization error which in turn will affect the estimation process and is known as filter divergence.For this reason the EKF requires greater care in modeling and tuning than the linear Kalman filter.Let us describe bellow the algorithm of EKF [18]: based on state space model as: and on the linearization using taylor approximation at the first order we get the state space model given in [19].
  k F  is the Jacobian matrix of and Update: The meaning of the extended Kalman filter can be understudied by appreciating the same equation of gain calculation as in the Kalman filter at the difference that in the nonlinear filtering, EKF is sub-optimal filter.

Iterated Filter
One can distinguish the importance of the two different steps; prediction and update, it allows to observe the effect of new information given by the measurements in the filtering step.Let us focus on the estimation of the mean and the covariance of the state vector.In Equation (5) it is clear that ˆk x contains more information about k x than ˆk x .Nonetheless, the linearization was made in ˆk x .This fact is used and the linearization can be made in the kth step again but this time in ˆk x .This provides a new value of estimates and such a procedure may be repeated as long as a difference between two subsequent estimates is lower than a specified  .Thus, the following equations will be implemented using the iterated form: 1  ˆk The iteration is stopped if and the value i + 1, i.e. the time instant of iteration, is denoted as i max .It is not possible to use the same iteration formulation for the prediction step because the prediction utilizes no new information from reality.So, the relation of prediction step is the same as in the extended Kalman filter.So, where and . max All the previous relations define the iterated filter which is an improvement of the extended kalman filter and improves local approximation for filtering estimate calculation.On the other hand, it is again local approximation and convergence of the estimate is not guaranteed as well.

2 nd Order Kalman Filter
In this part, another alternative of the extended Kalman filter is presented and will be used in simulations.Based on the Taylor series but used to the second term, let us consider the following approximations: where ˆk x , correspond to the approximation used for the extended Kalman filter.
The dimension of the vector function   k h  is z n and the dimension of the state vector k x is x n .Then, the new approximation can be written such as described by: Because of the second order terms in (10), analytical computation of the filtering step is not possible, so, we can solve this problem by replacing the quadratic form by its mean and we obtain then: Let us write: and then, we obtain also: From this result, we can learn that it is a linear function of k x , as all the remaining terms are known in the (k−1)th step.Thus we obtain the following integration equations: It is possible to observe that the innovation sequence is different from the one of the extended Kalman filter.Now, to compute the prediction step, it is possible repeat the same steps as for the update formulation and we obtain as bellow: By computing means of the nonlinearities, it is possible to write: Finally, we get: and we obtain: The equation of the Covariance integration is the same as in the extended Kalman filter, with different prediction step adding ak f .So, the Equations ( 16) and ( 17) represent the second order filter.After describing the different nonlinear approximation used usually in nonlinear filtering , let us pass to the second kind of filters which are the information filters both for linear and nonlinear case, let us describe the information filter [20] and the extended information filter [21,22], then novel formulations will be developed.

Information Filter and Nonlinear Information Filters
The information filter is mathematically equivalent to the Kalman filter except that it is expressed in terms of measures of information about the states of interest rather than the direct state and its covariance estimates.Indeed, the information filter is known to have a dual relationship with the Kalman filter.If the system is linear with an assumption of Gaussian probability density distributions, the information matrix   When an observation occurs, the information state contribution i(k) and its associated information matrix I(k) are given by the following expressions: By using these variables, the information prediction and update equation can be derived from Kalman filter.
Prediction: The predicted information state is obtained by pre-multiplying the information matrix in Equation ( 22) and by representing it in information space, where the information propagation coefficient matrix (or The corresponding information matrix is obtained by taking the inverse of Equation ( 18) and by representing it in information space, Estimation: The update procedure is simpler in the information filter than in the Kalman filter.The observation update is performed by adding the information contribution from the observation to the information state vector and its matrix: If there is more than one observation at time k, the information update is simply the sum of each information contribution to the state vector and matrix, where n is the total number of synchronous observations at time k.Note: As the information matrix is defined like the inverse of the covariance matrix, the information filter deals with the "certainty" rather than "uncertainty" such as in Kalman filter.Furthermore, given the same number of states, process and observation models, the computational complexity of the information filter and the Kal-man filter are comparable.The update stage in the information filter is quite simple however the prediction stage is comparatively complex, which is exactly opposite in the Kalman filter.
However both filters can show different computational complexity depending on the dimension of the state and observations.If the number of observations increases, as in the case of the multi-sensor systems, the dimension of the innovation matrix of the Kalman filter increases as well, and the inversion of this matrix becomes computationally expensive.In the information filter, however, the information matrix has the same dimension of the state and its inversion is independent to the size of observations.This means that the information filter is an efficient algorithm when the dimension of observations is much greater than that of the state, thus, they are more suitable in complex data fusion problems based on multiple sensors.
In addition, the information filter can perform a synchronous update from multiple observations in contrast to the Kalman filter.The reason is that the innovations in the Kalman filter are correlated to the common underlying state while the observation contributions in the information filter are not.This makes the information filter attractive in decentralizing the filter.Finally, the information filter can easily be initialized to zero information.

Extended Information Filter
The extended information filter can also be derived for the nonlinear process/observation model defined in Equations [22].
Prediction: The predicted information vector and its information matrix are obtained by using the Jacobians of the nonlinear process model Estimation: When an observation occurs, the information contribution and its corresponding matrix are: where the innovation vector is also computed as in the EKF These information contributions are again added to the information state vector and matrix as in the linear information filter In practice, the EKF and EIF are considered as the most useful filters.

Contribution in Information Nonlinear Filtering
In this section, new filters based on the presented techniques are introduced, using iterations to improve the second Order Kalman filter and the extended Information filter, these filters were called: Iterated 2 nd Order Kalman Filter and Iterated Extended Information filter, we applied these two new formulation in simulations and it is expected to have a good results.The main is to proof that the iteration can be also applied to High order Kalman filter and can improve the accuracy of the Extended information filter.Of course, the computational time will increase instead of more accuracy.
The second contribution in his paper is to extend information filter to the second order based on the 2 nd order Kalman filter and to apply also the iterations through the update of the new filter in order to improve its efficiency.These algorithms are called 2 nd Order Information filter and Iterated 2 nd Order Information filter.Let us begin by describe the Iterated 2 nd order Kalman filter , the iterated Extended Information filter, the 2 nd order Information filter and finally, the Iterated 2 nd order Information filter.

Iterated 2 nd Order Kalman Filter
The iterated filter from the previous section represents a way to improve the point of linearization of the nonlinear function and the second derivation of this function.In this part, another alternative of the extended Kalman filter is presented and will be used in simulations.Based on the Taylor series but used to the second term, let us consider the approximations given in the Equation (9).Where x is x n .The same assumption such as made in the 2 nd order Kalman filter is considered at expect of introducing the iterations through the update step of the algorithm.So, the same idea of the iterated filter will be applied again.
Thus we obtain the integration equations given by Equation (15).It is proposed to transform this step by introduce iterations till the error between subsequent estimates will be less then specified error minimum limit.
So, the new update is given bellow: 1 ˆk k X X  and .
Then stop the iterations, else, continue, end.So, one can observe that the update equations are the same such as in the iterated Kalman filter according to the covariance integration but is different in state estimation due to the correction term in the innovation.Now, to compute the prediction step, it is possible to repeat the same steps as for the update formulation and we obtain the following equation: By following exactly the same steps such as in the 2 nd order Kalman filter; we finally obtain: where and .max i i k k P P  The equations of the state and Covariance integration are the same such as given in the 2 nd order Kalman filter.So, the Equations (37) and (38) represent the second order filter.

Iterated Extended Information Filter
The extended information filter can also be derived from the nonlinear process/observation model equations.
Prediction: The predicted information vector and its information matrix are obtained by computing the Jacobian of the nonlinear process model given in the Equation (2).
Estimation: When an observation occurs, the information contribution and its corresponding matrix are written such as below: where the innovation vector is also computed like in EKF These information contributions are again added to the information state vector and matrix such as in linear information filtering: where 0   , End.

2 nd Order Information Filter
Again, the same technique used in the extended information filter is used, based this time on the second order approximation of the state and the covariance using the corrected innovation.Prediction: The predicted information vector and its information matrix are obtained by computing the Jacobians of the nonlinear process model and the corrected form of the predict state in the second Kalman filter: where ak f is the second order term used to correct the predict state calculated in the previous section according the 2 nd Order Kalman filter.
Estimation: When an observation occurs, the information contribution and its corresponding matrix are: where the innovation vector is also computed as in the 2 nd order KF It is possible to observe that the innovation sequence is the same such as in the 2 nd KF but is more accurate than in the Extended Information Filter (EIF).These information contributions are again added to the information state vector and information matrix: Let us now consider the 2 nd Order Information Filter and compare with the 2 nd Order Kalman filter through simulations in the last section.

Iterated 2 nd Order Information Filter
The same philosophy such as in the previous section is used.Iterations are introduced through the update step, in order to increase accuracy of linearization and the second derivation.
Prediction: The predicted information vector and its information matrix are obtained by using the Jacobians of the nonlinear process model and the corrected form of the predict state in the second Kalman filter: where ak f is the second order term used to correct the predict state calculated in the previous section according the 2 nd Order Kalman filter.
Estimation: When an observation occurs, the information contribution and its corresponding matrix are: where the innovation vector is also computed such as in the EKF These information contributions are again added to the information state vector and matrix as in the linear information filter where 0   , End.
After these modifications, it is expected to obtain more wide kind of information filters useful for nonlinear filtering problems and especially under critical conditions.Let us pass to the simulations based on two series; used widely in the field of filtering in order to compare the different algorithms of estimation.Finally all the filters presented in this paper are expected and planned to be applied to integrated navigation system INS/GNSS usu-ally based on nonlinear filtering techniques [23][24][25][26][27][28].

Simulations
The simulations are divided in three parts; the first gives an example with low nonlinearity, only in the measurement equation.The second example shows the effects of the high nonlinearity present both in state and in measurement using much known time series equation very useful in the field of filtering.The third part of simulation is about applying such proposed methods to real problems in navigation using different input "observations" in order to compare both of accuracy and computational time of each algorithm.So, several examples are presented, which illustrate the operation of the improved information filters comparing with the classic solutions.

Consider the Following Set of Equations
Such as an Illustrative Example All the difference between these filters can be seen between 0 and 30 because of the nonlinear measurement equation in this interval of time.On Figure 3, it is easy to observe again that in the case of high noise level, the information filters are more efficient and more accurate than the classic approximated nonlinear filters based on Kalman filter.

Consider the Following Set of Equations Such as This Illustrative Example
On Figures 4(a) and (b), when the noises are low level, we can apply more nonlinear approximated filters based on Kalman filter than the information filters.One of the most known applications in aerospace and navigation problems are connected with integrated navigations systems and data fusion.This combines between different output of several sensors in order to estimate one or more state variables according to state space model including process and measurement stochastic differential equations.information filters especially for high level noise which affects both of state and measurement.The observation is that low noise level; the classic algorithms perform the information estimators.In other way, the main advantage of Information based new formulations is that for zero information initialization of the several filers, the algorithms based on the information propagation are better, more quickly and more accurately than the approximated nonlinear filters, which have real consequences in real time application, for example, in navigation, tracking and multi-sensors data fusion.Really, through simulations based on two well-known mathematical state space models, it is possible to appreciate the difference between the classic formulations of the nonlinear filters such as EKF, EIF, 2 nd OK compared with the proposed information filters.It is observed then, more accurate estimate is due

Figure 1 .
Figure 1.(a) State, (b) MSE, illustration for the first system with high noise level.yr(1) = y(1); Q(1) = 100; R(1) = 10; Xest(1) = 0.0.x(1);P(1) = 100; Iterations number: i max =1000.On Figure3, it is easy to observe again that in the case of high noise level, the information filters are more efficient and more accurate than the classic approximated nonlinear filters based on Kalman filter.On Figures4(a) and (b), when the noises are low level, we can apply more nonlinear approximated filters based on Kalman filter than the information filters.One of the most known applications in aerospace and navigation problems are connected with integrated navigations systems and data fusion.This combines between different output of several sensors in order to estimate one or more state variables according to state space model including process and measurement stochastic differential equations.

Figure 2 .
Figure 2. (a) State illustration for the first system with low noise level; (b) MSE, illustration for the first system with Low noise level.

Figure 3 .
Figure 3. MSE, illustration for the first system with high noise level.

Figure 4 .
Figure 4. (a) State illustration for the first system with low noise level; (b) RMSE illustration for the first system with low noise level.