Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

Full-Text HTML XML Download Download as PDF (Size:459KB) PP. 1-14
DOI: 10.4236/am.2017.81001    542 Downloads   698 Views  


In this paper, an efficient computational approach is proposed to solve the discrete time nonlinear stochastic optimal control problem. For this purpose, a linear quadratic regulator model, which is a linear dynamical system with the quadratic criterion cost function, is employed. In our approach, the model-based optimal control problem is reformulated into the input-output equations. In this way, the Hankel matrix and the observability matrix are constructed. Further, the sum squares of output error is defined. In these point of views, the least squares optimization problem is introduced, so as the differences between the real output and the model output could be calculated. Applying the first-order derivative to the sum squares of output error, the necessary condition is then derived. After some algebraic manipulations, the optimal control law is produced. By substituting this control policy into the input-output equations, the model output is updated iteratively. For illustration, an example of the direct current and alternating current converter problem is studied. As a result, the model output trajectory of the least squares solution is close to the real output with the smallest sum squares of output error. In conclusion, the efficiency and the accuracy of the approach proposed are highly presented.

Cite this paper

Kek, S. , Li, J. and Teo, K. (2017) Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences. Applied Mathematics, 8, 1-14. doi: 10.4236/am.2017.81001.


[1] Ahmed, N.U. (1973) Optimal Control of Stochastic Dynamical Systems. Information and Control, 22, 13-30.
[2] Ahmed, N.U. and Teo, K.L. (1974) Optimal Feedback Control for a Class of Stochastic Systems. International Journal of Systems Science, 5, 357-365.
[3] Kliemann, W. (1983) Qualitative Theory of Stochastic Dynamical Systems—Applications to Life Sciences. Bulletin of Mathematical Biology, 45, 483-506.
[4] Adomian, G. (1985) Nonlinear Stochastic Dynamical Systems in Physical Problems, Journal of Mathematical Analysis and Applications, 111, 105-113.
[5] Li, X., Blount, P.L., Vaughan, T.L. and Reid, B.J. (2011) Application of Biomarkers in Cancer Risk Management: Evaluation from Stochastic Clonal Evolutionary and Dynamic System Optimization Points of View. PLOS Computational Biology, 7, Article ID: e1001087.
[6] Lopez, R.H., Ritto, T.G., Sampaio, R. and de Cursi, J.E.S. (2014) Optimization of a Stochastic Dynamical System. Journal of the Brazilian Society of Mechanical Sciences and Engineering, 36, 257-264.
[7] Lozovanu, D. and Pickl, S. (2014) Optimization of Stochastic Discrete Systems and Control on Complex Networks: Computational Networks, Volume 12 of Advances in Computational Management Science. Springer, Cham Heidelberg, Switzerland.
[8] Mellodge, P. (2015) A Practical Approach to Dynamical Systems for Engineers. Woodhead Publishing, Swaston, Cambridge, UK.
[9] Craine, R. and Havenner, A. (1977) A Stochastic Optimal Control Technique for Models with Estimated Coefficients. Econometrica, 45, 1013-1021.
[10] Teo, K.L. Goh, C.J. and Wong, K.H. (1991) A Unified Computational Approach to Optimal Control Problems. Longman Scientific and Technical, New York.
[11] Zhang, K., Yang, X.Q., Wang, S. and Teo, K.L. (2010) Numerical Performance of Penalty Method for American Option Pricing. Optimization Methods and Software, 25, 737-752.
[12] Rawlik, K., Toussaint, M. and Vijayakumar, S. (2012) On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference. Proceeding of 2012 Robotics: Science and Systems Conference, Sydney, 9-13 July 2012, 3052-3056.
[13] Kek, S.L., Mohd Ismail, A.A. and Teo, K.L. (2015) A Gradient Algorithm for Optimal Control Problems with Model-Reality Differences. Numerical Algebra, Control and Optimization, 5, 252-266.
[14] Socha, L. (2000) Application of Statistical Linearization Techniques to Design of Quasi-Optimal Active Control of Nonlinear Systems. Journal of Theoretical and Applied Mechanics, 38, 591-605.
[15] Li, W. and Todorov, E. (2007) Iterative Linearization Methods for Approximately optimal Control and Estimation of Nonlinear Stochastic System. International Journal of Control, 80, 1439-1453.
[16] Zitouni, F. and Lefebvre, M. (2014) Linearization of a Matrix Riccati Equation Associated to an Optimal Control Problem. International Journal of Differential Equations, 2014, Article ID: 741390.
[17] Hernandez-Gonzalez, M. and Basin, M.V. (2014) Discrete-Time Optimal Control for Stochastic Nonlinear Polynomial Systems. International Journal of General Systems, 43, 359-371.
[18] Kek, S.L., Teo, K.L. and Mohd Ismail, A.A (2010) An Integrated Optimal Control Algorithm for Discrete-Time Nonlinear Stochastic System. International Journal of Control, 83, 2536-2545.
[19] Kek, S.L., Teo, K.L. and Mohd Ismail, A.A. (2012) Filtering Solution of Nonlinear Stochastic Optimal Control Problem in Discrete-Time with Model-Reality Differences. Numerical Algebra, Control and Optimization, 2, 207-222.
[20] Bryson, A. and Ho, Y.C. (1975) Applied Optimal Control: Optimization, Estimation, and Control. Taylor & Francis, Abingdon.
[21] Lewis, F.L., Vrabie, D.L. and Syrmos, V.L. (2012) Optimal Control. 3rd Edition, John Wiley & Sons, Hoboken.
[22] Kirk, D.E. (2004) Optimal Control Theory: An Introduction. Dover Publication Inc., Mineola.
[23] Verhaegen, M. and Verdult, V. (2007) Filtering and System Identification: A Least Squares Approach. Cambridge University Press, Cambridge.
[24] Minoux, M. (1986) Mathematical Programming: Theory and Algorithms. John Wiley & Sons, Paris.
[25] Chong, K.P. and Zak, S.H. (2013) An Introduction to Optimization. 4th Edition, John Wiley & Sons, Hoboken.
[26] Poursafar, N., Taghirad, H.D. and Haeri, M. (2010) Model Predictive Control of Nonlinear Discrete Time Systems: A Linear Matrix Inequality Approach. IET Control Theory and Applications, 4, 1922-1932.
[27] Haeri, M. (2004) Improved EDMC for the Processes with High Variations and/or Sign Changes in Steady-State Gain. COMPEL, 23, 361-380.
[28] Passino, K.M. (1989) Disturbance Rejection in Nonlinear Systems: Example. IEE Proceedings D, 136, 317-323.

comments powered by Disqus

Copyright © 2017 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.