On the Connection between the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks


In the framework of stochastic processes, the connection between the dynamic programming scheme given by the Hamilton-Jacobi-Bellman equation and a recently proposed control approach based on the Fokker-Planck equation is discussed. Under appropriate assumptions it is shown that the two strategies are equivalent in the case of expected cost functionals, while the Fokker-Planck formalism allows considering a larger classof objectives. To illustratethe connection between the two control strategies, the cases of an Itō stochastic process and of a piecewise-deterministic process are considered.

Share and Cite:

Annunziato, M. , Borzì, A. , Nobile, F. and Tempone, R. (2014) On the Connection between the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks. Applied Mathematics, 5, 2476-2484. doi: 10.4236/am.2014.516239.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] Ghman, I.I. and Skorohod, A.V. (1972) Stochastic Differential Equations. Springer-Verlag, New York.
[2] Bertsekas, D. (2007) Dynamic Programming and Optimal Control, Vols. I and II. Athena Scientific.
[3] Fleming, W.H. and Soner, H.M. (2006) Controlled Markov Processes and Viscosity Solutions. Springer-Verlag, Berlin.
[4] Annunziato, M. and Borzì, A. (2010) Optimal Control of Probability Density Func-tions of Stochastic Processes. Mathematical Modelling and Analysis, 15, 393-407.
[5] Annunziato, M. and Borzì, A. (2013) A Fokker-Planck Control Framework for Multidimensional Stochastic Processes. Journal of Computational and Applied Mathematics, 237, 487-507.
[6] Cox, D.R. and Miller, H.D. (2001) The Theory of Stochastic Processes. Chapman & Hall/CRC, London.
[7] Fleming, W.H. and Rishel, R.W. (1975) Deterministic and Stochastic Optimal Control. Springer-Verlag, Berlin.
[8] Borkar, V.S. (2005) Controlled Diffusion Processes. Probability Surveys, 2, 213-244.
[9] Falcone, M. (2008) Numerical Solution of Dynamic Programming Equations. In: Bardi, M. and Dolcetta, I.C., Eds., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Birkhauser.
[10] Kushner, H.J. (1990) Numerical Methods for Stochastic Control Problems in Continuous Time. SIAM Journal on Control and Optimization, 28, 999-1048.
[11] Forbes, M.G., Guay, M. and Forbes, J.F. (2004) Control Design for First-Order Processes: Shaping the Probability Density of the Process State. Journal of Process Control, 14, 399-410.
[12] Jumarie, G. (1992) Tracking Control of Nonlinear Stochastic Systems by Using Path Cross-Entropy and Fokker-Planck Equation. International Journal of Systems Science, 23, 1101-1114.
[13] Kárny, M. (1996) Towards Fully Probabilistic Control Design. Automatica, 32, 1719-1722.
[14] Wang, H. (1999) Robust Control of the Output Probability Density Functions for Multivariable Stochastic Systems with Guaranteed Stability. IEEE Transactions on Automatic Control, 44, 2103-2107.
[15] Primak, S., Kontorovich, V. and Lyandres, V. (2004) Stochastic Methods and Their Applications to Communications. John Wiley & Sons, Chichester.
[16] Risken, R. (1996) The Fokker-Planck Equation: Methods of Solution and Applications. Springer, Berlin.
[17] Lions, J.L. (1983) On the Hamilton-Jacobi-Bellman Equations. Acta Applicandae Mathematica, 1, 17-41.
[18] Crandall, M., Ishii, H. and Lions, P.L. (1992) User’s Guide to Viscosity Solutions of Second Order Partial Differential Equations. Bulletin of the American Mathematical Society, 27, 1-67.
[19] Le Bris, C. and Lions, P.L. (2008) Existence and Uniqueness of Solutions to Fokker-Planck Type Equations with Irregular Coefficients. Communications in Partial Differential Equations, 33, 1272-1317.
[20] Aronson, D.G. (1968) Non-Negative Solutions of Linear Parabolic Equations. Annali della Scuola Normale Superiore di Pisa. Classe di Scienze, 22, 607-694.
[21] Bogachev, V., Da Prato, G. and Röckner, M. (2010) Existence and Uniqueness of Solutions for Fokker-Planck Equations on Hilbert Spaces. Journal of Evolution Equations, 10, 487-509.
[22] Lions, J.L. (1971) Optimal Control of Systems Governed by Partial Differential Equations. Springer, Berlin.
[23] Lachapelle, A., Salomon, J. and Turinici, G. (2010) Computation of Mean Field Equilibria in Economics. Mathematical Models and Methods in Applied Sciences, 20, 567-588.
[24] Carlier, G. and Salomon, J. (2008) A Monotonic Algorithm for the Optimal Control of the Fokker-Planck Equation. IEEE Conference on Decision and Control, CDC 2008, Cancun, 9-11 December 2008, 269-273.
[25] Davis, M.H.A. (1984) Piecewise-Deterministic Markov Processes: A General Class of Non-Diffusion Stochastic Models. Journal of the Royal Statistical Society. Series B (Methodological), 46, 353-388.
[26] Annunziato, M. (2008) Analysis of Upwind Method for Piecewise Deterministic Markov Processes. Compu-tational Methods in Applied Mathematics, 8, 3-20.
[27] Capuzzo Dolcetta, I. and Evans, L.C. (1984) Optimal Switching for Ordinary Differential Equations. SIAM Journal on Control and Optimization, 22, 143-161.
[28] Annunziato, M. and Borzì, A. (2014) Optimal Control of a Class of Piecewise Deterministic Processes. European Journal of Applied Mathematics, 25, 1-25.
[29] Moresino, F., Pourtallier, O. and Tidball, M. (1988) Using Viscosity Solution for Approximations in Piecewise Deterministic Control Systems. Report RR-3687-HAL-INRIA, Sophia-Antipolis.

Copyright © 2023 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.