Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes


A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

Share and Cite:

Kobayashi, K. , Maruta, I. , Sakurama, K. and Azuma, S. (2014) Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes. Applied Mathematics, 5, 1485-1495. doi: 10.4236/am.2014.510141.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] Camacho, E.F., Samad, T., Garcia-Sanz, M. and Hiskens, I. (2011) Control for Renewable Energy and Smart Grids. In: Samad, T. and Annaswamy, A.M., Eds., The Impacy of Control Technology, IEEE Control Systems Society, New York.
[2] Ruihua, Z., Yumei, D. and Yuhong, L. (2010) New Challenges to Power System Planning and Operation of Smart Grid Development in China. Proceedings of the 2010 International Conference on Power System Technology, Hangzhou, 24-28 October 2010, 1-8.
[3] Borenstein, S., Jaske, M. and Rosenfeld, A. (2002) Dynamic Pricing, Advanced Metering, and Demand Response in Electricity Markets. Center for the Study of Energy Markets, University of California, Berkeley.
[4] Roozbehani, M., Dahleh, M. and Mitter, S. (2010) On the Stability of Wholesale Electricity Markets under Real-Time Pricing. Proceedings of the 49th IEEE Conference on Decision and Control, Atlanta, 15-17 December 2010, 19111918.
[5] Samadi, P., Mohsenian-Rad, A.-H., Schober, R., Wong, V.W.S. and Jatskevich, J. (2010) Optimal Real-Time Pricing Algorithm Based on Utility Maximization for Smart Grid. Proceedings of the 1st IEEE International Conference on Smart Grid Communications, Gaithersburg, 4-6 October 2010, 415-420.
[6] Vivekananthan, C., Mishra, Y. and Ledwich, G. (2013) A Novel Real Time Pricing Scheme for Demand Response in Residential Distribution Systems. Proceedings of the 38th Annual Conference of the IEEE Industrial Electronics Society, Monteral, 25-28 October 2012, 1954-1959.
[7] Bello, D. and Riano, G. (2006) Linear Programming Solvers for Markov Decision Processes. Proceedings of the 2006 IEEE Systems and Information Engineering Design Symposium, Charlottesville, 28 April 2006, 90-95.
[8] Eco-Point System for Housing.
[9] Kyoto Eco Money.
[10] Sawashima, K., Kubota, Y., Lu, H., Takemae, T., Yoshida, K. and Wan, Y. (2011) Socio-Personal Energy Management System. Keio ALPS2011 Group K Final Report.
[11] Rust, J. (1994) Structural Estimation of Markov Decision Processes. In: Handbook of Econometrics, Elsevier, Amsterdam, Vol. IV, Chapter 51, 3081-3143.

Copyright © 2023 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.