Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes

Koichi Kobayashi, Ichiro Maruta, Kazunori Sakurama, Shun-ichi Azuma

Graduate School of Engineering, Tottori University, Tottori, Japan.
Graduate School of Informatics, Kyoto University, Kyoto, Japan.
School of Information Science, Japan Advanced Institute of Science and Technology, Ishikawa, Japan.

DOI: 10.4236/am.2014.510141                

Abstract

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

Keywords

Markov Decision Process, Optimal Control, Real-Time Pricing System

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Camacho, E.F., Samad, T., Garcia-Sanz, M. and Hiskens, I. (2011) Control for Renewable Energy and Smart Grids. In: Samad, T. and Annaswamy, A.M., Eds., The Impacy of Control Technology, IEEE Control Systems Society, New York.
[2]	Ruihua, Z., Yumei, D. and Yuhong, L. (2010) New Challenges to Power System Planning and Operation of Smart Grid Development in China. Proceedings of the 2010 International Conference on Power System Technology, Hangzhou, 24-28 October 2010, 1-8.
[3]	Borenstein, S., Jaske, M. and Rosenfeld, A. (2002) Dynamic Pricing, Advanced Metering, and Demand Response in Electricity Markets. Center for the Study of Energy Markets, University of California, Berkeley.
[4]	Roozbehani, M., Dahleh, M. and Mitter, S. (2010) On the Stability of Wholesale Electricity Markets under Real-Time Pricing. Proceedings of the 49th IEEE Conference on Decision and Control, Atlanta, 15-17 December 2010, 19111918.
[5]	Samadi, P., Mohsenian-Rad, A.-H., Schober, R., Wong, V.W.S. and Jatskevich, J. (2010) Optimal Real-Time Pricing Algorithm Based on Utility Maximization for Smart Grid. Proceedings of the 1st IEEE International Conference on Smart Grid Communications, Gaithersburg, 4-6 October 2010, 415-420.
[6]	Vivekananthan, C., Mishra, Y. and Ledwich, G. (2013) A Novel Real Time Pricing Scheme for Demand Response in Residential Distribution Systems. Proceedings of the 38th Annual Conference of the IEEE Industrial Electronics Society, Monteral, 25-28 October 2012, 1954-1959.
[7]	Bello, D. and Riano, G. (2006) Linear Programming Solvers for Markov Decision Processes. Proceedings of the 2006 IEEE Systems and Information Engineering Design Symposium, Charlottesville, 28 April 2006, 90-95. http://dx.doi.org/10.1109/SIEDS.2006.278719
[8]	Eco-Point System for Housing. http://www.vec.gr.jp/english/topics/100217_1.htm
[9]	Kyoto Eco Money. http://www.city.kyoto.jp/koho/eng/topics/2012_8/index.html
[10]	Sawashima, K., Kubota, Y., Lu, H., Takemae, T., Yoshida, K. and Wan, Y. (2011) Socio-Personal Energy Management System. Keio ALPS2011 Group K Final Report. http://lab.sdm.keio.ac.jp/alps2011k/FinalReport-ALPS2011-K.pdf
[11]	Rust, J. (1994) Structural Estimation of Markov Decision Processes. In: Handbook of Econometrics, Elsevier, Amsterdam, Vol. IV, Chapter 51, 3081-3143.