Optimal Demand-Side Management for Smart Micro Grid with Storage

In this paper, an autonomous and distributive demand-side management based on Bayesian game theory is developed and applied among users in a grid connected micro-grid with storage. To derive that strategy, an energy consumption of shiftable loads belonging to a given user is modelled as a noncooperative three-player game of incomplete information, in which each user plays against the storage unit and an opponent gathering all the other users in the micro-grid. Each player is assumed to be endowed with statistical information about its behavior and that of its opponents so that he can take actions maximizing his expected utility. Results of the proposed strategy evaluated by simulating, under MATLAB environment, a connected micro-grid with storage device evidence its efficacy when employed to manage the charging of electric vehicles.


Introduction
Demand-Side Management (DSM), which is the management mechanism of demand side in the next generation of the grid [1], seeks to address various problems such as efficient energy usage, improvement of the demand profile, reduction of the operation cost, shift energy consumption to reduce PAR, and balance power supply and demand [2].Several previous works have been studied in order to implement and motivate users to participate in DSM program.Recently, research has concentrated on pricing mechanism which principally comprises ToU pricing,

System Model
In this study we consider a low voltage MG consists of { } 1, , , , n n N ∈     residential users, where N =  , equipped with RE (e.g., solar PV panel).Users are connected each other and to the public utility via power line.Residential consumers gathering in the MG community share their surplus of energy by storing it in a shared ESU managed by a controller, and act as a single entity when interacting with the public utility.Each user has two types of power loads: ULs and SLs.ULs are appliances that can be turned on at arbitrary instants of the day, i.e. their energy consumption schedule is strictly constrained; that category contains appliances such as refrigeratorfreezer, heating, electric stove and lighting [9].SLs are considered as appliances whose activation can be softly scheduled within specified interval of time during the day.
Furthermore, each household in the MG is assumed to be equipped with a SM which controls and monitors the energy sharing and the electricity consumption.
Each household's SM also exchange, with other SMs via data network, some information about the RE forecasts, prices of energy, the customers' demands at every instant and can get information of energy available in the storage unit.We assume that the communication between MG and power utility is supervised by a MSM, i.e. an upgraded SM adapted for operating at high power and serving as the intermediate link between NAN and the main grid global network (BN).The architecture of the proposed MG is shown in Figure 1.
At every instant of time t, each household n ∈  has following sets of power: the renewable power produced by his own RESs, the power demand from his appliances (SLs and ULs).The real time power exchanged by each customer with the MG is evaluated as follows: ( ) ( ) ( ) ( ) ( ) Equation (1) gives the instant power ( ) n l t exchanged by the user itself at an instant of time t with the MG.It is the sum of: ( ) ( ) r n l t accounting both the power from RESs for the considered user and the power absorbed by its ULs and, ( ) ( ) s n l t depending on the activation of its SLs.That user's power quantity can be positive (if he is absorbing power from the MG) or negative (if he is supplying power to the MG) and we highlight that it is constrained to the following inequality: where ( )   ,max n g P is the maximum power generated by the n th user's renewable re- The overall power monitored by the MSM (see Figure 1 for details) is derived as follows; let ( ) T l t be that power at an instant t of time; it is expressed as Figure 1.Architecture of the MG [10].

follows:
( ) ( ) ( ) That overall power is also constrained to: ( ) inj pu P : the negative maximum power that can be injected in the main grid; ( )   abs pu

P
: the maximum positive power that can be absorbed from the public utility, and the previous expressions lead to the following inequality: The maximum power that can be injected to the publicutility is a negative value of the summation of power generated by all RESs and the maximum power that can come from the battery when discharging; it is expressed as follows:

Demand-Side Management Based on Bayesian Game Theory
In this Section a brief description of our game model is provided and, on the basis of this model, a mixed strategy for the activation of SLs is developed.

Rules and Description of the Game
The SM installed at each prosumer's premise is considered as player (taken as player 1 in the following) behaving in a selfish and rational manner, capable of turning the load on or off.Furthermore, as this SM competes with the rest of the MG community in the exploitation of the energy resources available, we can model other 1 N − prosumers as a single aggregated opponent called player 2 and the shared battery as player 3 because it will be competing with all the prosumers for charging and discharging.
The power flow for player 2 is defined as the difference between the overall power available in the MG and power for player 1 and player 3; it can be expressed as follows: which verify the following inequality: where ( )   ,max   be used to describe the interactions between each player with the rest of the MG community.
From the every player's point of view, there is a payoff associated to each of its actions.The evaluation of payoffs gives a full description of the game.For our work we will assume that: • For prosumers, the action of keeping SLs off is associated with a payoff equals to 0 for the corresponding prosumer without taking into account the power absorbed or generated by other prosumers and the battery.
• The activation of SLs will entail a variation in the payoff for the corresponding prosumer because it will change the operating conditions of the MG, which means that the associated payoff n EP will depend on the expected (statistical) future consumption/generation of the whole MG community.
• The battery (player 3)'s payoff will depend on its charging and discharging efficiency.In other words, its payoff will decrease as the number of charging cycles gets larger; we assume that the battery's life will be reduced as a function of its charging cycles which will influence its capacity.
The derivation of the expected payoffs n EP in the following sections of this work will take into account: 1) A pricing model for power shared between players and the MG; means that each power exchange will be paid or rewarded with a certain amount of monetary units.
2) Specific statistical information available at each prosumer's SM and the battery's controller.

Economic Model of the Smart Micro-Grid
The pricing model takes into account any power exchange between players and the MG.A provision of service, accounted by a power exchange between player and the MG, involves a variation in the total amount of virtual currency owned by the corresponding player.In our work, such variation depends on the MG's condition and the cost function.
We assume in the following that it depends on the operating condition of the MG represented by a state variable that can take two values.Those values describe the normal (briefly state 0) and stress (briefly state 1) operating conditions.To bring out the characteristics of those states, we consider a positive power threshold that verifies the following inequalities: ( ) ( ) ( ) for the stress state The normal state represents the regular operating condition of the MG, whereas the stress state corresponds to high consumption of power which may end up by some risk of blackout.

Cost Function
The derivation of the cost function is given by: , , max , 0 min , 0 , , , , .
The cost function expresses, for given players' powers, the cost (if negative) or reward (if positive) for the considered player.The first term of the Equation (11) represents the cost associated with the power absorbed by player 1 from the MG, the second one represents the gain coming from the power supplied to the MG and the third one is a fairness term referring to the immediate power exchange between player 1 with that of player 2 and player 3, and the coefficients , and F k are weight functions that can be adjusted by the MSM in order to in- fluence the behavior of the MG community.The variation of powers according to time has been omitted to ease the reading.
In our work, the weight functions appearing in the right hand side of Equation (11) are given by the following expression: In the Equation ( 12), X can take two values: A (absorbing) and G (generating), j can also take two values: 0 (normal state) and 1 (stress state).

Statistical Information Evaluated by Each Player
As Bayesian game theory is concerned, we assumed in (2) the availability of statistical information at every SM and controller; then player 1 is provided with three different probability density functions (pdfs); ( ) ( ) prosumer's behavior and the battery's behavior respectively.In short, the controller and n th SM's statistical knowledge about the complete MG can be summarized as follows: 1) The first order probability density function (pdf) ( ) ( ) which refers to the overall power absorbed by the MG or supplied to the public utility without taking into account DSM.
2) The first order pdf ( ) ( ) 3) The first order pdf ( ) In order to derive the payoffs function n EP , what is needed to the prosumer's SM is the knowledge of the joint probability ( ) ( ) ( ) , , ; . The number of prosumers forming player 2 influences the statistical behavior of ( ) l − in a way that they may exhibit different behaviors when speaking of power consumption/ generation compared to player 1 and player 3. We will assume in the following that the joint probability can be factored as follows: , , ; , , , It is interesting to mention that in order to estimate the above indicated pdfs, specific learning algorithms have to be developed for a real world implementation of the suggested strategy.
Firstly, the pdf ( ) ( ) x τ can be evaluated by the MSM and then its representation distributed to all players.To achieve that, the MSM must be provided with the exact knowledge of past consumption/generation of players.After receiving the necessary data about users and the weather predictions, the MSM exploits them using improved machine learning tools such as regression models to reliably forecast the statistical behavior of the MG.The assumption can be adopted when the number of N prosumers is big.
Secondly, the approximation of the pdf ( ) ( ) xτ can be reliably evaluated by each prosumer's SM using machine learning tools on the basis of its real time energy consumption data stored over a number of different days.
Lastly, the estimation of the pdf ( ) f x τ can be accomplished by the bat- tery controller capable of a real time sensing of the battery power level and be able to predict the statistical charging/discharging behavior of the battery.

Derivation of the Expected Payoff
Knowing the cost function given in Equation ( 11) and the statistical information previously described, the expected payoff related to the switching on (briefly ON) of SLs can be calculated as follows.We first define the expected overall cost  5) and ( 7)): Next, the expect payoff n EP associated with the ON action of player 1 is defined as the difference between the expected cost related to the activation of the considered load at = and that associated with keeping it off, which leads to: (15) can be found as follows.We first replace (13) in Equation ( 14) and we get: We can further simplify the equation by substituting the upper limit of the second integral appearing in the right hand side of Equation ( 16) We finally replace Equation (17) in Equation ( 15) which leads to the following equation after some manipulations: where: The parameter in Equation ( 19) can be interpreted as an expected cost density because it indicates how the overall expected cost that has a discount factor ω ( 0 1 ω < < ).This consideration is justified by the fact that: 1) The game is replayed by player 1 every s T s until th sl n SLs is activated or the maximum activation time limit is reached; 2) For each shiftable load, the activation interval is scheduled during ( ) n l N slots; which means that the activation time interval for the considered load is: 3) The density cost function

N
expressions, each related to a different time slot.To each time slot, we assign a weight factor decreasing exponentially with the slot index [11].

H. Ininahazwe et al.
The new expression of the expected cost density taking into account the above considerations can be expressed as follows: where: mainly depends on the power consumption from other players which makes difficult to derive the equilibrium point for the optimal strategy (22).
In our work, we did not adopt the strategy given by Equation ( 22) for the following reasons: 1) As stated before, the MSM estimates and periodically broadcasts an update of two probabilities density functions; one related to the overall power in the MG: ( ) ( ) x τ related to the power consumption of a prosumer is esti- mated by the n th SM at least on the daily basis.That is why different values of the cost function appearing in the right hand of Equation ( 22) may emerge when computed at different instants of time and need to be recalculated when an update of the mentioned pdfs is broadcasted.
2) When multiple SLs are simultaneously activated by the n th prosumer, they need to be properly and efficiently scheduled.
These remarks and considering previous work on load management have led us to developing a mixed strategy.It will be part of the following section.

Mixed Strategy for the Game
In the proposed game, player 1 replays the game at instants ) corresponding to the ON and OFF actions respectively.In our game model, ( ) [ ] n on P p is the activation probability for the n th prosumer in the p th attempt.We need to highlight that: • Given the activation vector ( )   23) will be written as follows after factoring: We can derive the activation probability from Equation (24) which gives: is the activation probability to be selected at each trial to get the probability of success equals to ( ) The objective of our mixed strategy is to adjust the activation probabilities so as to minimize, on the average over the set of prosumers, the reduction in the expected utility of the expected cost density , which represents for a given n l − the deviation of the expected cost density from its average and it can be expressed as follows: , n n sl sl t t is given by the following equation: The integration domain of the integral appearing in the right hand side of Equation    which means that the value assigned to n P is the same as the value that each element of the sequence ( ) [ ] n on P p should take on if all the activation trails made by the n th prosumer were equally likely.On the other hand, the evaluation of n γ follows an optimization approach based on the following considerations.After 2) Each prosumer repeats the game every s T until he opts for the ON action or the OFF action.The slot duration s T is proportional to the overall power.
Then, if the OFF action has been selected in a certain repetition of the game, the next attempt should occur when the overall power ( ) T L t has undergone a significant change.
3) The energy available in the MG is shared between the three players on the basis of probabilistic mechanism.It may happen an instant when the scheduled power for SLs is not sufficient.In that case, the MSM is supposed to broadcast a disconnection message of specific portion (if not all) of active SLs to avoid overload risks.
4) The reference power r P which is selected by the MSM and broadcasted to all prosumers, plays an important role because its change in value modifies the equilibrium of the MG.The value of that threshold is practically fixed on the basis of the expected consumption over the whole day.

Numerical Results
This chapter describes the results obtained by simulating the DSM strategy for a MG collecting 100 N = residential prosumers sharing a battery capable of generating 300 KW for 1 hour.Analysis of the results to evaluate the effectiveness of the Bayesian DSM strategy is also presented.

Assumptions for the Simulation
The following assumptions have been made in all our simulations: In order to account the daily fluctuations in the power generated by solar PV panels, we have superposed the average power generated by those renewable sources with a zero mean random Gaussian process.
2) We have considered 15 appliances for each prosumer that are characterized by a probability Mass Function (pmf).For a specific appliance, a pmf represents its activation probability for a single day and is dependent to the prosumer's be- havior.An example of a daily power consumption of a random prosumer is shown in Figure 5 with 1 hour step size.The approximation of pdf ( ) ( ) 3) The Gaussian approximation has also been used for the pdf ( ) ( ) 4) In our simulation of the proposed DSM strategy, the parameters presented in Table 1 have been used in all simulations.
5) The MG load demand has been observed for three days corresponding to 72 hours and 288 slots when taking 15 minutes as slot duration.The activation requests of SLs (only the PHEV for simplicity), has been concentrated in the second day after reaching a steady state condition in order to evaluate the effectiveness of our DSM strategy under different load demand.
6) The energy storage unity (the battery in our case) has been sized according to the total load demand.In our simulation we have estimated the battery to     use of DSM and a shared battery in the MG brings between 40% and 42% on average in the improvement of the MG PAR.Note that this improvement is substantially better compared to that provided by the use of the Bayesian game theory developed in [7], where 34% improvement of the PAR has been reached.2) −0.57mu in the presence of DSM and −0.66 mu in its absence and with the battery charging;

Evaluation of the Expected Payoff
3) −2.23 mu in the presence of the DSM and −4.44 mu in its absence and with the battery discharging.

Conclusion and Future Work
In this paper, a game theory based on DSM strategy relying on statistical information about prosumer consumption, the charging/discharging of a shared battery and the overall consumption of a MG has been developed.The proposed strategy helps to mitigate fluctuations in the load demand when applied to a MG with SLs and preserve privacy for users.Numerical results obtained when using the strategy to the MG and considering a shared battery in a multi user scenario, evidence a significant reduction in the MG PAR for the management of the recharge of PHEVs considered as shiftable load owned by each user.Furthermore, the strategy allows a substantial satisfaction for the activation of those SLs when the community storage is contributing to the power supply in the MG.Future work concerns the management of the community storage by autonomously scheduling the charging and discharging of the energy storage units in a MG.
power consumed by the same user.The battery model needs more clarifications concerning its power consumption.The battery's controller provides real time monitoring of the power battery itself at time instant t with the MG; this quantity is positive (negative) if the battery is charging from (discharging to) the MG and satisfies the following inequality at any instant of time t: maximum power that can be discharged from the battery and the maximum power needed for charging the battery.

≤
are respectively the maximum powers absorbed and generated by player 2; We note that

<
when player 2 is providing power to the MG.Those previous MG parameters have led us to a simplified three players game instead of a complicated game of 1 N + players.Our 3 players game model can H.Ininahazwe et al.
which are related to the overall power available in the MG, the n th one related to the battery state (charging or discharging) and power level: ; in the same way, the probability densi- ty function ( ) ( ) chooses to turn the load ON or the maximum number of activation trial ( n K ) is reached.The selection of a specific action in the p th attempt is randomly chosen in a given set of action based on the probabilities divided into two parts:( )

Figure 2 .
Figure 2. Representation of the function 3 shows the variation of the activation probability as a function of the error signal [ ] n e p (34).We notice that this probability becomes zero when the signal error negative, which corresponds to the MG power consumption higher than the reference power n P .The parameters appearing in the right-hand side of Equation (35) are defined as follows: n P represents a reference probability level, n γ is a real positive parameter and [ ] represents a clipping function dependent on each prosumer, which limits the variation interval of ( ) [ ] n on P p , evaluated using Equation (35) to the range [0, 1].The computation of Equation (35) requires the knowledge of the parameters n P and n γ .In our work the value suggested by (25) has been selected:

Figure 4 .
Figure 4. Steps for executing the Bayesian DSM strategy.
using a Gaussian model since it gives an accurate representation of this pdf.The parameters (mean and variance) of that pdf are perfectly known by the n th prosumer's SM.
re- lated to the overall power of the MG in absence of the DSM.As previously mentioned, the approximation ( ) ( ) ( ) ( ) and its parameters (mean value and variance) are perfectly known by all prosumers.

Figure 5 .
Figure 5. Representation of a specific realization for the daily power consumption associated with 15 home appliances (unshiftable loads) owned by a MG prosumer.

Figure 6
Figure 6.MG overall power ( ) T l t and overall power

Figure 7
Figure 7. MG overall power ( ) T l t and overall power

Figure 8 .
Figure 8. MG overall power ( ) T l t and overall power

Figure 9
Figure 9. Pdf n EP (15) evaluated for the activation of the PHEVs (SLs) owned by the prosumer in the presence of the DSM and the battery in relation to the cases where they are not present may vary from one prosumer to another.

Figures 10 -
show the gap of the expected payoff existing between the use of DSM (blue squares) and the battery compared to the cases where DSM is absent (red squares) and the battery charging or discharging.Note that the line that connects the two squares referring to a specific user is blue (red) if the first value is greater (smaller) than the second one.Especially Figure10shows the gap between the realization of the values of the mentioned expected payoffs { } , 1, 2, ,100 n EP n =  evaluated for the activation of the PHEV owned by the n th prosumer without considering the battery.The realization of the values of the above mentioned expected payoffs { } , 1, 2, ,100 n EP n =  when the battery is charging is shown in Figure 11.The last case in the evaluation of the expected payoffs { } , 1, 2, ,100 n EP n =  related to the activation of the PHEV has been simulated when the battery is discharging; which is exemplified in Figure 12.These results show that, in the MG community, 70, 64, and 73 prosumers respectively for the MG operates without battery, with battery charging and finally with battery discharging, benefit from the DSM strategy in terms of monetary

Figure 11 .
Figure 11.Expected payoffs , 1, 2, ,100 n EP n =  evaluated for the activation of the PHEVs with the battery charging.

Figure
Figure 12.Expected payoffs , 1, 2, ,100 n EP n =  evaluated for the activation of the PHEVs with the battery discharging. , , d , [ ] PHEV).The PHEV will be considered as the only SLs for each prosumer.Furthermore, the prosumer's PV panels are able to generate up to