Optimal Reservoir Operation Using Stochastic Dynamic Programming

This paper focused on the applying stochastic dynamic programming (SDP) to reservoir operation. Based on the two stages decision procedure, we built an operation model for reservoir operation to derive operating rules. With a case study of the China’s Three Gorges Reservoir, long-term operating rules are obtained. Based on the derived operating rules, the reservoir is simulated with the inflow from 1882 to 2005, which the mean hydropower generation is 85.71 billion kWh. It is shown that the SDP works well in the reservoir operation.


Introduction
Stochastic programming is a framework for modeling optimization problems that involve uncertainty [1][2].It has been widely used in water resources planning and management [3][4][5][6][7][8][9].The classic approach is the stochastic dynamic programming (SDP), which is able to tackle not only linear but also nonlinear objective function and constraints.The SDP recurrent formulation is where  t t f x t denotes the optimal benefit-to-go function from decision x and g 0 (x t ) is the benefit with a decision x t .
However, SDP needs to discrete the state space and this makes it heavy computational burden.In order to tackle this "curse of dimensionality", there are several types of methods have been studied.1) Method based on aggregation and decomposition.A method was proposed in which the control problem for a system of M reservoirs in series was decomposed into M sub-problems each with two reservoirs: one a reservoir from the original problem and the other an aggregate representation of the reservoirs downstream of that reservoir [3].The solution time increases linearly with the number of reservoirs in the system and so the approach is practical for large systems [4].This method was extended to a model consisting of that reservoir and a two-dimensional representation of the rest of the system [4].A Benders decomposition algorithm was proposed and applied it to systems including 37 reservoirs [5].2) Method based on approximate of utilization function.An approach based on efficient discretization of the state space and approximation of the value functions was proposed over the continuous state space by means of a flexible feed forward neural network [6].This method can be used for nonlinear objective function and has been applied to multi-reservoir systems with 30 reservoirs.3) Method based on Stochastic Dual Dynamic Programming (SDDP).A sampled hyperplane was used to provide an outer approximation of the benefit-to-go function.These hyperplanes are equivalent to Benders cuts in Benders decomposition [7].
An interval-parameter multistage stochastic programming method was proposed for supporting water resources decision making, where uncertainties expressed as random variables and interval numbers could be reflected [8].This method was improved that handles uncertainties through constructing a set of scenarios that is representative for the universe of possible outcomes, as well as reflects dynamic features of the system conditions and risk levels of violating system constraints within a multistage context [9].They also recognized that it was a challenge to extend this method to flood control operation of reservoir due to large scenarios.

Methodology
The operating rules are a function between decision and observation (or prediction).Typically, single reservoir operating rules provide a release decision when inflow and current water storage have been obtained.If the operation horizon is infinite to future, .
where S i is the initial water storage at time period i, I and R i are inflow and release during time period i. i S  and i are the minimum and maximum allowable water storages during time period i, respectively.i S  R  and i denote the minimum and maximum reservoir releases subjected to physical constraints during time period i, respectively.
far behind (e.g., 100 years late), we estimate it by using periodic characterization.That is, during a cycle periodic time 1 to n, By using a two-stage stochastic programming framework, , , , , ,

 
, , , , i is a benefit-to-go function from i to end time .The means the terminal condition of optimization.
The utilization function i i i may be a linear function (e.g., for water supply) and a nonlinear function (e.g., for hydropower generation).

The Three Gorges Reservoir
The Three Gorges Reservoir (TGR) is a vital project for water resources development of China's largest river, the Yangtze River (Figure 1).The reservoir receives inflow from a 4.5 × 10 3 km long channel with a contributing drainage area of 10 6 km 2 .The mean annual runoff at the dam site is 4.51 × 10 11 m 3 .The TGR is to date the largest multipurpose hydro-development project ever built in the world.Its benefits include flood control, power generation and navigation improvement.Streamflow records from Yichang flow gage station, located about 40 km downstream of the TGR, are used as inflow of the TGR.The interval of the time period is ten-day-long, a traditional Chinese measure of time.The streamflow series from 1882 to 2005 is used in this study.

SDP
The SDP involves two steps as follows: 1) Discretized the Inflow and Reservoir Storage: The inflow during the same time period (such as Jan), are discretized into N intervals from minimum to maximum.The probability of inflow interval i (during time period t) to inflow j (during time period t + 1) is also computed from the observed inflow series.
2) Finding the optimal solutions: By using Equation ( 4), the optimal solution is found and saved as optimal operating rules.The following operating rules are used in this paper. where is the reservoir storage need to decide, and I are the current reservoir storage and inflow.

Results
The above SDP method is used to find the optimal operating rules, and it consumes about 3 minutes.Part of the operating rules is shown in Figure 2.
Based on the derived operating rules, the reservoir is simulated from 1882 to 2005.Finally, the mean hydropower generation is 85.71 billion kWh.The above operating rules only use the current reservoir storage and inflow as variables for make-decision.In other words, this is a two-dimensional SDP problem.Indeed, SDP needs to discrete the state space and this makes it heavy computational burden.

Conclusion
In this paper, we focused on the applying stochastic programming to water resources management.We built one operation model for reservoir operation for operating rules derivation.With a case study of the China's Three Gorges Reservoir, long-term operating rules are obtained.Based on the derived operating rules, the reservoir is simulated with the inflow from 1882 to 2005, which the mean hydropower generation is 85.71 billion kWh.It is shown that the SDP works well in the reservoir operation.

Figure 1 .
Figure 1.The location of the Three Gorges Reservoir basin in China.

Figure 2 .
Figure 2. The optimal operating rules for the middle-tendays in April.