Long-Term Electrical Load Forecasting in Rwanda Based on Support Vector Machine Enhanced with Q-SVM Optimization Kernel Function ()
1. Introduction
With nowadays Artificial Intelligence (AI) role in the countries’ fourth industrial revolution development, the Government of Rwanda (GoR) intends to shift from a lower to a middle-income country. To achieve the objectives, the GoR has continued to implement energy programs and projects aimed at delivering universal access to electricity by 2024. With the current total installed generation capacity with load demand rate, probabilistic load generation, and demand forecasting [1] , several reports and research on the subject focused on the available energy resources, transmission, and distribution plans and programs. Rwanda has endowed with natural energy resources, including hydro, solar, wind, and methane gas. In this scheme with a highly new connection rate, the Public Private Partnership (PPP) strategy and Energy Access Roll-out Program (EARP) were adopted to accelerate the connectivity rate and the government focuses on transmission and distribution projects through REG, the country’s utility company [2] . In accordance with Ministry of Infrastructure (Mininfra) report [3] , Rwanda’s connectivity rate is currently evaluated at 72%, up from 6% in 2008, whereas 51% and 22% are grid and off-grid connected, respectively.
Moreover, despite Africa’s vast energy resources and progress in establishing regional power pools over the past two decades, the country’s power grid and distribution plans still are underdeveloped and inadequate power generation system, which leaves millions of people inaccessible to electricity [4] . Electrical load forecasting is based on available load demand data, Rwanda used to generate, import, and export electricity loads. Billed loads (kWh) in 2000 were 203,856,357 kWh of which domestic loads were 110,843,390 kWh, import loads were 94,088,250 kWh and export loads were 1,074,283 kWh. However, there has been a significant increase in 2020, with billed loads of 1,762,393,073 kWh, of which the domestic load was 1,338,801,836 kWh, the imported load was 204,219,150 kWh and the exported load was 11,145,955 kWh [5] . Electricity load forecasting as a complex non-linear problem related to economic, demographic, GDP, and weather factors. Long-term load forecasting provides useful information for power system maintenance planning, proper assessment, and limited energy resources.
Therefore, proper electrical load forecasting requires particular “forecasting intervals” for incomplete time series with max-margin classification [6] . However, when implementing a quadratic SVM prediction model, the interval length is chosen arbitrarily, which has a significant positive impact on model performance. Based on forecasting intervals, load forecasting is divided into three types: short, medium, and long-term as STLF, MTLF, and LTLF, respectively [7] . In this paper, the SVM forecasting model can be used to solve long-term load forecasting problems in power systems. LTLF methods and models were described in [8] , which stated a forecasting interval of one year to several years. Capuno et al. proposed a Very Short-Term Load Forecasting (VSTLF) based on Algebraic Prediction (AP) [9] . Therefore, Mobarak et al. developed a hybrid of Long Short-Term Memory (LSTM), Gated Recurrent Units (GRU), and Recurrent Neural Networks (RNN) algorithms for LF [10] .
Electrical load forecasting has been a subject of research for decades, and the accuracy of long-term load forecasting has implications for generation planning and development of energy infrastructure. Liu et al. introduced LTLF based on a Time-variant ratio multi-objective optimization with a fuzzy time series model [11] . Many AI models and methods, especially LTLF, were critical for many applications such as provisioning power generation and distribution planning. The SVM model with LTLF method is very important for system security and management as the critical to economic performance. In recent years, a new strategy for improving the accuracy of load forecasting methods has been adopted [12] .
The above methods are mainly applied for accurate forecasts, which enable better execution of basic operational tasks such as unity deployments, economic use, and fuel planning and unit maintenance [13] . However, load forecasting is a challenging issue as the load of a particular hour depends not only on the load of the previous hour, but also on the same hour of the previous day, with the same denomination, on the same hour of the day with non-linear, social variability, and economic environments, etc. [14] . Different methods have been applied, Wi et al. divided into statistical methods and computational intelligence methods for holiday load forecasting using fuzzy polynomial regression [15] , the preceding comprises of regression analysis and time series [16] . The modern includes neural networks, expert systems, fuzzy logic, etc. [17] [18] [19] . Many scholars pay more attention to the uncertainty hybrid methods, Amjady et al. introduced a new hybrid forecast technique for MTLF [20] , Hooshmand et al. proposed a hybrid intelligent algorithm based on STLF [21] , and Soliman et al. modeled a long-term electrical load forecasting [22] .
Notwithstanding the above-mentioned methods available, Electrical Energy Demand (EED) forecasting, a deep belief network-based electricity load forecasting is carried out on the Macedonian load system [23] . Jiang et al. [24] disposed of deep learning in power system operation and energy trading. As LTLF can play key strategies for energy dispatch, ANN load forecasting and economic dispatch based on PSO have been demonstrated [25] . Regardless of the difficulty in electrical load forecasting, the optimal and proficient economic set-up of electrical power systems has continually occupied a vital position in the electrical power industries [26] . The main purpose of this study is to apply SVM and quadratic models with different types of kernel functions to predict next year’s load demand for different load types for 23 consecutive years (kWh). The effectiveness of the proposed method is demonstrated using Rwanda utility load data (1998 to 2020). The SVM method consists of various methods to test the robustness of the solution to the proposed problem. Moreover, the Q-SVM method is adopted in conclusion to improve forecasting performance.
2. The Combined Forecasting Model
In recent years, short-term, medium-term, and long-term electrical load forecasting has been intensively studied. Theoretical investigation on a combined model for medium and long-term load forecasting based on load decomposition and big data technologies [27] . Combining a predictive model based on Back Propagation (BP) powered by PSO with its application was tackled [28] , and a BP neural network based on a gray forecast model and a Markov chain was used to forecast China’s load demand [29] . Application of neural network and fuzzy theory in STLF [30] , optimization of electrical load forecasting for SVM model based on data mining and Lyapunov exponent was described [31] , SVM forecasting method improved by chaotic PSO and its application [32] , combined RS-SVM forecasting model is applied in power supply demand [33] .
Although the above-mentioned long-term load forecasting methods have come a long way, neural networks and support vector machines still have certain issues: Subject to localized extrema, overlearning, etc. Lorena et al. tackled the issues of parameter selection for SVM based on genetic algorithms [34] , and parameter selection for LS-SVM based on modified ant colony optimization is used to optimize SVM parameters [35] . However, GA requires many complex operations such as encoding, selection, crossover, and mutation, whereas PSO is relatively smooth. Moreover, according to Issam, a new quadratic kernel-free non-linear SVM called QSVM is introduced [36] . SVM optimization can be specified as follows:
• Maximized the underlying geometric margin of all training data with a feature margin greater than a constant (equal to 1 in this case).
• Comparison of QSVM and SVM using Gaussian and polynomial kernel functions.
Furthermore, Liu et al. performed the QSVM algorithm utilizing quadratic kernel SVM functions to improve data source authentication for wide-area synchro-phasor measurements [37] . For better understanding, SVM was introduced, and then the quadratic kernel function is involved to illustrate the process of the QSVM algorithm. To demonstrate the effectiveness of the model, Xu et al. conducted research on a proposed method based on fusing quadratic with residual forecasting [38] . Therefore, in this study, we built a long-term load-forecasting model and used SVM to optimize the core parameters of QSVM. The proposed model was analyzed and validated using actual electrical load data from Rwanda.
3. Applied SVM to Load Forecasting
Learning machine methods, such as SVM considered a promising alternative, either for multi-factors modified PSO-SVM algorithm [39] , or classification and regression [40] . Abe introduced a tutorial about SVMs applications in pattern classification and recognition problems [41] . Extending this technique to deal with regression problems, the SVM method has been considered highly competitive, and it’s possible to highlight the applications involving nonlinear gated experts for time-series forecasting [42] . The learning strategy of SVM is based on the theory of statistical learning and aims to propose a learning method that maximizes the generalization ability [43] . Besides this, the number of free parameters of SVM does not explicitly depend on the input dimensions of the problem at hand. Another important feature of SVM is that the optimization problem is unique and lacks local minima resulting from the application of the Mercer constraint when defining the kernel function [44] . However, the applicability of SVM was hampered by the need to pre-select the kernel function (responsible for mapping), the optimal parameters of the kernel function (responsible for configuration), and the loss function (penalty).
4. Long-Term Load Forecasting Background
Long-term load forecasting is a crucial component in transforming the energy system, and it is gaining traction in academics and industry. Theoretically, a load-forecasting model, in theory, seeks to quantitatively express the relationship between load and influencing parameters. As such, the model was identified with coefficients used to forecast future values by extrapolating the relationship with desired lead times. Ultimately, Essallah et al. highlighted the SVM, regularization, optimization, and beyond with kernels function [45] , the accuracy of the model depends on both the chosen model and the estimated parameters.
Literature analysis revealed that LTLF received more attention than STLF due to the complexity involved in making accurate forecasts. LTLF is based on integrating concepts from the theoretical foundations of economic theory with knowledge of finance, statistics, probability, and applied mathematics to draw conclusions about load growth, decay, and technological development. As illustrated by Zhang et al., China’s power load development is facing a new situation in which policies such as the new economic norm, industrial structure adjustment, energy conservation, and emission reduction are deeply promoted load growth [46] , Hong conducts a survey of past, current, and future trends in energy forecasting to highlight trends in spatial, short-term, LTLF, and energy price forecasting [47] . Feinberg et al. [48] and Hong et al. [49] propose three techniques suitable for long-term load forecasting, including time series, econometric, and end-use techniques. Hong et al. reported long-term probabilistic load forecasting and normalization [50] . Overestimating the long-term electrical load results in a large and wasted investment in building power infrastructure, while underestimating the future power load leads to under-production and under-demand. Distinctively, considering volatility, the Multiplicative Error Model (MEM) was used for long-term electrical load forecasting [51] whereas MEM with heterogeneous components was highlighted by Han et al. [52] . And Kobali et al. explained that LTLF plays an important role in energy systems and planning [53] . However, previous studies on LTLF are based on regression methods and cannot accurately represent energy system behavior in volatile electricity markets. A new approach for LTLF had been introduced [54] , forecasting based on economic and demographic data was applied on Türkiye [55] , using grey system theory [56] , based on Mg-CACO and SVM method [57] , Grey Feed-back modification [58] , with Neural Fuzzy model [59] , using CPSO-GM model [60] , improved recurrent neural network [61] .
5. Materials and Methods
5.1. Overview of SVM Regression and Forecasting
SVM was first proposed by Vapnik, based on small sample statistical learning theory [62] . However, it was mostly employed in the investigation of small samples according to SVM financial time series forecasting [63] , and was widely used in pattern classification and SVM time series forecasting [64] .
The dataset sample given as
, where
represents the input variables and
predicts the output variables. According to Equation (1), the SVM finds an inconsistent mapping from the input space to the output space φ. Through this mapping, data X, is mapped to a feature space Γ, and linear regression is carried out in the feature space with the following function:
(1)
where b is a threshold value, Γ is the future space. According to statistical learning theory, SVM determines the regression function by minimizing the objective function:
(2)
S.t.
where C, is a weight parameter, also called penalty factor, to balance model complexity and training error. This is pre-established to control the contribution to the complexity of each term of the regression function in Equation (2) and the resulting of SVM. The value of the parameter k is adjusted according to the residual loss function, which is an auxiliary regression function of ε, which is the insensitive loss function. In Equation (3),
is a relaxation factor and expressed as follows:
(3)
By solving the dual problem Equation (2), Lagrange factors
can be obtained, so that the regression Equation (4) coefficient is as follow:
(4)
The SVM regression equation is as follows:
(5)
For simplicity, Equations (2), (5) were usually converted into an equivalent equation defined in a dual space. Considering the parameter k, f can be determined by solving a novel quadratic optimization problem:
(6)
Subject to Equation (6),
,
,
,
, where
is a special kind of function called a kernel. It strictly follows the constraints imposed by Mercer’s theorem and provides a one-step implicit computation of intermediate products
and
.
Different learning machines with unique nonlinear decision surfaces performed depending on how this inner product core has been achieved.
Is the kernel function of SVM, which includes linear kernels, polynomial kernels, and radial basis function. The penalty factor C, the insensitive loss function ε, and the kernel function parameter σ determine the SVM performance. σ responds to the properties of the training dataset, and ε determines the complexity of the solution, and affects the generalizability of the penalty for large adjustment deviations. Too large a value can lead to over-learning, while too small a value tends to lead to under-learning, all measures minimized in the optimization procedure, which is vital in improving SVM performance.
5.2. The SVM Flowchart Model
Furthermore, the proposed Q-SVM model will be performed based on the SVM optimization results and aforesaid selected technique. For our case, we put forward a structure that is established on load data processing, selection by classification and regression, extraction, evaluation, and forecasting. SVM was used for processing and optimization for QSVM results. Besides, for model evaluation, SVM and QSVM results are assessed separately. The forecasting flowchart (Figure 1) displayed a detailed used model.
6. Exploring the Load Data
The accuracy of forecasts is highly dependent on the quality of available historical data. As shown in Table 1, hourly load data from 1998 to 2022 was extracted to sample monthly aggregate load. Monthly timestamps were used to understand the monthly load mix consumption. This study used 23 years (annual and monthly) of Rwanda’s electricity load demand and billing data load, including data collected from the period of quarter one of 1998 to the fourth quarter of 2020, and the evaluation was made annually.
In addition, the years 1998 to 2010 were chosen for the dataset training file, while the years 2011 to 2020 were picked for the dataset testing file. The following data analysis can provide relevant information that will be considered in the future phase when adjusting a forecasting model.
Figure 1. The flowchart of the modeling SVM process.
Table 1. A country’s electrical load scenario (kWh).
7. SVM Modelling and Verification
7.1. Proposed a Quadratic SVM Model
SVM with quadratic kernel function (Q-SVM) is divided into two parts: one part is for training and the other part is for testing.
➢ We apply the proposed model to the training data to make an estimate, find the forecasted value on the test data, and apply the QSVM optimization to maximize the geometric margin on all training data where the feature margin exceeds a constant.
➢ Q-SVM consists of a training phase in which the parameters are fitted to the model and a validation phase in which the performance of the fitted parameters is analyzed.
➢ We evaluate the forecasting accuracy of the models using five metric models, with the goal of formulating a multi-objective function with parameters defined in Equations (7)-(11) as follows:
• Mean Absolute Percentage Error (MAPE)
(7)
• Root Mean Square Error (RMSE):
(8)
• Mean Absolute Error (MAE):
(9)
• Maximum Absolute Percentage Error (MAP):
(10)
• Mean Square Error (MSE):
(11)
where Rt denotes the actual load and Pt denotes the forecasted load at time instant t, and N is the number of forecasters made for in a particular time interval.
Monthly pattern consumption of the energy mix, including both import and export loads, starts with low values (no production) of loads from solar energy, peat, methane gas, and exports from the first quarter of 1998 to 2008. However, peat energy holds off until 2017 for the first production. The significant increase in production is driven by hydropower and import loads, which are the dominant load patterns for multi-year load demand (refer to Figure 2).
7.2. Benchmark Model Metric Parameter Descriptions
However, the forecasting method employs actual data to build a good LTLF model. We are required to start with a large historical dataset, secondly, build models, find appropriate models, and lastly analyze anticipated results. Figure 3 illustrates the load forecasting process using Q-SVM. RMSE was chosen as the standard metric for calculating loss functions, but it is more difficult to interpret than MAE. According to [65] , lower values for MAPE, MAE, MSE, and RMSE
Figure 2. Energy mixing and electrical load series over 276 months (1998-2020).
Figure 3. SVM model for Q-SVM forecasting optimization.
indicate a more accurate regression model. Moreover, high R-squared values and low MAE are considered desirable in our case. Among that, De Myttenaere et al. [66] , highlighted metrics of a MAPE of less than 5% is considered to indicate that the forecast is fairly accurate, while a MAPE greater than 10% and less than 25% indicates poor but acceptable accuracy, finally, a MAPE greater than 25% indicates very poor accuracy.
In addition to visual inspection, an expected loss function is required to evaluate model performance and test model accuracy. Using an appropriate loss function also aims at summarizing the accuracy of point estimates and future distributions. The two loss functions used in this study are MAPE and MSE.
7.3. Proposed Model: Q-SVM Optimization Process
The LTLF for QSVM optimization model is shown in Figure 3. Following are steps required to forecast for the electricity load: load data, feature extraction, model establishment and classification, optimization, and performance evaluation.
As highlighted in Figure 4, (a) generated, (b) exported, (c) imported, and (d) total billed load forecasting.
In Figure 5, horizontal bands indicate the criterion of statistical significance at the 80% level. There is a significant autocorrelation between the residuals of the QSVM models for all hydropower load demands over 23 years, and the models represent a common load forecast. For all years of the study period, the autocorrelation of the model residuals shows the modeled dynamics, because the result plots the positive and negative lags closer to the forecast margin bar. Results are summarized in Table 2.
Figure 6 proves that the year 2003, 2004, 2005, 2009, 2010, and 2011 had recorded a significant negative (lagging), while the 2018 year recorded a positive
Figure 4. Comparison of Q-SVM forecasting for the different load series from 20 years of actual load scenario.
Figure 5. Autocorrelation of QSVM residuals in the training data set.
Table 2. Q-SVM residual load forecasting accuracy.
Figure 6. Forecasted comparison of total residual billed load.
(lead) billing residual load and the load forecast was meet. The rest of the year ended up with zero residual.
Residual loads (c) and (d) show positive prediction results, with the imported loads showing very low residual loads compared to the exported loads as indicated in Figure 7.
In Figure 8, each of the four approximations: (green) is the actual and (yellow) is the forecasted electrical load obtained with a kernel function, quadratic SVM.
As can be observed in Figure 9, horizontal bands indicate the relationship with the significance level of the model. There is a significant autocorrelation between the residuals of SVM models for hydro, methane, thermal, solar, and peat with common residual lead loads. The autocorrelation and an optimal forecasting margin (Figure 10) of the model residuals over the entire year of the inspection interval suggested the modeled dynamics and (Figure 11) depicts the quadratic SVM performance using hyper-parameter metrics across the SVM model.
According to Figure 12(a) and Figure 12(b), an SVM forecasting model presents a significant increase in residual load, which implies its difficulties in forecasting accuracy. While Figure 12(c) and Figure 12(d) figure out the QSVM forecasting model as the results of a and b (SVM) optimization response.
Figure 7. Generated electrical load, with imported (a), and exported (b) load forecasting balances.
Figure 8. Actual (hydropower vs methane gas, thermal, solar, and peat) electrical load forecast.
Figure 9. Autocorrelation of SVM residuals in the test data set.
Figure 10. Optimal forecasting margin from actual load response and predicted load response under different forecasting steps of the SVM model.
Figure 11. Minimum hyper-parameter metrics for Q-SVM forecasting performance.
Figure 12. Billing load forecast accuracy for SVM and QSVM.
Table 5 illustrates the overall results of the models, with QSVM results outperforming SVM. An R-squared of 1 for both testing and training gave superior MAPE outcomes of 5.80% and 6.17%, respectively, with faster prediction speed and longer training and testing times. SVM is statistically outperformed by the proposed approach. Furthermore, as QSVM raises PS and TT, the outcomes become more ideal.
8. Forecasting Performance, Q-SVM Optimization Results, and Discussions
Electrical distribution systems count on load forecasting, this study used the Rwanda Energy Group’s historical data on electrical loads for the last 23 years (1998-2020).
Simulation results for SVM and Q-SVM optimization show reasonable and ideal results for both models, as stipulated in Table 5 and Figure 11. To better, assess the accuracy of the model, simulations run on 276 load samples with 95% confidence intervals. A total of 30 iterations were performed for all data training and test analysis. Table 2, Table 3, Table 4, and Table 5 summarized the model
Table 3. Forecasting accuracy for different Q-SVM model parameters.
Table 4. Energy mix forecasting results.
Table 5. SVM and Q-SVM forecasting error metric results and accuracy.
forecasting results. MAPE and MSE results adhere to the benchmarking principle of billed, imported, and exported load forecast metrics. Figure 12 clearly shows positive forecast results for the billing load for QSVM compared to SVM. The model optimization is summarized as follows:
1) Simulation results are trained and tested from an actual load demand observed in various load scenarios over a specified time interval.
2) To evaluate the effectiveness of the model, the original dataset was split into training and testing phases, during the training phase, the performance of individual models is evaluated.
3) Compare methods to their respective benchmarks using MAPE and MSE error metrics.
The best possible values of Dtrain and Dtest given in Table 2 and Table 5 are used as constraints for a variable multi-objective function with model parameters. RMSE, MAE, MAPE, and R-squared have been applied as constraints on the ratios of other variable metrics. While a hyper-parameter as a metric benchmark model utilized during Q-SVM optimization to assess its efficiency. Epsilon and kernel scales, minimum error hyper-parameters, the best point hyper-parameters, and training phase solutions are provided in Table 5. This research will enable REG to study dynamic increases in electric load demand patterns and facilitate continuity planning for more appropriate and accurate load generation and expansion strategies. Consequently, inaccurate forecasts can lead to power shortages and surpluses, leading to “dumsor” and unnecessary threats to the power system.
9. Conclusion and Future Work
Electricity load forecasting is key to promoting energy equity and integration across households in the country. The purpose of this study on Rwanda is to achieve universal energy access by 2024. The proposed artificial intelligence model (QSVM) aims to solve the long-term electricity demand forecasting problem from 1998 to 2020. QSVM was optimized by SVM results to accurately formulate the correlation between historical load data and forecasted load. Note that experimental results show that the QSVM model is applicable to LTLF due to its higher prediction accuracy and significantly lower error metric compared to those of the SVM model. Compared with other long-term forecasting methods and models, the proposed models use less prediction speed and lower training time. By setting the forecasting process reasonably, the QSVM load forecasting effect is more accurate than the current application methods. In spite of its higher forecasting accuracy, this study has some limitations that lead to unconformity of model metrics performance: 1) small sample size, 2) hourly dataset arrangement. In future work, with hourly dataset arrangement, the model accuracy will be improved and compared with PSO and ANN for the Middle-Term Load Forecasting method.