ANN-Time Varying GARCH Model: Simulations and Application in Modelling Temperature for Weather Derivatives

Elias K. Karuiru^1*, John Mwaniki Kihoro², Thomas Mageto³, Anthony Gichuhi Waititu³

¹The Pan African University Institute for Basic Sciences, Technology and Innovation (PAUSTI), Nairobi, Kenya.
²School of Computing and Mathematics, Co-Operative University, Nairobi, Kenya.
³Department of Statistics and Actuarial Sciences, JKUAT, Nairobi, Kenya.

DOI: 10.4236/ojs.2022.123027                

Abstract

In economics and finance, minimising errors while building an abstract representation of financial assets plays a critical role due to its application in areas such as risk management, decision making and option pricing. Despite the many methods developed to handle this problem, modelling processes with fixed and random periodicity still remains a major challenge. Such methods include Artificial Neural networks (ANN), Fuzzy Inference system (FIS), GARCH models and their hybrids. This study seeks to extend literature of hybrid ANN-Time Varying GARCH model through simulations and application in modelling weather derivatives. The study models daily temperature of Kenya using ANN-Time Varying GARCH (1, 1), Time Lagged Feedforward neural network (TLNN) and periodic GARCH family models. Mean square error (MSE) and coefficient of determination R² were used to determine performance of the models under study. Results obtained show that the ANN-Time Varying GARCH model gives the best results.

Keywords

Artificial Neural Network, Time Varying GARCH, Weather Derivatives, Temperature

1. Introduction

Weather influences many economic activities and livelihood of people. Some of the sectors mainly affected by weather include energy production and consumption and production of agricultural commodities among others. In the recent times, people have established a weather derivative which is a new type of security to help in hedging risks against weather driven poor performance in business activities. The payoffs of these instruments may be linked to various weather related variables, including heating degree days, cooling degree days, maximum temperature, minimum temperature, humidity, sunshine and precipitation [1]. Weather forecasting is getting more and more crucial to guiding people’s activities and even to government, like setting disaster prevention budget such as hunger prevention cost [2].

Weather derivative differs from financial derivative due to some interesting considerations. First, weather is not traded in spot market. Second, weather derivatives are critical in quantity hedging but not necessarily hedging price as in financial derivative. That is, weather derivative products provide protection against weather-related changes in quantities, complementing extensive commodity price risk management tools already available through futures [1]. Third, weather is naturally a location specific and nonstandardised commodity. Further, weather forecasting is crucial to both the demand and the supply sides of the weather derivatives market. In this study, we take a nonstructural time series approach to temperature modeling and forecasting.

Modelling daily average temperature has attracted many people who as well have developed many methods. Autoregressive Moving Average (ARMA) model and its various forms are the oldest commonly used method despite its shortcomings. Autoregressive Conditional Heteroskedasticity (ARCH) model and its family are also another commonly used modeling volatility in time series. ARCH is a stationary non linear model. An ARCH (q) first models the stationary process by an AR (q) model, and takes the variance of the residual term as a q-th autoregressive polynomial relating to the history squared residuals back to lag q. The model was first proposed by Engle in 1982 and latter generalised by Bollerslev and Taylor in 1986 and termed as Generalised Autoregressive Conditional Heteroskedasticity (GARCH) model.

These models have improved the application range to the real world problems in time series. However, in periodic phenomena, randomness and periodicity estimation is still considered separately. The GARCH model for example fits a periodic function to the seasonal trend and adds it to the ARMA type process of the residuals linearly. This is hence seen as a limitation of the model.

Recently, time series modelling and forecasting have attracted the use of ANNs. ANNs do not assume any statistical distribution and they exhibit excellent attributes in non linear modelling [3]. They develop the appropriate model on the given data adaptively. They are hence considered to be data driven and self adaptive. Literature has been developed over the past decades towards the ANN application in time series forecasting. Zhang et al. (1998) presented a very comprehensive discussion on ANN application in time series forecasting. Multi-Layer Perceptions (MLPs) are the most popular models of ANNs. Hamzacebi (2008) developed SANN model for forecasting seasonal time series. This model has been proved to be quite successful when applied in forecasting seasonal time series [4]. The model does not require data pre-processing and it learns the data patterns adaptively without removing them as opposed to other traditional approaches discussed earlier.

This study uses ANN-Time Varying GARCH (1, 1) model for processes with fixed and random periodicity proposed by [5] in modelling daily average temperature of Kenya from 1991 to 2016. The performance of ANN-Time Varying GARCH (1, 1) model is compared with that of four other models using MSE and R². The models include; Periodic-ARMA, Periodic-ARCH, Periodic-GARCH and Time Lagged Feedforward Neural Network models.

2. Methodology

2.1. ARMA Model

The process $X\left(t\right),t=0,\pm 1,\pm 2,\cdots$ is said to be an ARMA (p, q) process if $X\left(t\right)$ is stationary and for each t,

$X\left(t\right)-{\varphi }_{1}X\left(t-1\right)-\cdots -{\varphi }_{p}X\left(t-p\right)=Z\left(t\right)+{\theta }_{1}Z\left(t-1\right)+\cdots +{\theta }_{q}Z\left(t-q\right)$ (1)

where $Z\left(t\right)~WN\left(\mathrm{0,}{\sigma }^2\right)$.

Conveniently, ARMA (p, q) can also be written using the lag operator B as follows

$\varphi \left(B\right)X\left(t\right)=\theta \left(B\right)Z\left(t\right)$ (2)

where $\varphi \left(\mathrm{.}\right)$ and $\theta \left(\mathrm{.}\right)$ are the p^th and q^th degree polynomials.

A unique stationary solution of $\text{ARMA}\left(p\mathrm{,}q\right)$ model exists if

$\varphi \left(c\right)=1-{\varphi }_{1}c-\cdots -{\varphi }_p{c}^p\ne 0$ (3)

for all $\left|c\right|=1$.

An ARMA (p, q) model is invertible if

$\varphi \left(c\right)=1+{\varphi }_{1}c+\cdots +{\varphi }_p{c}^p\ne 0$ (4)

for all $\left|c\right|\le 1$.

2.2. Periodic Autoregressive Moving Average Model

The periodic autoregressive moving average process $X\left(t\right)$ of order p and q denoted as PARMA (p, q) with period s has representation

$X\left(t\right)-{\displaystyle \underset{i=1}{\overset{p}{\sum }}}{\varphi }_{il}X\left(t-i\right)={\displaystyle \underset{i=0}{\overset{q}{\sum }}}{\theta }_{il}Z\left(t-i\right)$ (5)

where $Z\left(t\right)~WN\left(\mathrm{0,}{\sigma }^2\right)$ and the parameter ${\varphi }_{il}$ and ${\theta }_{il}$ for $l=1,2,3,\cdots ,s$ are the autoregressive and moving average parameters respectively.

2.3. ARCH Model

A stochastic model $X_t$ is said to be an ARCH (p) if

$X_t={\sigma }_t{Z}_t,$ (6)

where

${\sigma }_t=\sqrt{\omega +{\displaystyle \underset{i=1}{\overset{p}{\sum }}}{\varphi }_i{X}_{t-i}^2}$ (7)

is the conditional standard deviation of $X_t$ given the past values of this process and $Z_t$ is a Gaussian white noise with unit variance.

The properties of ARCH (p) model include:

1) $E\left[X\left(t\right)\mathrm{<mo>\; |\; </mo>}{\mathcal{F}}_{t-1}\right]=0$.

2) $Var\left[X\left(t\right)\mathrm{<mo>\; |\; </mo>}{\mathcal{F}}_{t-1}\right]={\sigma }_t^2$.

ARCH models are used to describe a changing, possibly volatile variance. The estimation of ARCH (p) parameters is done using maximum likelihood estimation method. The generalization of ARCH (p) model by extension with autoregressive terms of the volatility gives rise to the GARCH model.

2.4. GARCH Model

A stochastic model $X_t$ is said to be an GARCH (p, q) if

$X_t={\sigma }_t{Z}_t,$ (8)

where

${\sigma }_t=\sqrt{\omega +{\displaystyle \underset{i=1}{\overset{p}{\sum }}}{\varphi }_i{X}_{t-i}^2+{\displaystyle \underset{i=1}{\overset{q}{\sum }}}{\theta }_i{\sigma }_{t-i}^2}$ (9)

is the conditional standard deviation of $X_t$ given the past values of this process and $Z_t$ is a Gaussian white noise with unit variance. The properties of GARCH (p, q) model include:

1) $E\left[X\left(t\right)\mathrm{<mo>\; |\; </mo>}{\mathcal{F}}_{t-1}\right]=0$.

2) $Var\left[X\left(t\right)\mathrm{<mo>\; |\; </mo>}{\mathcal{F}}_{t-1}\right]={\sigma }_t^2$.

Because past values of the ${\sigma }_t$ process are fed back into the present value, the conditional standard deviation can exhibit more persistent periods of high or low volatility than seen in an ARCH process.

2.5. Periodic GARCH (p, q) Model

The periodic-GARCH process $X\left(t\right)$ of order p and q denoted as P-GARCH (p, q) with period s has representation

$X_t={\sigma }_t{Z}_t,$ (10)

where

${\sigma }_t=\sqrt{{\omega }_l+{\displaystyle \underset{i=1}{\overset{p}{\sum }}}{\varphi }_{il}{X}_{t-i}^2+{\displaystyle \underset{j=1}{\overset{q}{\sum }}}{\theta }_{jl}{\sigma }_{t-j}^2}$ (11)

is the conditional standard deviation of $X_t$ given the past values of this process and $Z_t$ is a Gaussian white noise with unit variance. The parameters ${\omega }_l$, ${\varphi }_{il}$ and ${\theta }_{jl}$ for $l=1,2,\cdots ,s$ vary with period s.

2.6. Time Lagged Neural Networks (TLNN)

TLNN is one of the artificial neural networks models applied in time series modelling and forecasting. The model takes the lagged values upto to lag s as the input values where s is the period of the time series. The forecasted value therefore only depends on the past s values. In order to avoid the need of adding a bias term, a constant input term which may be taken to be 1 is connected to every neuron in hidden and output layer [4]. For a TLNN with one hidden layer, the general prediction equation for computing a forecast may be written as [6]:

$X_t={\Phi }_0\left\{W_{c0}+{\displaystyle \underset{j}{\sum }}{W}_{j0}{\Phi }_j\left\{W_{cj}+{\displaystyle \underset{i}{\sum }}{W}_{ij}{X}_{t-i}\right\}\right\}+Z_t$ (12)

where, $X_{t-i}$ are input terms, $W_{cj}$ are the weights for the connections between the constant input and hidden neurons and $W_{c0}$ is the weight of the direct connection between the constant input and the output. Also $W_{ij}$ and $W_{j0}$ denote the weights for other connections between the input and hidden neurons and between the hidden and output neurons respectively. ${\Phi }_j$ and ${\Phi }_0$ are the hidden and output layer activation functions respectively.

2.7. ANN-Time Varying GARCH Model

Define the process with fixed periodicity s and random periodicity $\tau$ as $Y\left(t\right)$ to be an ANN-Time Varying GARCH process, if for each t,

$Y_t={\mu }_t+R_t,$ (13)

where

${\mu }_t={\alpha }_0+{\displaystyle \underset{j=1}{\overset{m}{\sum }}}{\alpha }_{j}f\left\{V_j+{\displaystyle \underset{i=1}{\overset{d}{\sum }}}{V}_{ij}{Y}_{t-is}\right\},$ (14)

$R_t={\sigma }_t{Z}_t,$ (15)

such that

${\sigma }_t=\sqrt{w\left(t\right)+{\displaystyle \underset{i=1}{\overset{p}{\sum }}}{\varphi }_i\left(t\right)R_{t-i}^2+{\displaystyle \underset{i=1}{\overset{q}{\sum }}}{\theta }_i\left(t\right){\sigma }_{t-i}^2}$ (16)

and $Z_t$ is an i.i.d white noise. Therefore, the model is an ANN model with Time Varying GARCH disturbances. Where,

1) ${\alpha }_j$ ( $j=1,2,\cdots ,m$ ) and $V_{ij}$ ( $j=1,2,\cdots ,m$ and $i=1,2,\cdots ,d$ ) are the hidden and input connection weights respectively.

2) ${\alpha }_0$ and $V_j$ ( $j=1,2,\cdots ,m$ ) are the output and hidden layers connection bias respectively.

3) d is the number of nodes in the input layer.

4) m is the number of nodes in the hidden layer.

5) f is the hidden layer transfer function.

6) $R_t$ is a random periodic process.

7) $w\left(\mathrm{.}\right)$, $\varphi \left(\mathrm{.}\right)$ and $\theta \left(\mathrm{.}\right)$ are non negative functions of time t.

Hence, the ANN model of (14) in fact performs a nonlinear functional mapping from the past observations $Y_{t-s}\mathrm{,}{Y}_{t-2s}\mathrm{,}{Y}_{t-3s}\mathrm{,}\cdots$ to the future value $Y_t$ therefore making connection weights of the model vary with period s.

Equation (15) is a Time Varying GARCH (p, q) process with time varying parameters $w\left(t\right)\mathrm{,}{\varphi }_i\left(t\right)$ and ${\theta }_i\left(t\right)$.

The estimation of parameters for the model is done through non parametric techniques as discussed by [5].

Simulation Study of ANN-Time Varying GARCH Model

We carried out a simulation study to judge the performance of ANN-Time Varying Garch model and the estimation procedure proposed in [5]. For computational simplicity, we used ANN-Tv GARCH (1, 1) model in our simulations. A sample of size $n=1000$ was generated from the following model:

$Y_t={\mu }_t+R_t,$ (17)

where

${\mu }_t={\alpha }_0+{\displaystyle \underset{j=1}{\overset{10}{\sum }}}{\alpha }_{j}f\left\{V_j+{\displaystyle \underset{i=1}{\overset{20}{\sum }}}{V}_{ij}{Y}_{t-i20}\right\},$ (18)

$R_t={\sigma }_t{Z}_t,$ (19)

such that

${\sigma }_t=\sqrt{w\left(\frac{t}{n}\right)+\varphi \left(\frac{t}{n}\right)R_{t-1}^2+\theta \left(\frac{t}{n}\right){\sigma }_{t-1}^2}$ (20)

and $Z_t$ is an i.i.d white noise, $t=1,2,\cdots ,1000$, $w\left(u\right)=2u\left(1-u^2\right)+0.1$, $\varphi \left(u\right)=0.2\cos \left(2\pi u\right)+0.25$, $\theta \left(u\right)=2u\left(1-u\right)+0.2u^3$ and $0<u\le 1$. The number of input nodes in the input layer was fixed at 20 while the number of neurons in the hidden layer is fixed at 10. The hyperbolic tangent activation functionf was adopted and applied. The Quasi Newton training method also known as BFGS algorithm was adopted in the training the artificial neural network. Gaussian kernel was used in local polynomial estimation of the parameter functions. Cross validation method is used in selecting the bandwidth. A plot of the simulated model is shown in Figure 1 below.

Figure 1. Simulated data from ANN time varying GARCH (1, 1) model.

The parameters were estimated using the local polynomial techniques as proposed in [5]. Plots of the estimated parameters are as shown in Figure 2 below.

Figure 2. Plots of actual (Red) and estimated (Green) parameters.

From the simulation, it is evident that parameters estimation techniques as proposed in [5] are workable and gives consistent estimates.

3. Data and Analysis

3.1. Data

The data set used in this case is the daily maximum temperature of Kenya obtained from Meteorological department. Plot of the data set is as shown in Figure 3 and its descriptive statistics is as shown in Table 1 below.

Figure 3. Central Kenya monthly temperature.

Table 1. Descriptive statistics of temperature data set.

Sample: Jan 1991 to Jan 2016

Testing for periodicity in the data set using the Fisher’s g test statistic gives a p-value of 2.091002e−56 indicating that the time series is highly periodic. After deseasoning the time series and repeating the Fisher’s g test it gives a p-value of 1.911331e−11 indicating that the de-seasoned time series is still periodic. Therefore, we conclude that the daily average temperature time series have fixed and random periodicity.

3.2. Data Analysis

The P-ARMA, P-ARCH, P-GARCH, TLNN and ANN-TvGARCH models were fitted to the temperature data set and the fitted values plotted against the actual temperature values for comparison. Figure 4 below shows the plot of actual temperature values as compared to the fitted values.

Figure 4. Plot of Actual (black) Temperature against P-ARCH (blue), P-GARCH (red), TLNN (Yellow) and ANN-TvGARCH (Green) fitted temperature.

Mean square error (MSE) which is the average of the square of the errors was used to compare the performance of the models. The larger the number the larger the error. Error in this case means the difference between the observed temperature values and the fitted temperature values from every model.

$\text{MSE}=\left(\frac{1}{n}\right){\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left(Y_i-{\hat{Y}}_i\right)}^2$ (21)

The coefficient of determination R² is a measure that provides information about the goodness of fit of a model. In the context of time series it is a statistical measure of how well the fitted model approximates the actual data. Generally, a higher R² indicates a better fit for the model.

$R^2=1-\left(\frac{{\displaystyle {\sum }_{i=1}^n{\left(Y_i-{\hat{Y}}_i\right)}^2}}{{\displaystyle {\sum }_{i=1}^n{\left(Y_i-{\bar{Y}}_i\right)}^2}}\right)$ (22)

It is evident that ANN-Time Varying GARCH model outperforms all the other models in terms of lowest MSE and highest R². Table 2 below shows the value of MSE and R² as calculated from all the models.

Table 2. Comparison of models MSE and R².

The model performance results show that the ANN-TvGARCH (1, 1) model out performs the P-ARMA, P-ARCH (1), P-GARCH (1) and TLNN models in modelling daily average temperature since it has the smallest MSE and the largest R².

4. Conclusion

This study simulates ANN-TvGARCH (1, 1) model and models daily average temperature of Kenya using ANN-TvGARCH (1, 1), P-ARMA, P-ARCH (1), P-GARCH (1, 1) and TLNN models. The ANN-TvGARCH (1, 1) simulations prove that its parameters estimation techniques proposed by [5] are workable. The performance of models in modelling daily average temperature in Kenya is determined using MSE and R² measures. Results obtained indicate that ANN-TvGARCH (1, 1) models give the best estimates. The ANN-TvGARCH model therefore proves to be a superior model in modelling processes with fixed and random periodicity like weather derivatives. Finally, further work can be done on the construction of confidence intervals for the ANN-TvGARCH (p,q) parameters estimators as well as integrating new methods in artificial neural networks.

Acknowledgements

This research work was funded by the African union commission. The authors would like to thank the African Union commission for the support.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Campbell, S.D. and Diebold, F.X. (2005) Weather Forecasting for Weather Derivatives. Journal of the American Statistical Association, 100, 6-16. https://doi.org/10.1198/016214504000001051
[2]	Liu, Y.J. (2019) ARMA Model for Random Periodic Processes. Loughborough University, Loughborough.
[3]	Kihoro, J., Otieno, R. and Wafula, C. (2004) Seasonal Time Series Forecasting: A Comparative Study of ARIMA and ANN Models. African Journal of Science and Technology, 5, 41-49.
[4]	Adhikari, R. and Agrawal, R. (2013) An Introductory Study on Time Series Modeling and Forecasting. arXiv:1302.6613.
[5]	Karuiru, E.K., Kihoro, J.M., Mageto, T. and Waititu, A.G. (2021) ANN-Time Varying GARCH Model for Processes with Fixed and Random Periodicity. Open Journal of Statistics, 11, 673-689. https://doi.org/10.4236/ojs.2021.115040
[6]	Faraway, J. and Chatfield, C. (1998) Time Series Forecasting with Neural Networks: A Comparative Study Using the Air Line Data. Journal of the Royal Statistical Society: Series C (Applied Statistics), 47, 231-250. https://doi.org/10.1111/1467-9876.00109