ABSTRACT

This article is devoted to a time series prediction scheme involving the nonlinear autoregressive algorithm and its applications. The scheme is implemented by means of an artificial neural network containing a hidden layer. As a training algorithm we use scaled conjugate gradient (SCG) method and the Bayesian regularization (BReg) method. The first method is applied to time series without noise, while the second one can also be applied for noisy datasets. We apply the suggested scheme for prediction of time series arising in oil and gas pricing using 50 and 100 past values. Results of numerical simulations are presented and discussed.

Keywords:

Nonlinear Autoregression, Time Series, Prediction, Data Analysis, Deep Learning, Scaled Conjugate Gradient Method, Bayesian Regularization Method

1. Introduction

Any branch of science which deals with observational data, requires a repeatedly usage of the mathematical concept of time series. Time series are formed by discrete measurements of a specific quantity at successive time instants. They are used in such disciplines as statistics [1] including statistical physics and mechanics; astronomy including astrophysics, celestial mechanics and cosmology [2] ; oceanography [3] ; econometrics and mathematical finance including actuarial mathematics [4] [5] ; meteorology and climatology [6] ; seismology [7] ; biology [8] ; engineering [9] ; earth science and geomechanics [10] , etc. Examples of time series include (in an order corresponding to the disciplines above) positions of particles under Brownian motion; tidal heights; assets of a company; speed and direction of a weather flow; seismic activity; population of species; load-deformation dependence of structures; displacement of the crust of the Earth.

The mathematical analysis of time series is a very important branch of statistics. A proper mathematical analysis may reveal the most important features of the temporal measurements and use the output to make a meaningful insight. The most desirable insight might be a prediction of future behavior of a phenomena/process based on the past data. We learn that from the ancient time humans were able to do astronomical, meteorological, seismic, etc. predictions. For example, based on the seismic activity of a specific region, many centuries ago people on different continents were able to predict a time frame when the highest-risk seismic behavior will occur in that region and thus had an opportunity for safe evacuation. Another example, the analysis of astronomical data gathered by known astronomers such as Tycho Brahe, Johann Kepler was able to formulate his law, which was laid in the foundations of celestial mechanics much later.

All this and much more are now easily performed based on the fundamental methods of contemporary statistical analysis. One of the most powerful methods allowing analyzing time series and make predictions, is the nonlinear auto-regression algorithm (NAR). For a detailed introduction into the subject, see [11] [12] and the related references therein. The algorithm was developed to analyze periodically sampled data [13] [14] (more often equally or regularly spaced term is used). However, recently the algorithm has been extended to be applicable for analyzing equally or regularly spaced data [15] . The algorithm and its modifications are used by several authors to train artificial neural networks for predictive analysis of time series with many applications [16] [17] .

In this article we apply the nonlinear autoregressive model for a predictive analysis of time series arising in gas and oil pricing. The data used in this paper are freely available at the Macrotrends webpage¹. The paper is organized as follows. We first briefly describe the nonlinear autoregressive model we use in Section 2. In Section 3 we present the results of numerical simulations performed in MatLab. In particular, among others, it is established how the performance of the algorithm for a specific time series depends on the number of data used for forecasting future values of the series.

2. Nonlinear Autoregressive Model: Main Relations and Characteristics

The nonlinear autoregressive model is defined for an order $p\in \mathbb{N}$ . Thus, the nonlinear autoregressive model of order p is defined as [17]

$f\left(t\right)=F\left(f\left(t-1\right),f\left(t-2\right),\cdots ,f\left(t-p\right)\right)+\epsilon \left(t\right),$

where $f\left(t\right)$ is the current value and, apparently, $f\left(t-i\right),i=1,2,\cdots ,p$ is the i^th past value of the time series, F is a nonlinear function defining the dependence of the current value on the past p values of the time series, $\epsilon$ is a white noise. This means that every current value depends on the previous p values.

The main aim of any method of statistical analysis is the low-error approximation of F. In the simplest case, F is a linear function, leading to

$f\left(t\right)=F_0+{\displaystyle \sum _{i=1}^p}{F}_{i}f\left(t-i\right)+\epsilon \left(t\right),$

where $F_0$ and $F_i,i=1,2,\cdots ,p$ are given constants.

Structure of the Nonlinear Autoregressive Model Based Artificial Neural Network

The nonlinear autoregressive model based neural network, is a feed-forward network aiming to approximate F in the above definition. The feed-forward algorithm is defined by [17]

$\widehat{f}\left(t\right)=\widehat{F}\left(f\left(t-1\right),f\left(t-2\right),\cdots ,f\left(t-p\right)\right),$ (1)

$\widehat{f}\left(t\right)={\alpha }_0+{\displaystyle \sum _{i=1}^N}{\alpha }_{i}A\left({\beta }_k+{\displaystyle \sum _{k=1}^p}{w}_{ik}f\left(t\right)\right),$ (2)

where ${\alpha }_i,i=1,2,\cdots ,N$ are constants, A is the activation function, $w_{ik}$ are the weights, ${\beta }_k$ are the biases.

NAR methods are efficiently used for forecasting as deterministic models, as well as stochastic models. In this study, we consider a three-layer neural network, in which the feed-forward algorithm (1), (2) is learned using the well-known supervised learning algorithm scaled conjugate method developed in [18] , as well as the Bayesian regularization presented in [19] in details.

3. Application of the Nonlinear Autoregressive Model for Prediction of Gas and Oil Prices

Let us proceed with the implementation of the algorithm described in the previous two sections. For simplicity, let the time series of past values be composed of 50 data. This means that the nonlinear autoregressive model of order p = 50 must be involved. To measure the error between the target and output, we use the mean square error function

$\text{Er}\left(o_i\mathrm{,}{t}_i\right)={\displaystyle \sum _{i=1}^N}{\left(o_i-t_i\right)}^2,$

with $o_i$ being the outputs and $t_i$ being the targets. We also compute the correlation coefficient to compare the statistical properties of the analysis for different order NAR models:

$C=\frac{1}{\text{Er}\left(o_i\mathrm{,}\overline{o}\right)\cdot \text{Er}\left(t_i\mathrm{,}\overline{t}\right)}{\displaystyle \sum _{i=1}^N}\left(o_i-\overline{o}\right)\left(t_i-\overline{t}\right)\mathrm{.}$

As a training algorithm we use the scaled conjugate gradient algorithm. After 19 iterations (epochs), the performance of the neural network is plotted on Figure 1. At this, the best performance occurs at 13^th iteration and is equal to 1512.9115. The gradient and validation check results are plotted in Figure 2. It is evident that the gradient is a locally decreasing function of epochs, which means that if more iterations are involved, the error should be decreased. On the other hand, we see that the validation checks increase when the number of epochs increases.

From Figure 3, where the error histogram is plotted, it is easy to observe that near the zero error (i.e., when the target and output are equal) the training errors for positive difference is comparably higher than for negative difference. It is also noteworthy that the training regression plot shows a high efficiency of the fitting tool of the algorithm (see Figure 4).

Finally, let us summarize the main result of the numerical simulation. As it is

shown in Figure 5, there is a specific interval containing both target and outputs. In other words, the analysis based on the nonlinear autoregression algorithm of order p = 50, allows predicting future data with a specific accuracy. Increasing the model order, generally, it will be possible to make the prediction more accurate. However, it is not always the case because of the overfitting phenomenon. Nevertheless, it is the case in the present study. Indeed, Figures 7-12 show that the training, performance and output parameters are better compared with those for p = 50. Note that in this case 55 iterations (epochs) are performed. Finally note that, as it is seen from Figure 6 and Figure 12, the error autocorrelation is lower when p = 100.

4. Conclusion

In this article we show the efficiency of the nonlinear autoregression algorithm based artificial neural network in time series analysis and prediction. As a particular model we choose the time series generated by daily prices of gas and oil (the data are freely available at www.mactrotrends.com). The artificial neural network consists of 3 layers: an input layer, a hidden layer, an output layer. Using the scaled conjugate gradient method to learn the neurons, we test the algorithm for 50 and 100 number of past values. The main characteristics of the neural network performance are reported. In particular, a significant improve is observed in validation and regression, the gradient is decreased, the target/output error is decreased almost two time and autocorrelation error is decreased almost 2.5 times.

Acknowledgements

We express our sincere gratitude to anonymous referees for their critical remarks that helped to improve the presentation of the material.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

Cite this paper

Raturi, R. and Sargsyan, H. (2018) A Nonlinear Autoregressive Scheme for Time Series Prediction via Artificial Neural Networks. Journal of Computer and Communications, 6, 14-23. https://doi.org/10.4236/jcc.2018.69002

References

NOTES

¹http://www.macrotrends.net/1369/crude-oil-price-history-chart