Predicting stock high price using forecast error with recurrent neural network

Artificial Neural Networks (ANN) [10, 11] is a research hot spot in the AI circle. It simulates the network structure of the human brain, and different network models can be constructed according to different connection methods. It is often referred to directly as the Neural Network (NN).

Multi-Layer Perceptron (MLP) [12,13] refers to a forward-structured artificial neural network. Its function is to map input vectors group to output vectors group, as shown in Figure 1. MLP can be considered to be a directed graph composed of multiple node layers, among which each node connects to the next layer. Every node besides input nodes is a neuron with non-linear activation. A supervised learning method called a back-propagation algorithm is usually used to train MLPs.

Fig. 1

In the 1980s, MLP was a popular method with various applications, including image recognition, machine translation, etc. In recent years, deep learning becomes the focus of people’s attention, and MLP attracts people’s attention again.

RNN [4, 5] is an artificial neural network in which the connected nodes form a directed graph along a sequence. The RNN can use its internal state as input, enabling it to display the time series’ temporal dynamic behaviour and thus complete tasks such as handwriting recognition or speech recognition.

RNN is an easy tool with which to process sequence data. The input of the hidden layer comes from the output of both the input layer and the previously hidden layer, as shown in Figure 2. In theory, the RNN can process sequence data of any length. However, in practice, to reduce complexity, the current state is generally assumed to only relate to certain previous states.

Fig. 2

In contrast with the traditional machine learning model, hidden layer units are thoroughly equal. The hidden layer in the RNN is a time series from left to right. During the analysis, we often extend the RNN in time to get the structure shown in Figure 3.

Fig. 3

The LSTM [6, 7] network is an RNN that can process and predict important events with longer intervals and delays in time series. LSTM is often applied in the technology field in various ways. The systems based on LSTM can learn tasks such as translation language, speech recognition, handwriting recognition, predictive disease, and stock forecasting.

The LSTM and RNN are different in that a ‘processor’ is added to the algorithm to identify useful information in the LSTM case. The structure of this processor is called a unit. The three doors are arranged in a unit named the input gate, the forgetting gate and the output gate, as shown in Figure 4. When messages enter the LSTM network, they will be recognised under the rules. If the information follows the algorithm certification, it will be left. Otherwise, the unmatched information will be discarded through the Forgotten Gate. LSTM is very efficient in solving long-order dependency, which is highly versatile and has many possibilities.

Fig. 4

A typical model of LSTM is defined as follows: (1) ft=σ(Wf⋅[ht-1,xt]+bf) {f_t} = \sigma \left({{W_f} \cdot \left[{{h_{t - 1}},{x_t}} \right] + {b_f}} \right) (2) it=σ(Wi⋅[ht-1,xt]+bi) {i_t} = \sigma \left({{W_i} \cdot \left[{{h_{t - 1}},{x_t}} \right] + {b_i}} \right) (3) C˜t=tanh(Wc⋅[ht-1,xt]+bc) {\tilde C_t} = \tanh \left({{W_c} \cdot \left[{{h_{t - 1}},{x_t}} \right] + {b_c}} \right) (4) Ct=ft*Ct-1+it*C˜t {C_t} = {f_t}*{C_{t - 1}} + {i_t}*{\widetilde C_t} (5) ot=σ(Wo⋅[ht-1,xt]+bo) {o_t} = \sigma \left({{W_o} \cdot \left[{{h_{t - 1}},{x_t}} \right] + {b_o}} \right) (6) ht=ot*tanh(Ct) {h_t} = {o_t}*\tanh \left({{C_t}} \right)

x_t means the input vector at time t, h_t means the output vector, c_t means the memory cell state, i_t means the input gate vector, f_t means the forget gate vector, o_t means the output gate vector, W_i, W_f, W_o and W_c mean the weight matrices, b_i, b_f, b_o and b_c mean the bias vector and σ means activation function.

GRU [8, 9] is a gating mechanism in a recurrent neural network. The GRU looks like an LSTM carrying two gates but with a fewer number of parameters, as shown in Figure 5. GRU’s performance in music modelling and speech signal modelling is similar to that of LSTM. GRU excels LSTM on the performance of some smaller data sets.

Fig. 5

A typical model of GRU is defined as follows: (7) rt=σ(Wr⋅[ht-1,xt]+br) {r_t} = \sigma \left({{W_r} \cdot \left[{{h_{t - 1}},{x_t}} \right] + {b_r}} \right) (8) zt=σ(Wz⋅[ht-1,xt]+bz) {z_t} = \sigma \left({{W_z} \cdot \left[{{h_{t - 1}},{x_t}} \right] + {b_z}} \right) (9) h˜t=tanh(Wh⋅[rt*ht-1,xt]+bh) {\tilde h_t} = \tanh \left({{W_h} \cdot \left[{{r_t}*{h_{t - 1}},{x_t}} \right] + {b_h}} \right) (10) ht=(1-zt)*ht-1+zt*h˜t {h_t} = {\left({1 - {z_t}} \right)^*}{h_{t - 1}} + {z_t}*{\tilde h_t}

x_t is the input vector at time t, h_t is the output vector, r_t is the reset gate vector, z_t is the update gate vector, W_r, W_z and W_h are the weight matrices, b_r, b_z and b_h are the bias vector and σ is the activation function.

To assess the forecasting effect of the proposed model, the results were tested by some evaluation criteria, which are a Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE) and Mean Absolute Scaled Error (MASE), as shown in Eqs (11)–(13) [14]. The RMSE, MAPE and MASE represent the difference between the values y_t and y^p_t, where y_t is the actual value of the t-th sample, y^p_t is the forecast value of the t-th sample and m is the sample size of the test set. Thus, the smaller their values are, the better the model performs. The predictive directionality index (Dstat) is defined by Eq. (14). Dstat indicates the consistency between the actual trend and the predicted trend, and a greater degree of consistency is indicative of a better result. (11) RMSE=∑t=1m(ypt-yt)2m RMSE = \sqrt {{{\sum\limits_{t = 1}^m {{\left({{y^p}t - {y_t}} \right)}^2}} \over m}} (12) MAPE=1m(∑t=1m|ytp-ytyt|)×100% MAPE = {1 \over m}\left({\sum\limits_{t = 1}^m \left| {{{y_t^p - {y_t}} \over {{y_t}}}} \right|} \right) \times 100\% (13) MASE=1m∑t=1m|ytp-yt|1m-1∑t=2m|yt-yt-1| MASE = {1 \over m}{{\sum\limits_{t = 1}^m \left| {y_t^p - {y_t}} \right|} \over {{1 \over {m - 1}}\sum\limits_{t = 2}^m \left| {{y_t} - {y_{t - 1}}} \right|}} (14) Dstat=1m-1∑t=2mat,at={1,if(yt-yt-1)(ytp-yt-1p)>00, otherwise {\rm{Dstat}} = {1 \over {m - 1}}\sum\limits_{t = 2}^m {a_t},{a_t} = \left\{{\matrix{{1,if\left({{y_t} - {y_{t - 1}}} \right)\left({y_t^p - y_{t - 1}^p} \right) > 0} \hfill \cr {0,\quad {\rm{otherwise}}} \hfill \cr}} \right.

The Shanghai Composite Index (000001.SH) of China is used as experimental data. The daily data includes six variables, such as opening price, high price, low price, closing price, change of price and a number of transactions.

The data from the previous few days is used as input data, and the high price of the next date is used as the output data. In the emulations, the stock prices from June 2010 to August 2017 are used as training data (as shown in Figure 8) and those from September 2017 to April 2018 are used as test data (as shown in Figure 9).

Fig. 8

Fig. 9

The experiments were implemented in Python 3.5.3; and measured on a computer with Intel(R) Core(TM) i7-6700 CPU at 3.40 GHz, 8.0 GB RAM, and Microsoft Windows10 Professional 64 bits.

Tables 1–3 show the experiment results. We tested each sample for more than 10 times independently; we obtained the average values of 10 individuals. “Time” refers to the mean test time; the unit of time is in seconds.

In Table 1, the NN1 is LSTM, and the NN2 is LSTM, GRU or MLP, respectively. The values of the error-index (RMSE, MAPE, MASE) of proposed models are smaller than that of LSTM. The Dstat value of the proposed models is larger than that of LSTM. Thus, the predicting effect of the proposed method excels that of LSTM. However, the running time of the proposed methods is longer than that of LSTM.

In Table 2, the NN1 is GRU, and the NN2 is LSTM, GRU or MLP, respectively. The values of the error-index of the proposed models are smaller than that of GRU. The Dstat values of the proposed models are larger than those of GRU. However, the running time of the proposed methods is longer than that of GRU.

In Table 3, the NN1 is MLP, and the NN2 is LSTM, GRU or MLP, respectively. The values of the error-index of the proposed models are smaller than that of MLP. The Dstat values of the proposed models are larger than that of MLP. However, the running time of the proposed methods is longer than that of MLP.

Tables 1–3 show that the performance of the proposed method (NN1:GRU, NN2:GRU) is better than others, because GRU has better performance on some smaller data sets than LSTM.

Figure 10 shows a prediction sample of the proposed method (NN1:GRU, NN2:GRU). The blue zigzag line is the actual trend; the red zigzag line is the predicted trend. The proposed method can give a precise prediction.

Fig. 10

The Shenzhen Composite Index (399001.SZ) of China is used as experimental data. In the simulations, the stock prices from May 2011 to July 2018 are used as training data, and those from August 2018 to March 2019 are used as test data. The Standard Deviation of data set 399001.SZ is bigger than that of data set 000001.SH, as shown in Table 4.

In Table 5, the performance of the proposed method (NN1:GRU, NN2:GRU) is also better than others. On the other hand, the values of the error-index (RMSE, MAPE, MASE) in Table 5 are bigger than those in Table 2 because the Standard Deviation values of data set 399001.SZ are bigger than that of data set 000001.SH.