Mathematical model of back propagation for stock price forecasting

BP algorithm is used to train multi-layer feedforward neural network (the feedforward network using BP learning algorithm is called BP neural network), which belongs to the learning algorithm with tutors. BP network has the characteristics of clear structure, easy implementation, powerful computing function and superior performance, so it is widely used in many fields such as pattern recognition and text classification. BP neural network adopts a parallel network structure, including input layer, hidden layer and output layer. BP network has been proved to have strong nonlinear mapping ability and generalisation function, and multi-layer network can approximate arbitrary nonlinear function. Before the training of BP neural network, the parameters of the network should be determined and initialised first, and the training of the network can only be started when everything is ready. The input signal enters the network from the input layer, and is output through the output layer after the weighted sum of each layer and the transformation of activation function [5]. This process is the forward propagation of the input signal. In this process, the input of each layer of neurons is only affected by the output of the previous layer of neurons and the weight and Min value of the network remain unchanged. If the error between the actual output and the expected output of the network is large, the error signal is transferred via back propagation to reduce the error and make the actual output approach the expected output gradually.

Figure 1 shows a three-layer feedforward neural network, in which the input layer and the output layer each have two nodes. The hidden layer has three nodes. Each node in the hidden layer and the output layer is a sigmoid cell, which is based on a smooth differentiable threshold function. For each Sigmoid cell, its output is calculated as follows: o=σ(w→⋅x→), among them σ(y)=11+e−y, x→ is the input vector of this node, and w→ is its weight vector. σ is often called the sigmoid function or we can also call it the logistic function.

Fig. 1

The input of the hidden layer node comes from the input layer. When the weight of each input layer node to each hidden layer node is determined, the output of the hidden layer node is determined. The output of the hidden layer is used as the input of the output layer. Similarly, when the weight of each hidden layer node to each output layer node is determined, the output value of the output layer node is also determined. Therefore, the weight vector learning is the key. In essence, the weight learning problem is a search problem, i.e. it is necessary to find a reasonable w→ in the Rw→ space to minimise the error of the corresponding network on the training samples. The formal expression is, find w→, i.e. minimise this expression.

(1)E(w→)=12∑d∈b∑k∈ out puts tkd−okd2 where D is the training sample set; D is a training sample; outputs are collections of network output units; t_kd is the KTH dimension of the expected output vector of d; and o_kd is the value of the KTH dimension in the output vector of the neural network to D. It is noteworthy to observe that genetic algorithm, particle swarm optimisation algorithm and so on can be used to find the approximate optimal solution of w→. In this paper, BP algorithm based on stochastic gradient descent is used to search w→.

BP algorithm is a learning algorithm with a tutor. Taking three-layer BP neural network as an example, the derivation of BP algorithm is made. We assume that the input learning samples are P, x₁,x₂…, x_p, the corresponding expected output is t₁,t₂,… ,t_p, the actual output is y₁,y₂,…,y_p and the number of neurons in the hidden layer is s. The idea of BP algorithm is to correct the connection weight and Min value by calculating the mean square error between the actual output and the expected output, so that the actual output and the expected output are as close as possible.

Output of the ith neuron in the hidden layer is as follows: (2)ai=f∑j=1pxjwij−θj where w_ij is the connection weight between the input layer and the hidden layer, and θ_j is the Min value of neurons in the input layer.

The output of the KTH neuron in the output layer is: (3)yk=f∑r=1parwkr′−θr′

If netnetk′=∑r=1parwkr′−θr′ is taken, Eq. (3) is converted into Eq. (4), as follows: (4)yk=f net k′ where wir′ is the connection weight between the hidden layer and the output layer, and θr′ is the Min value of the neurons in the hidden layer. The error function is: (5)E(w,θ)=12∑k=1ptk−yk2

When the actual network output is not consistent with the expected output, the gradient descent method is used to correct the network connection weight. The connection weight adjustment formula between the hidden layer and the output layer is: (6)Δwkr′=−η∂E∂wkr′

The weight adjustment formula from the input layer to the hidden layer is: (7)Δwij=−η∂E∂wij

According to the properties of partial derivatives, Eq. (6) can be written as: (8)Δwkr′=−η∂E∂wkr′=−η∂E∂netk′⋅∂netk′∂wkr′

According to Eqs (3) and (5), the required formula can be obtained as the following in Eq. (9): (9)E=12∑k=1ptk−yk2=12∑k=1ptk−f∑r=1parwkr′−θr′2

For the output layer, there is the formula: (10)∂E∂netk′=∂E∂yk⋅∂yk∂netk′=∂E∂yk⋅f′net′k

And because (11)∂E∂yk=−tk−yk∂netk′∂wkr′=ar the formula of weight adjustment between hidden layer and output layer is: (12)Δwkr′=ηtk−yk⋅f′netk′⋅ar

The stock price prediction model (SPPM) designed in this paper is shown in Figure 2. The model is divided into two stages: training stage and application stage. The training stage refers to the process of stock price prediction based on the neural network learned in the training stage [8,9]. In order to improve the generalisation ability of SP-PM and enhance the effectiveness of the model applied to stock price prediction, the neural network integration technology is adopted. Neural network integration has proved to be a very effective approach to improve the processing power of the learning system even if it is only a simple voting or averaging of a set of networks. In this paper, the integration of neural network is embodied in two aspects: individual generation and result fusion.

Fig. 2

We assume that t₁,t₂, … ,t_n(n ≥ 2) is a continuous time series. At the moment t_i, any attribute of a stock (such as opening price, highest price, lowest price, closing price, trading volume, transaction amount etc.) can be obtained. For the data sequence obtained on the time series t₁, t₂, … , t_n by an attribute (for instance, the closing price indicated below), we can tentatively ascertain the data sequence to be: x₁,x₂, … ,x_n.

First, we normalise each dimension of the data sequence: (13)xi=xi−minmax−min where 1≤i≤n, max and min respectively represent x₁,x₂,…, the maximum and minimum values of x_n. Second, n-N-M +1 samples are constructed according to Eq. (13), where n and M represent the future closing prices of M(M≥1) days predicted by the closing prices of the previous N(N ≥1) days; n锛濶M≥2.

(14)x1x2…x3⇒X1=x1,x2,⋯,xN,Y1=x2,x3,⋯,xM+1X2=<x2,x3,⋯,xN+1>,Y2=x3,x4,⋯,xM+2…Xn−N−M+1=xn−n−M+1,xn−N−M+2,xn−N−M+3,⋯,xn−MYn−N−M+1=<xn−M+1,xn−M+2,⋯,xn>

The sample set we get is denoted by D, and obviously |D| is n—N—M+1. The i (1≤ i ≤ n-n-m +1) sample in D is expressed as < X_i.Y_i>, where |X_i| is N and |Y_i| is M. For a time series, as shown in Figure 3, when the window size is adjusted, different sample sets can be obtained. Different neural network individuals can be trained by using different sample sets [10]. The following, D1, D2… and Dk represent k sample sets, respectively, and k represents the number of Windows or the total number of neural networks.

Fig. 3

If the input layer and output layer adopt linear transformation function and the hidden layer adopts Sigmoid transformation function, then the multi-layer neural network with a hidden layer can approximate any rational function with arbitrary accuracy. Therefore, the feedforward network used in this paper has three layers of structure: input layer, hidden layer and output layer. The number of nodes in the input layer of the network is determined by the dimension of the input vector. Since the dimension of the input vector is N, the number of nodes in the input layer is determined to be N. The number of nodes in the output layer is larger by the dimension of the output vector, which is the direct cause of ‘overfitting’ in training. However, it is a pity that there is no scientific and universal method to determine it in theory at present. In order to avoid the phenomenon of ‘overfitting’ during training as much as possible and ensure high enough network performance and generalisation ability, the most basic principle to determine the number of nodes in the hidden layer is rendered as compact as possible under the premise of meeting the accuracy requirements, i.e. the nodes in the hidden layer are kept to as few a number as possible. The following conditions must be satisfied when determining the number of hidden layer nodes: (1) The number of hidden layer nodes must be less than (|D| – 12); The number of training samples must be more than the connection weight of the network model, which is generally around 2-10 times.

Let the number of nodes in the hidden layer be h, and the connection weight be: N ×h+M ×h. The second condition can be expressed as: 2≤|D|N×h+M×h≤10, |D|10×(N+M)≤h≤|D|2×(N+M)..

Thus, the above two conditions can be expressed as: (15)|D|10×(N+M)≤h≤|D|2×(N+M)

To sum up, this paper adopts N-H-M neural network, with hidden layer node as sigmod element and output layer node as linear element.

K data sets can be trained to obtain K neural networks. We assume that for an unknown sample < X, ? >, let SPPM output be rY₁,Y₂, … ,Y_k, respectively. Next, we will discuss how to ‘merge’ these outputs and finally give the specific method of the stock price trend in the future M days. One of the easiest ways to do this is to take the average value of each output, as follows: (16)Y=∑i=1kYik where the magnitude of Y is M, and each dimension represents the predicted value for a certain day in the future.

In many cases, we don’t need to know the exact value of the future – only whether it is going up or down. In this case, we can just look at the value of the current day in Y compared to the previous day.