Application of machine learning in stock selection

Linear model is the simplest and most commonly used model in machine learning. There are many kinds of linear models, such as regression model (Linear Regression, Ridge Regression and Lasso Regression), classification model (Logistic Regression), Principal Component Analysis, Linear Discriminant Analysis, etc. The linear model is characterized by simple structure, easy analysis and understanding, and difficult over fitting. The disadvantage is that the assumption is strong, and the complex financial signal may not meet the linear assumption.

Fig. 1

Clustering is a kind of unsupervised machine learning method, which classifies similar (related) objects into the same cluster. The biggest difference between clustering and classification is that the we know the target of classification in advance, but we do not know the target of clustering. Clustering can be applied to almost all objects. The more similar (related) the objects in the cluster, the better it effects.

K-means clustering is the most classical clustering algorithm. It groups the samples according to the similarity between samples and divides them into K clusters. This is a very effective and practical idea in stock selection.

Clustering is simple and intuitive. It can provide multiple possible solutions for exploratory research. Selecting the final explanation requires subjective judgment and subsequent analysis. Stock selection is to select a high-yield stock portfolio from multiple stocks, which is completely consistent with the clustering characteristics.

Steve [7] uses clustering algorithm and time series outlier analysis, uses PAM clustering algorithm to constrain the initial set of stocks, and uses outlier analysis to define two independent active trading strategies. These results are compared with the passive strategy of fully investing in the S&P 500 index, the stock portfolio constructed by the combination of clustering algorithm and time series outlier analysis is better than the pure passive index strategy. Newton [8] uses hierarchical clustering algorithm for cluster analysis, groups the stocks in the spot market according to the risk return criterion, and then uses Ward method to sort to form the final stock portfolio. Wang [9] conducted cluster analysis on the constituent stocks of SSE 180 index. By dividing the constituent stocks of SSE 180 index into 8 categories, using conditional stock selection and cluster analysis, the success rate and return rate were tested. It was proved that the return rate of stock portfolio selected by cluster analysis method was higher than that of stocks without cluster analysis. Ratchata [10] proposed a stock selection method combining the moving average of the index, trend and momentum technical indicators to identify the stock selection method that is most likely to be better than the market index, and determined the best combination of these indicators through cluster analysis. Li [11] improves the semi supervised k-means algorithm for the multi factor stock selection model, adds the modified Gaussian kernel function on the basis of introducing the gravitational image of the marked data to the unmarked data, and constructs the improved Gaussian kernel function and the semi supervised K-means clustering method based on the gravitational image factor. Without increasing the complexity of the algorithm, it greatly improves the ability of K-means model to deal with high-dimensional and linear inseparable problems.

Clustering method has a lot of research and application in stock selection, which is mainly due to the fact that the characteristics of clustering are very similar to stock selection. Different clustering combinations have been studied in the literature, and the stock selection results have also been significantly improved compared with the comparison model.

Support vector machine is one of the most widely used supervised machine learning methods. Support vector machine can be divided into linear support vector machine and kernel support vector machine. The former aims at linear classification problem, and the latter belongs to nonlinear classifier.

The idea of support vector machine is to construct “hyperplane classification”, and calculate the hyperplane by introducing kernel. For the linear case, the equation of the hyperplane is: (8) f(x)=β0+Σi=1nαi〈xi, x〉 {f_{(x)}} = {\beta _0} + \Sigma _{i = 1}^n{\alpha _i}\left\langle {{x_i},{\kern 1pt} x} \right\rangle where n represents the number of training samples, x is the characteristic of the new sample, 〈x_i, x〉 is the inner product of the new sample and all training samples, and β₀ and α It is calculated by training samples. After adding the core, replace the inner product to obtain a new equation: (9) f(x)=β0+Σi=1nαiK(xi, x) {f_{(x)}} = {\beta _0} + \Sigma _{i = 1}^n{\alpha _i}K({x_i},{\kern 1pt} x)

There are usually three kernel functions of support vector machine:

Linear kernel: (10) K(xi, xi′)=Σj=1pxijxi′j K({x_i},{\kern 1pt} x_i^\prime) = \Sigma _{j = 1}^p{x_{ij}}{x_{i'j}}

Support vector machine has many advantages. And it has many models in dealing with linear and nonlinear problems. However, the training time of support vector machine is generally long, especially when the number of training samples is too large. Moreover, the kernel method requires to store the kernel matrix, resulting in the high spatial complexity of the algorithm. Therefore, support vector machine is more suitable for the task of small batch samples.

Lee [12] proposed a hybrid feature selection method, which combines the advantages of filtering method and packaging method, selects the best features subset from the original feature set. And then, it finds the best parameter value of SVM kernel function. Through this method, the stock selection prediction of SVM is realized, and its accuracy is better than the traditional BP network. Zikowski [13] uses volume weighted support vector machine to select short-term stocks and then construct stock portfolio. Ru [14] uses AdaBoost enhancement method to process parameters and uses SVM as classifier. The addition of AdaBoost makes the model more profitable and less income fluctuation than traditional SVM. Zhang [15] tested SVM models with linear kernel, polynomial kernel, Gaussian kernel and sigmoid kernel, and constructed different K-SVM stock selection models on this basis. The experiments proved that K-SVM has strong prediction ability of stock selection income. Gao [16] proposed an integrated model of fuzzy dynamic SVM for stock selection. Experiments show that the average rate of return of the model is higher than that of the industry. Meng [17] proposed a multi-level stock selection model of GBDT-SVM based on a large number of factors, which uses machine learning technology to optimize factor selection and dynamic adjustment of factor weight, so as to improve the acquisition ability of multi factor model to stock excess return.

Due to the introduction of kernel method, support vector machine model can better solve the nonlinear classification problem than the previous machine learning model, which is very effective in complex stock selection data, solves the problem of linear indivisibility, and the obtained model is more effective and stable.

Random forest is a classifier containing multiple decision trees. It trains a series of decision trees with Bagging method. It is a strong classifier that can be better applied to different scenarios of classification and regression.

Specifically, the random forest algorithm constructs each decision tree according to the following two-step method. The first step is called “row sampling”, which is to sample back from all training samples to obtain a bootstrap data set. The second step is called “column sampling”, m features (m less than m) are randomly selected from all m features, and M features of bootstrap data set are used as a new training set to train a decision tree. If it is classified prediction, the category or one of the categories with the most votes cast by N decision trees is the final category. In case of continuous numerical regression prediction, the value obtained by arithmetic averaging the regression or 0–1 classification results obtained from n decision trees is the final model output.

Random forest has good accuracy, can effectively run on large data sets, and can process input samples with high-dimensional features. Random forest can evaluate the importance of each feature. Therefore, dimensionality reduction is not required in the construction of random forest model. These characteristics make the training and prediction of random forest faster. The performance of random forest in stock selection is better than the traditional stock selection methods in many literatures, and it is widely used in the construction of stock selection model.

Random forest algorithm can solve two kinds of problems: classification and regression. Tan [18] and Li [19] uses random forest to select stocks by multiple factors through financial data, industry data, stock data and other data to build a stock portfolio, and its return rate is higher than the average return. Zheng [20] uses the minimum spanning tree method to investigate the relationship between stocks, explore the structure of stock network, use Kruskal algorithm, Prim algorithm and weight matrix method to solve the minimum spanning tree of stock network, model and solve the factors of stocks according to grouping technology, find out the similarities and differences between stocks, and select stocks. Jia [21] proposed a random forest support vector machine model. The model uses random forest to reduce the dimension of the original data, which improves the reliability and effectiveness of decision-making. By comparing the model with PCA-SVM, it is confirmed that the stock selection accuracy of the model is higher.

As an ensemble learning model, random forest can solve the instability of a single model in classification. Random forest can process higher dimensional data, ensure the diversity of stock selection factors, and make the stock selection results more comprehensive and effective.

Artificial neural network is a model that imitates the information transmission between brain neurons. It can approximate any function with any accuracy. It can deal with all kinds of complex nonlinear relations. It is mostly used to deal with classification problems. Artificial neural network can be divided into multi-layer and single-layer. Each layer contains several neurons and they are connected by a directed arc with variable weight.

In the process of model training, we calculate the value a of the hidden layer according to the input feature X, and then calculate the predicted value y of the output layer according to the value of the hidden layer. This step is called forward propagation algorithm. We hope that the closer the predicted value y is to the real value T, the better, and the closer the output value is to the label depends on the connection weight in each layer.

Theoretically, there is a close relationship between the number of hidden layers and nodes and the fitting degree of the model to the data. More number leads to more accuracy. However, more number of layers and nodes will bring many problems, such as the number of weight parameters, which makes the solution space of the optimization problem too large and the algorithm is difficult to converge. Secondly, the back-propagation algorithm will also fail. The error gradient becomes minimal after several layers of transmission, and it is almost impossible to modify the connection weight of the previous layers. In addition, it also brings the problem of over fitting.

Asriel [22] used multilayer feedforward neural network to predict the stock excess return through technical and fundamental factors, and constructed a hedging portfolio composed of the same capitalized long and short positions, with a return higher than that of S&P 500 index. Ghosn [23] and Quah [24] and Stanley [25] and Yeh [26] and Kim [27] and Huang [28] use artificial neural network to select stocks in different stock markets and build stock portfolios, and their returns are higher than those of the general index. Tsai [29] and Lan [30] use back propagation neural network to conduct quantitative stock selection in Taiwan stock market and Shenzhen stock market. Principal component analysis is added to select indicators, and its accuracy and yield are higher than that of BPNN network [30]. Tong [31] compared multilayer perceptron, adaptive neuro fuzzy inference system and general growth radial basis function, and proposed how to systematically select stocks by using relative operating characteristic curve (ROC).

Neural network can build more complex stock selection model. Through the study of different literature, we find that the organization of different neurons has a great impact on the effect of stock selection model. And different stock selection factors also have a great impact on the results of the model, and the performance of the same model in different stock pools is also quite different.

As the most important branch of machine learning, deep learning has developed rapidly in recent years and attracted extensive attention at home and abroad. As one of the hottest trends in machine learning and artificial intelligence research, deep learning realizes the feature extraction of external input data from low-level to high-level by establishing and simulating the hierarchical structure of human brain, so as to achieve the purpose of interpreting external data. In recent years, as a new branch of machine learning, deep learning has achieved great success in many fields, including image processing, computer vision, speech recognition, machine translation, natural language processing and so on.

Deep learning can be regarded as a neural network structure with multiple hidden layers. It can form more abstract high-level representation attributes or features by combining low-level features, so as to find the distributed feature representation of data.

Deep learning can also be regarded as a general term of a series of methods. Typical deep learning models include deep feedforward neural network, deep belief network, convolutional neural network, generative countermeasure neural network, graph convolution neural network, etc. There are great differences in the structure of each model. According to our research on stock selection literature, we find that cyclic neural network is widely used in quantitative investment, and there are many applications of cyclic neural network in stock selection.

Yang [32] proposed a multi-index feature selection method based on multi-channel convolutional neural network structure. The maximum information coefficient feature selection method is used to select candidate indicators to ensure the correlation with stock trend and reduce the redundancy between different indicators, so as to build a high-yield stock portfolio. Zhang [33] and Zhang [34] and Sun [35] is based on the LSTM model for stock selection. The deep stock ranker model proposed in [34] is based on the LSTM model, which can predict the future earnings ranking of stocks and make stock selection. Sun [35] selects stocks based on the sequence of yield prediction, and studies LSTM and GRU series prediction models. The prediction and stock selection accuracy of the two models is better than the traditional forward neural network model, and GRU model is slightly better than LSTM model. Zhou [36] constructs a multi-factor stock selection model based on adaptive recurrent neural network algorithm. Compared with RNN, the model has less calculation time, low cost and high accuracy. Compared with LSTM, the accuracy and convergence speed of the model are significantly improved.

The deep learning model can better process high-dimensional data and build more complex stock selection models. Moreover, for the stock trend with frequent changes, the deep learning model has better fault tolerance, can make better adjustments, and the stock selection results are more efficient and stable.