Agricultural Product Recommendation Model based on BMF

Collaborative topic regression (CTR) is the first to use LDA and probability matrix factorisation (PMF) to combine item content information and scoring matrix for collaborative training, and it has achieved good results. With the development of deep learning, Wang HAO et al. proposed a collaborative deep learning (CDL) model based on deep learning. To solve the problem that the sparse text information of CTR is inversely proportional to the validity, the model enables the Stacked Denoising Auto-Encoder (SDAE) to better extract the hidden features of the content information of items, and the hidden features are used to constrain the hidden features V of items decomposed by the PMF, which alleviates the cold start problem and further improves the recommendation effect. The model is shown in Figure 1.

Fig. 1

As shown in the figure, on the right is the naive Bayesian SDAE proposed by the author. W⁺ represents the set of offset vectors and weight matrices of each layer in SDAE and L represents the number of layers of SDAE. X₀ is a set of vector representing N item content information. When the noise is added, it becomes X₁. The role of SDAE is to restore X₁ by making X_C approach X₀ through self-supervised training, thus obtaining the product of the middle hidden layer X_L/2, i.e. the potential features of the content information of the item. The content information vector of the item adopts the method of bag-of-words. By constructing a dictionary containing S keywords, the content information of each item can be converted into vectors. Excluding the SDAE part on the right, the rest is the PMF part. M represents the number of users in the scoring matrix R, N represents the number of items in R, U represents the potential feature vector of users and V represents the potential feature vector of items. As can be seen from the figure as a whole, CDL is essentially the fusion of SDAE and PMF models, and X_L/2 plays the role of a bridge. It connects SDAE and PMF models, thus combining the scoring matrix and the content information of items to jointly represent the user hidden vector V. Therefore, a convolutional matrix factorisation (ConvMF) with better effect has emerged.

SDAE in CDL uses bag-of-words to represent the vectors of the text. Although it can automatically extract the potential features of the text, this method considers only the frequency of words, ignores the context of words, and cannot well represent the description information of items. To solve this problem, ConvMF uses CNN to process and extract potential features in the item description text. Its model is shown in Figure 2.

Fig. 2

As shown in the figure, R ∈ R^N×M is observation value scores, N represents the number of users in the scoring matrix R,M represents the number of items in R,U ∈ R^k×N represents the potential feature vector of users, V ∈ R^k×M represents the potential feature vector of items and k represents the potential feature. The right part of the model is the CNN model, and the left part is the PMF model.

To better complete the task of data crawling, Web Collector framework is used to build crawlers. Web Collector is an integration framework whose kernel is written in Java. Due to its feature that it does not need to configure and provide portable API interfaces, it can define crawlers that meet its own task requirements with a small amount of code, thus carrying out online data crawling. Using the initial URL queue to be crawled as the starting URL, the user-defined web crawler is started to be executed, and the page is analysed according to the stored starting URL, thus generating the URL to be analysed.

HTML document is a kind of text document written with unified standards. Label pairs represent various information and information display methods in web pages. Browsers can display web pages by analysing these label pairs. In general, the positions of various label pairs of HTML documents are fixed, and the HTML documents can be read and filtered by line. By writing corresponding filtering rules (such as regular expressions), these features of HTML can be utilised to obtain data.

After parsing the data from the web page successfully, the next step is to store the data. Generally, when storing content, one can choose either relational database represented by MySQL or non-relational database represented by MongoDB. The advantage of non-relational database is that it can directly store the whole data without considering the influence of fields. Using MySQL to store data, setting the form of fields and data types in advance would take up more time and storage space, but it would be more convenient to extract or further process the data, thus reducing some preprocessing work in disguised form.

This article mainly uses three fields of user comment, commodity description and user score to carry out model research and experimental proof, but the initial data directly crawled from the web page are not stan-dardised, so it is necessary to process the crawled original data to generate a training data set for later use. The specific processing steps are as follows:

Filter system default records

In the previous ConvMF model, the traditional Word2vec was used to construct word vectors, but the word vectors constructed in this way were static. Even in different contexts, the same word has only one coded expression, thus ignoring the situation of polysemy. For example, ‘apple’ can be either ‘fruit’ or ‘mobile phone’. It is obviously unreasonable to use the same static word vector to express it. In addition, although ConvMF uses CNN to extract certain context information from text, due to CNN's own convolution calculation characteristics, the model cannot extract context information between long-distance words, i.e. long-term dependence. To solve this problem, the current common method is to replace the traditional Word2vec with BERT language model. As a powerful pretraining language model, BERT can construct word vectors with rich contextual semantics, which not only solves the problem of polysemy but also directly obtains sentence vectors. A user often comments on multiple items, but there is only one item description, so the processing of user comments is more complicated. Here, taking the processing of user comments as an example, the module is shown in Figure 4.

Fig. 4

ConvMF model uses CNN to obtain better item features and successfully integrates deep learning model into collaborative filtering algorithm through PMF model. Although good recommendation effect has been achieved, it considers neither the polysemy, nor the improvement of user features to recommendation model. Therefore, to improve these problems, a recommendation model based on user comments and item description is proposed in this chapter. The model introduces both the user's comment information and the item description information, uses the feature extraction model constructed in Section 3.1 to obtain the potential features of the user and the item, respectively and finally uses these features and the potential features decomposed by PMF to combine into the final potential features of the user and the item. The specific model diagram is shown in Figure 5.

Fig. 5

The meaning of each parameter in the figure is basically the same as that in ConvMF, and Y represents the introduced user comment text. The middle part of the model is PMF part, and the left and right sides are feature extraction part in 4.3. 2. These two parts are connected by acting on the final item hidden feature V and user feature U together. The generation process of the model is as follows:

For the feature extraction part, to prevent confusion, the weight and offset parameters in this part are collectively referred to as W, then each parameter in W has W∼N(O,σw2I)W \sim N(O,\sigma _w^2I) .

The calculation of the model is similar to that of ConvMF. Since feature extraction is also carried out from user comments, the update rule of u_i and v_j becomes: ui←(VIiVT+λUIK)−1(VRi+λUblstm(WD,Yi))vi←(UIjUT+λVIK)−1(URj+λVbert(WS,Xj))\matrix{ {{u_i} \leftarrow {{(V{I_i}{V^T} + {\lambda _U}{I_K})}^{ - 1}}(V{R_i} + {\lambda _U}{\rm{blstm}}({W_D},{Y_i}))} \cr {{v_i} \leftarrow {{(U{I_j}{U^T} + {\lambda _V}{I_K})}^{ - 1}}(U{R_j} + {\lambda _V}{\rm{bert}}({W_S},{X_j}))} \cr } When U and V remain unchanged for the time being, the optimisation rule for W_D and W_S are updated to: ϵ(WS)=λV2∑IM||Vj−bert(WS,Xj)||2+λWS2∑K|Wk|||Wk||2+constantϵ(WD)=λU2∑IN||ui−blstm(WD,Yi)||2+λWD2∑K|WK|||WK||2+constant\matrix{\epsilon {({W_S}) = {{{\lambda _V}} \over 2}\sum\limits_I^M ||{V_j} - {\rm{bert}}({W_S},{X_j}{{)||}^2} + {{\lambda {W_S}} \over 2}\sum\limits_K^{|{W_k}|} ||{W_k}{{||}_2} + {\rm{constant}}} \cr {\epsilon({W_D}) = {{{\lambda _U}} \over 2}\sum\limits_I^N ||{u_i} - {\rm{blstm}}({W_D},{Y_i}{{)||}^2} + {{\lambda {W_D}} \over 2}\sum\limits_K^{|{W_K}|} ||{W_K}{{||}_2} + {\rm{constant}}} \cr } User i's prediction of item j's score is changed to: rij≈N(uiTvj|σ2)=(blstm(WD,Yi)+θi)T(bert(WS,Xj)+ϵj){r_{ij}} \approx N(u_i^T{v_j}|{\sigma ^2}) = ({\rm{blstm}}({W_D},{Y_i}) + {\theta _i}{)^T}({\rm{bert}}({W_S},{X_j}) + {\epsilon_j})

The data set includes 55,231 scores and comments generated by 13,239 users on 1,246 commodities. The composition of the data set is given in Table 1.

The experimental environment is shown in Table 2.

The main parameters set in the experiment are as follows: the BERT version of the pretraining model used is Chinese_L-12_H-768_A-12, which is officially pretrained and open source by Google. All parameters in this section use their default parameters except the max_seq_length (MSL) parameter. The MSL parameter represents the text length of the input model. Text beyond this length will be truncated. The truncation length of each commodity description text will cover 90% of the commodity description, as will the truncation of each user comment. The data set is randomly divided into three parts according to the proportion of 0.8, 0.1 and 0.1, which are used as training set, verification set and test set, respectively. In addition, the number of user comments contained in each user comment set is set to 5. In the model, the number of hidden units l of LSTM is set to 200, and the dimension t of attention weight vector is set to 400. To prevent overfitting, dropout is added and set to 0.2. Some parameter settings of PMF are the same as those described by ConvMF.

According to the different application fields and objects of concern, the evaluation indices of the recommendation model are also slightly different. MSE is widely used in scoring prediction tasks, so MSE is used as the evaluation standard of the experiment. The equation is as follows: MSE=1N∑(i,j)(Rij−rij)2MSE = {1 \over N}\sum\limits_{(i,j)} {({R_{ij}} - {r_{ij}})^2}

Among them, R_ij is the observation value of the item j scored on user i, r_ij is the prediction score and N is the right number in the whole training set (user score). The higher the prediction accuracy of a model, the lower the MSE value.

The model proposed in this article is based on PMF and improved by introducing additional data sources and deep learning technology, so we use PMF, CDL and ConvMF to carry out comparative experiments under this data set. The experimental results are shown in Table 3.

According to the advantages and disadvantages of the above three models, the proposed improved model uses BERT to represent word vectors, which not only solves the problem of polysemy of a word, but also retains rich contextual relationships. Therefore, it obtains more complete feature expression from item description. The model also introduces user comments. BERT is used to obtain rich user features, and Bi-LSTM is used to fully explore the relationship between user comments, which can not only effectively model user features, but also further alleviate the sparse problem of scoring data. Therefore, the model proposed in this article is superior to CDL and ConvMF at the same time, and the MSE value obtained by training is 1.031, which is 9.7% lower than ConvMF.