Personalized Recommendation Multi-Objective Optimization Model Based on Deep Learning

The recommendation model in this paper learns based on feedback from two types of users: (1) participatory behaviors, such as clicking and watching; (2) Satisfying behaviors, such as sharing, commenting, and collecting. Given the historical behavior information of each user, the ranking system takes user characteristics, video content features and historical behavioural features are used as inputs and learn to predict multiple user behaviours. We formulate the sorting problem as a classification task and compute the cross-entropy loss. [7]. Given user characteristics and video characteristics, ranking models predict the probability that users will take actions such as clicks, watch time, shares, and comments.

After identifying multiple ranking objectives and their problem types, multitask ranking models can be trained for these prediction tasks. For each candidate, take these multiple predictions as inputs and output the combined score using a combinatorial function in the form of weighted multiplication to achieve best performance in terms of user engagement and user satisfaction [8].

Cheng et al. proposed a Wide & Deep Learning model by using multi-source heterogeneous data such as user characteristics, situational characteristics, and project characteristics [9]. As shown in Figure 1, this model combines the training of a broad linear model (on the left side of the figure) and a deep neural network (left side of the figure) to ensure a balance between the ability of model to memorise and generalize. Guo et al. based on Wide & Deep, combined with factorization machine and deep learning, proposed a factorization-Machine based Neural Network (^Deep_FM) for click-through rate prediction, using factorization machine and deep neural network to model low-level and high-level feature interactions, respectively, compared with Wide & Deep [10].^Deep_FM does not require manual feature engineering.

Figure 1.

The shared-substrate multitasking model was proposed in 1998 and is shown in Figure 2. In which the model structure is characterized by the fact that all targets share the same input, and because the underlying parameters are shared by all targets, the risk of overfitting is greatly reduced [11]. At the same time, different goals can also transfer knowledge through these shared parameters when learning, and use the knowledge learned by other goals to help their own goals learn. This model is often regarded as a iconic benchmark approach in multi-objective modelling.

Figure 2.

Input features in the Web domain are often discrete and sparse, and the interaction between features is critical to effectively model this type of data [12]. In the various multi-objective models of deep learning, it is common to use a model structure that shares underlying parameters. This model structure reduces the risk of overfitting and facilitates the learning of the target because the underlying parameters can be shared by all targets. However, when the correlation between targets is relatively low, this hard parameter sharing will limit the freedom of each target fit, impairing multi-objective learning.

Figure 3 illustrates the overall modelling framework results, and earning feature relationships for user behaviour using fully connected neural networks, and using noise reduction auto-coding to initialize user behavior information. An efficient multi-objective neural network architecture was designed, which extended the Wide & Deep model and adopted a multi-objective learning model architecture with a mixture of multiple experts. In addition, a shallow tower was introduced to model and eliminate selection bias, and an ESMM way of constructing loss functions was introduced to establish a connection between targets. In addition, a multi-objective sorting model based on deep learning is formed to integrate user behavior characteristics for interest recommendation.

Figure 3.

The idea taken in this paper is to combine the factorization machine MLP [13], first use the factorization machine to model the pairwise interaction between features, and then further model the higher-order feature interactions by adding a fully connected layer. To take full advantage of this technology of Deep_FM , the research at this stage chose to build a multi- objective model based on Deep_FM , Low-level and high-level feature interactions are modelled using factorial decomposers and deep neural networks, respectively.

In this phase, the two goals of click and watch time are modeled. Leave the FM part of ^Deep_FM unchanged, replace the DNN part of ^Deep_FM with the Share Bottom structure of hard parameter sharing, and obtain a multi-objective model combining ^Deep_FM and Share Bottom Model as the baseline for the study of the multi-objective model.

As shown in Figure 4, the FM subnetwork on the left calculates the second-order crossover fraction of sparse features and dense features, and the deep subnetwork on the right stitches dense features and continuous features into the network. Finally, the FM first-order, second-order fractions, and the last layer of deep inputs are stitched together, and the estimated value is obtained by sigmoid.

Figure 4.

The model predicts the following: 1 y^=sigmoid(yDeep +yFM)

Among them, y^{^}∈(0,1) is the predicte CTR y_FM and y_Deep are the outputs of the FM component and the output of the deep component, respectively. 2 yFM=<w,x>+∑j1=1d∑j2=j1+1d<Vi,Vj>xj1,xj2

Where x = [x_{field 1},x_{field 2},…,x_fieldj,x_fieldn] is a vector of d-dimension, x_field is the vector representation of the j_th domain of X. For feature i, the importance of the 1st order is measured by a scalar w_i, and the impact of its interaction with other features is measured by a latent vector v_i . The addition unit <w,x> in the network architecture functions to capture the significance of the 1st-order feature, while the inner product element signifies the impact of the 2nd-order feature interaction. This approach allows for the consideration of both individual feature importance and their interactions, enhancing the model's ability to make personalized and accurate recommendations. 3 ydeepk =hk(f(x))

The output result of the shared hidden layer, f(x), is input to the respective tower network (subnetwork)^hk , and finally, each target k gets an output ydeepk .y^=sigmoid(yFM+yDeep )

Because of the shortcomings of the Share Bottom model structure, Google proposed the Multi-gated Mixture of Experts (MMoE) model in 2018 [14], which introduces multiple expert subnetworks and gating structures, and uses different expert combinations to learn different goals through gating so that each goal can be better learned. As shown in figure 5 is a schematic diagram of the MMoE model structure.

Figure 5.

The design of this stage is influenced by the Multi-Task Mixture of Experts (MMoE) architecture, which involves incorporating a distinct gated network g^k for each target k. These networks are added to the deep part of the model, building upon the framework established in the previous stage. This approach allows for the creation of specialized networks for individual targets, enabling the model to effectively capture the intricacies and nuances specific to each task. g is the gating network that combines the results of the experts, the internal implementation of the gating is composed of the same multilayer perceptron with ReLU activation, and the gating network is a simple linear transformation of the input with the softmax layer, as shown in figure 6:

Figure 6.

The input vector and the output vector of each expert will be passed into the gating network, the input vector will first pass through MLP, and the last layer of softmax will get the weight of each expert, and the output of the gating is the weight on all experts: 4 gk(x)=softmax(Whkx)

Where ^{W_gk ∈ Rnsd} is the trainable matrix, n and d denote the number of experts and the edge dimension, respectively.

Multiple expert networks are added at the same time as the introduction of a gating network for each goal, and the gating network learns different combinations of the expert network for their respective tasks, and the output of the expert network is adaptively weighted. For a task, the output of its corresponding network of experts is: 5 fk(x)=∑i=1ngk(x)ifi(x)

Where ^f_i (i=1,…,n)is a network of n experts.

The new multi-objective model based on the gated network is shown in Figure 7, and the improved advantage is that each target can train a gated network individually, and the weight of each expert network owned by each target task is adjusted according to the objective adaptively. Get the output of the deep part target k as: 6 yk=hk(fk(x))

Figure 7.

In this stage of the model, the target uses the binary classification cross-entropy to make losses, and then the loss weights of the two targets are summed to obtain a total loss function, and the model parameters are solved by optimizing this total loss function. The total loss function is as follows: 7 L=miθnnw1∑i=1NL1(yi,Pctr(xi,θ))+w2∑i=1NL2(zi,Pcr(xi,θ))

Among them, ^L₁ and ^L₂ are the loss functions of fitting CTR and CTCVR, respectively, and both are binary classification cross-entropy; ^x_i indicates the input feature; ^y_i is the click target, click is 1, exposure unclicked is 0; ^z_i is the conversion goal, converted to 1, click not converted to 0; ^{P_ctr(x_i,θ)} is an estimate of CTR, ^{P_ctr(x_i,θ)} is an estimate of CVR, θ is the model parameter, ^w₁ and ^w₂ are the weights of the two losses, and N represents the total quantity of samples.

Multi-objective modeling often has a seesaw phenomenon, usually, multi-objective learning relative to multiple single-objective learning models can improve the effectiveness of a part of the goal, but some multi-objective models often at the expense of the performance of other targets to improve some goals, this problem is called seesaw phenomenon. One of the primary challenges in multi-objective learning arises from the gradients of diverse objectives, which tend to clash with each other in a way that is not conducive to progress, and in some cases can lead to a significant decrease in performance.

To solve the "seesaw" problem that multiple objectives are prone to, a multi-level expert network is introduced based on the previous FG model, and an independent expert network is established for each target while retaining a shared expert network [15]. Introducing multi-level experts, which consist of shared experts and specific target experts, helps mitigate harmful parameter interference and enables the integration of multi-objective features through gated networks. Implementing a novel progressive separation approach facilitates the emulation of interactions among experts, leading to more effective knowledge transfer between intricate and interconnected objectives. Get FM_Sharing Expert (FS) model to solve the seesaw problem, ensure stable optimization, and the deep part of the model is shown in the figure 8.

Figure 8.

Definition of the gated network in the ^jth extraction network of the kth sub-target in the FS model deep section: 8 gk,j(x)=wk,j(gk,j−1(x))Sk,j(x) )

Where ^wk,j the weight is function of the target k as the input to ^gk,j-1 , and ^Sk,j is the selection matrix of the ^j_th extraction network for the target k. After calculating all the gating networks and kexpert networks, the final output of the ^k_th sub target of the deep part of the FS model is: 9 yk(x)=tk(gk,N(x))

In a recommendation scenario, the user's behavior is generally more than one, and the different behaviors occur in order and dependencies. Each of a user's behavior can be a target in a multi-objective model, and there are dependencies between these targets. For this correlated multi-objective model, if each target fits independently, the information about the dependencies between the targets will be lost, and the accuracy of the model will be lost, affecting the sorting effect. Therefore, when doing correlation multi-objective modeling, it is necessary to model every step of the transformation in user behavior. In the case of video recommendations, for example, in this scenario, the goal we need to model is clicked and playtime and the conversion relationship involved in these two goals can be described as: showing the video to the user——the user clicks on the video——the viewing time exceeds a certain threshold.

Use x to represent the characteristics of the user and the video; y indicates the label of the click, y=1 indicates the click, and y=0 indicates that the exposure is not clicked; z indicates the label of the playback duration, z=1 indicates that the playback time exceeds the threshold, and z=0 indicates that the threshold has not been exceeded, including no clicks. The conversion relationship and probability quantification of exposure, clicks, and duration in user behavior can be expressed as follows [16]: 10 p(y=1,z=1∣x)︸pCTCVR=p(y=1∣x)︸pCTR*p(z=1∣y=1,x)︸pCVR

To do this, we combined the loss function of the previous version of the multi-objective model and the ESMM to obtain a multi-objective model as shown in the figure 9.

Figure 9.

Finally, the CTR loss and CTCVR loss weighted sum give a total loss, and the model parameters are solved by minimizing the total loss. 11 L=miθnnw1∑i=1NL1(yi,Pctr(xi,θ))+w2∑i=1NL2(zi,Pctr(xi,θ)*Pcvr(xi,θ))

Among them, ^L₁ and ^L₂ are the loss functions that fit CTR and CTCVR, respectively, and both are two-classification cross-entropy; ^y_i is the class label of the click; ^z_i as a class indicator for the length of playback (1 for longer playback than the threshold, 0 for otherwise), ^{P_ctr(x_i,θ)} is an estimate of CTR; ^{P_ctr(x_i,θ)} is an estimate of CVR; θ is the model parameter; ^w₁ and ^w₂ are the weights of the two losses respectively, with N being the total number of samples.