Construction of dynamic update and adaptive prediction model for user profile based on time series analysis

For the prediction of users’ dynamic interest labels, this subsection uses the temporal attention mechanism to focus on the key feature information in the model input stage based on the deep forest model to further optimize the model. The improved multi-granularity scanning structure as a feature processing module that incorporates the temporal attention mechanism can be seen in Fig. 3.

Figure 3.

Multi-size scanning structure of the convergence time attention mechanism

This is done by calculating the ratio of the user dwell time of each example item contained in each interest cluster in each user packet to the sum of the user dwell times of all examples in that packet, and using the resulting weight as the attention weight for that interest cluster. For each user behavior log Session_i = {x₁,x₂,⋯,x_n}, which corresponds to a click time of t_i = {t₁,t₂,⋯,t_n}, then the dwell time for each item is calculated as: (2) t˜n−1=tn−tn−1 (3) Ti={ t˜1,⋯,t˜n−1,max(t˜) }

Based on the importance of the last item in the serialized recommendation algorithm, the x_n dwell time is set to be the maximum dwell time max(t˜) in T_i. The weight vector of the time-attention mechanism is specifically computed from Eqs. (2) and (3): (4) Vi=∑j=1NWj*Vj=∑j=1Ljf(Tj)*xj (5) f(Tj)=Tj∑i=1LjTi

Compared with the single demand forecasting model, the combination forecasting model can have higher forecasting accuracy, and this chapter will investigate how to combine each single demand forecasting model with a suitable parallel combination and optimize the weight allocation based on the objective function of root mean square error.

In this paper, we adopt the adaptive combination prediction weight determination method, which corresponds to the weight coefficients of the combination prediction model that change with the prediction time nodes, and it can use the prediction value of the previous step as the input for predicting the prediction value of the next step, and constantly update the weights [23].

In the combined model of this paper, an important factor affecting the forecasting performance of the model combination is a set of weight combinations ω_m, m = 1,…,M. M is the number of individual models in the combined model, i.e., the number of M is 2. In this paper, a new scheme for determining the weight values is adopted, which captures the time-varying dynamics of the underlying time series.

The evaluation index of whether the weight allocation is reasonable is the root mean square error MSE, which is minimized when the predicted value is the same as the true value. When the sum of sub-model weights is satisfied to be 1 and the average absolute error is the smallest, the combined prediction error can be minimized and the weights of the ith prediction model at the corresponding t moments are obtained. The formula is shown in equation (6): (6) { minMSE=1net2=1n∑i=1d×w(λiteit)2s.t.∑i=1nλit=1,λit≥0 where y^t is the demand forecast value at time t, y_t is the actual demand value at time t, d is the window width of the proximity history sample representing the selected number of similar weeks, and w is the number of forecast points corresponding to the unit time.

When the error value of the k th demand prediction sub-model at time t in window width d is always minimized, the result of Eq. (6) can be expressed as Eq. (7): (7) { λkt=1λit=0,t−1,2,…,n;i≠k

The result of Eq. (6) can be expressed as Eq. (8) when time t minimizes the prediction error of the k₁rd model in some weeks in window width d and the k₂th model in some other weeks: (8) { λk2t=| ek2t || ek1t |+| ek2t |λk2t=| ep1t || ek2t |+| ek2t |

The weight values of each model can be calculated by Eq. The calculation is based on Eq. (7) and Eq. (8): (9) λi(p+1)=1p∑t=1pλitλi(p+2)=1p∑t=2p+1λit,⋯,λi(p+j)=1p∑t=jp+j−1λit

Where p is the amount of basic data for the chosen time t.

And the determination of the sample window width d is also a key factor that affects the effect of the combination weights. It is assumed that the demand forecast data of N similar weeks prior to the week to be measured is utilized for the determination of the sample window width d of the proximity history, and there are a total of n forecast points in the demand forecast data.

Let ε_kt be the MAE value of the k nd single prediction method in week t, ε_t be the average absolute prediction error of all the prediction methods in week t, and δ_j be the average absolute prediction error of all the prediction methods in the j th window period. First calculate the MAE value during the first window period. Next, slide one historical week and calculate the mean absolute error during the next window period. Repeat the process until sliding to the last window. A suitable window width can keep the MAE value within a stable range and ensure that the output of the combined pre-model has a certain degree of fit with the real data.

The specific steps for selecting the sample window width are as follows.

Since the weights need to be updated at each prediction time, an optimization method with fast computation and high accuracy is needed to obtain accurate weights. In this paper, the Bayesian optimization (BO) algorithm is used to optimize the weights of individual models.

The flowchart of the Bayesian optimization algorithm is shown in Figure 4.

Figure 4.

The BO algorithm process

In this paper, the collection function of the Bayesian optimization algorithm is used to measure the gain of sample point selection, when we use the sample points in the training set to model a certain function and obtain the corresponding probability model, the collection function can determine the next sample point selection strategy. The acquisition function a(x) reflects the gain of the newly selected sample points at x for model optimization, and by solving argmax(a(x)) through the optimization algorithm, we can get the corresponding sample point selection position with the highest gain. In this paper, the EI (Expected Improvement) function is used as the collection function, and the function is constructed as shown in equation (12): (12) EI(x)={ (μ(x)−f(x+))Φ(z)+σ(x)Φ(z)σ>00σ=0 z=μ(x)−f(x+)σ where Φ(z) is the normal cumulative distribution function and f(x⁺) is the maximum value available.

The computation process of the adaptive weighting strategy is as follows:

1) Determine the window width d, and write down the actual demand sequence of a similar week before the week to be tested as Y_j = {y_j1,y_j2,…,y_jw}, and the actual demand sequence of d similar weeks before the week to be tested as Y = {y₁,y₂,…,y_d}, where the predicted demand data sequence obtained by the k th single prediction model is Y_i = {y_i1,y_i2,…,y_id}.

The combination weights are updated based on the previous data before each demand forecast calculation, so that each single model can better utilize the strengths of their respective models, and so that the combination forecasts have more reasonable weights.

The experiments in this paper use four standard evaluation metrics: accuracy rate, check rate, F1 value and AUC.

The accuracy rate represents the ratio of the number of correctly predicted samples to the total number of predicted samples and is calculated as: (14) Accuracy=TP+TNTP+TN+FP+FN

The check rate represents the ratio of the number of correctly predicted positive samples to the number of all samples predicted to be positive and is calculated as: (15) Precious=TPTP+FP

F1 was chosen as a measure of model wholeness and was calculated as: (16) F1=Pt×RePt+Re

The AUC value provides a very intuitive assessment of model performance, with values closer to 1 indicating better model classification.

The construction of the forest in the deep forest model is the core of the model establishment, and the construction of the decision tree is the core of the forest, so the number and depth of the decision tree in the forest will directly affect the training efficiency and classification effect of the model. Deep forests can cascade multiple models, and diversity is especially critical to model design, so this paper tries to cascade multiple forest models in logistic regression (LR), random forest (RF), extreme random tree (ET), gradient boosted tree (XGB), and determines the model types and hyperparameters through experiments.

The performance of model accuracy evaluation with different settings of n_estimate parameter is shown in Table 1. It shows that each forest model as a whole exhibits a trend of increasing accuracy with increasing the n_estimate parameter and then smoothing out. Among them, the RF and XGB models have comparable prediction accuracies, with mean accuracy values of 0.8909 and 0.8910, respectively.

Due to the time overhead associated with the increase in the n_estimate parameter, the model training time evaluation performance is shown in Table 2. It can be clearly seen that the larger the n_estimate parameter, the longer the training time of the model. When the parameter setting increases from 5 to 180, the training time of RF, ET, and XGB models increases by 90.56s, 434.91s, and 936.59s, respectively.

In addition, we compare the models with respect to the variation of the parameter maxdepth. The accuracy metrics are evaluated as shown in Table 3. As the maxdepth parameter increases, the model accuracy not only does not improve, but also decreases. When the parameter setting increases from 5 to 180, the RF, ET, and XGB model accuracy decreases by 0.0021, 0.0166, and 0.0036, respectively.

The performance on training time is shown in Table 4. For the RF and ET models, the increase in the maxdepth parameter brings no time overhead, but the XGB model shows a very significant increase in runtime with the increase in the maxdepth parameter, which increases from 68.63s to 492.63s when the parameter is increased from 10 to 100.

By comprehensively analyzing the model performance, the hyperparameters of the three models were set as shown in Table 5.The hyperparameters of XGB, ET, and RF were set to 5, 25, and 15, respectively.

Each of the three models mentioned above has its own advantages in classification performance: the RF model has lower variance and bias, and thus has the highest accuracy and the fastest training efficiency in the experiments; the ET model has a further reduction in variance relative to RF, and the bias has increased, resulting in a slight decrease in classification accuracy; XGBoost, as a typical representative of gradient boosting integrated learning algorithms, has a very high accuracy, but the the time overhead is relatively large. The diversity of cascade models directly affects the classification effect, in this paper, by cascading more than one model, we get the experimental results of multiple cascade forests as shown in Table 6. The accuracy rate of RF+ET+XGB model reaches 89.81%, and taking into account the evaluation indexes such as the model accuracy rate and the running time, we choose three models, namely RF, ET and XGB, to form the cascade forest module of the deep forest.

In order to highlight the advantages of the deep forest algorithm, based on the above dataset, this paper introduces the traditional machine learning algorithms: logistic regression (LR), support vector machine (SVM), decision tree (DT), deep convolutional neural network (DCNN) and the integrated algorithms Random Forest (RF), and XGBoost for predicting and comparing the part of hyperparameters of each algorithm. The performance of each model on different metrics is shown in Table 7. It can be seen that the deep forest model performs better than traditional machine learning algorithms in predicting behavior, and compared with deep convolutional neural networks, although the model’s advantage in prediction accuracy is not obvious, the training time of the deep forest model is about 1/20 of that of the deep convolutional neural network.After incorporating the temporal attention mechanism into the deep forest model and implementing the adaptive weight combination strategy, the model prediction accuracy reaches 92.3%, the method in this paper significantly improves the model prediction accuracy.