Research on loyalty prediction of e-commerce customer based on data mining

Although there are many methods to predict the customer loyalty, the existing prediction and classification methods have certain limitations to predict the purchase possibility of customers when they hold activities in a shop because the data of different industries and businesses have different characteristics. As the data of customers’ purchasing behaviour on e-commerce platform has the characteristics of limited dimension, abundant data and discrete type, the effective features of data are category data and the logs generated by the past historical records of customers’ purchasing corresponding products in a store on Tmall platform. Therefore, in this study, the classifier model can’t get the ideal effect. So the Boosting algorithm is selected in machine learning to estimate customer loyalty.

XGBoost is a GBDT algorithm and its corresponding generalised algorithm in the Gradient Boosting architecture. Generally, the objective function and the prediction function will be constructed. By applying the training sample to the lowest objective function and making it learn the relevant parameters, detailed samples can be further classified and predicted. Equation (1) shows the specific form. (1) Obj (θ)=L(θ)+Ω(θ) Obj\;(\theta ) = L(\theta ) + \Omega (\theta ) where Ω(θ) represents a punishment for errors in complex models and L(θ) is an error function, which is used to evaluate the fitting degree of data and model which is defined as Eq. (2): (2) yi=∑K−1Kfk(xi), fk∈F {y_i} = \sum\limits_{K - 1}^K {f_k}({x_i}),{\kern 1pt} {f_k} \in F

Specific function f regard as the basic function of F, while F is regarded as a special set of all regression trees, as shown in Eq. (3). (3) Obj (θ)=∑inl(yi,yl)+∑k=1KΩ(fk) Obj\;(\theta ) = \sum\limits_i^n l({y_i},{y_l}) + \sum\limits_{k = 1}^K \Omega ({f_k})

The basic principle of Boosting is to ensure that a specific model remains fixed, and further add new functions f, so as to control the detailed objective function as much as possible. (4) Obj(t)=∑inl(yi,yl(t−1)+ft(xi))+Ω(fi) Ob{j^{(t)}} = \sum\limits_i^n l\left( {{y_i},y_l^{(t - 1)} + {f_t}({x_i})} \right) + \Omega ({f_i})

Use Taylor formula to expand and define the objective function approximately, as shown in Eq. (5). (5) Obj(t)≈∑i=1n[l(yi,yl(t−1))+gift(xi)+12hift2(xi)]+Ω(ft) Ob{j^{(t)}} \approx \sum\limits_{i = 1}^n \left[ {l\left( {{y_i},y_l^{(t - 1)}} \right) + {g_i}{f_t}({x_i}) + {1 \over 2}{h_i}f_t^2({x_i})} \right] + \Omega ({f_t})

When the constant term is removed, the objective function Obj(θ) is relatively dependent on the error function L(θ), as shown in Eq. (6). (6) Obj(t)≈∑i=1n[gift(xi)+12hift2(xi)]+Ω(ft) Ob{j^{(t)}} \approx \sum\limits_{i = 1}^n \left[ {{g_i}{f_t}({x_i}) + {1 \over 2}{h_i}f_t^2({x_i})} \right] + \Omega ({f_t})

Then the specific tree is further divided into a leaf weight W and a structure part q, W will give the leaf score corresponding to each index number: f_t(x) = w_q(x), where (7) w∈RT​, q​:Rd→{1,2,3,4,⋯​,T} w \in {R^T},{\kern 1pt} q:{R^d} \to \{ 1,2,3,4, \cdots ,T\}

Use Eq. (5) to judge the particularity of the decision tree, and the complexity also covers the detailed number and detailed score L₂ and its square of the module, the specific formula is shown in the following Eq. (8): (8) Ω(ft)=γT+12λ∑j=1Twj2 \Omega ({f_t}) = \gamma T + {1 \over 2}\lambda \sum\limits_{j = 1}^T w_j^2 where wj2 w_j^2 is the magnitude of w and L₂ squared, and T is the number of leaves in the tree. In an objective function L(θ), T uncorrelated quadratic functions of one variable are set, which are defined as shown in Eq. (9): (9) Gj=∑i∈Ijgi,Hj=∑i∈Ijhi {G_j} = \sum\limits_{i \in {I_j}} {g_i},{H_j} = \sum\limits_{i \in {I_j}} {h_i}

Bring Eq. (10) into the above formula to obtain: (10) Obj(t)=∑j=1T[Gjwj+12(Hj+λ)wj2]+γT Ob{j^{(t)}} = \sum\limits_{j = 1}^T \left[ {{G_j}{w_j} + {1 \over 2}({H_j} + \lambda )w_j^2} \right] + \gamma T

The derivative of w_j is obtained by Eq. (11), whose result is 0. The optimal solution of W and its corresponding objective function L(θ) are solved, as shown in Eq. (12): (11) wj*=−GjHj+λ w_j^* = - {{{G_j}} \over {{H_j} + \lambda }} (12) Obj=−12∑j=1TGj2Hj+λ+γT Obj = - {1 \over 2}\sum\limits_{j = 1}^T {{G_j^2} \over {{H_j} + \lambda }} + \gamma T

The research and analysis of customer churn is a classic two-category problem. Due to the deviation of the prediction model itself, the positive and negative cases are often wrongly divided, and the customers who have lost are mistaken for those who have not lost. On the contrary, the customers with higher loyalty are mistaken for those who have lost, which brings a lot of inconvenience to the daily management of enterprises, and even wastes the limited resources of enterprises. Therefore, the definition of loss function of XGBoost algorithm is optimised in this paper to reduce the loss caused by wrong classification.

Logarithmic function is a loss function commonly used in solving classification problems, as shown in Eq. (13): (13) L(θ)=∑i[yiln (1+e−y^t)+(1−yi)ln (1+ey^i)] L(\theta ) = \sum\limits_i [{y_i}\ln \;(1 + {e^{ - {{\hat y}_t}}}) + (1 - {y_i})\ln \;(1 + {e^{{{\hat y}_i}}})]

In the calculation process of XGboost algorithm, it is easy to find out through the loss function, which sets the probability of positive and negative examples being divided into the same. But in fact, there are more cases to divide positive examples into errors. So penalty coefficient α is added to the loss function to distinguish the probability of two different errors.

The definition of the loss function after adding the penalty coefficient is shown in Eq. (14), because there are more cases in which the positive examples are wrongly divided. The value range of α is (0.5, 1]. (14) L(θ)=∑i[yiln (1+e−αY^i)+(1−yi)ln (1+e(1−α)y^i)] L(\theta ) = \sum\limits_i [{y_i}\ln \;(1 + {e^{ - \alpha {{\hat Y}_i}}}) + (1 - {y_i})\ln \;(1 + {e^{(1 - \alpha ){{\hat y}_i}}})]

Before the data is imported into R, it is preliminarily sorted out. The steps are divided into four steps, as shown in Figure 5.

Fig. 5

Since the data are made ‘dirty’ by missing data, abnormal data and repeated data, we need to clean it.

Among them, Dist(p, o) is the n-dimensional Euclidean space distance between point p and o. |N(p)| is a field of his neighbour's number of the point p.

Local deviation index: If the object p neighbourhood is not dense, namely, |N(p)| < MinPts, the partial deviation index of p is defined as: (16) LDI(p)=∑o∈N(p)(LD(o)/LD(p))|N(p)| LDI(p) = {{\sum\nolimits_{o \in N(p)} {(LD(o)/LD(p))} } \over {|N(p)|}}

The local deviation index of object P represents the degree to which object P deviates from its neighbours.

In this paper, LOF algorithm is used to eliminate the outliers in the preliminary data set. The theoretical basis is to analyse the local density of a point, in order to compare it with the density of points distributed around it. If the comparison shows that the density of the area where this point is located is sparse, it indicates that this point is an abnormal value. In addition, the outlier score is calculated by the lofactor function in DMwR package, and the data with the top five scores are taken as outliers, and then eliminated. In this paper, a total of 200 pieces of abnormal data were eliminated.