Research of Email Classification based on Deep Neural Network

Email has become a major way of communication for people at present, but the problem of spam comes behind. The harm of spam is mainly manifested as the following aspects: occupying bandwidth, leading to the congestion of the email server and reducing the efficiency of the network; consuming the time of the user and affecting the work efficiency. Therefore, the effective distinction between normal email and spam, so as to maximize the possible of filtering spam has become a research hotspot currently.

Naive bayes algorithm is a kind of frequently-used email classification and it is a statistical-based classification algorithm[1-3], which has the characteristics of simple realization and fast classification. However, it assumes that the attributes are independent of each other when given the target value[4]. This hypothesis is apparently impossible in the email classification, so the accuracy of email classification based on naive bayes algorithm is low. In allusion to the problem of poor accuracy of email classification based on naive bayes algorithm, scholars have proposed some new email classification algorithms. The email classification algorithm based on deep neural network (DNN) is one kind of them.

The basic concept of artificial neural network is based on the hypothesis and model construction of how the human brain responds to complex problems[4-6]. The deep neural network is an artificial neural network with full connection between layer and layer, and its structure is shown in figure 1. The full connection between layer and layer means that any neuron in the i^th layer must be connected to any of the neurons in the (i + 1)^th layer. Although the deep neural network looks complex, it is still the same as the perceptron from a small local model.

Figure 1.

We use w j k l to respresent the weight coefficient between the k^th neuron in the (l − 1)^th layer and the j^th neuron in the l^th layer, b j l to represent the bias of the j_th neuron in the l^th layer, a j l to represent the activation value of the neuron in the layer. We can get the following relationship between the activation value of the j^th neuron in the layerand the activation value of all neuron sin the (1 − l)^th layer: (1) a j l =σ ( ∑ k w jk 1 a k l-1 +b j l )

We assume that w^l is the weight coefficient matrix of all the neurons in the l^th layer, b^l is the bias matrix of the l^th layer, a^l is the activation value of the layer, z^l is the weighted input of all neurons in the l^th layer, Then w j k l is the weight coefficient of row j, column k. The relationship between the activation value of the l^th layer and the activation value of the (l − 1)^th layer can be expressed by the following matrix relationship: (2) a l = σ ( z l ) = σ ( w l a l-1 + b l )

Here σ respresents the non-linear activation function of the nodes on the hidden layers, and the traditional DNN uses sigmoid function usually, as shown in expression (3). Because the sigmoid function has properties such as monotone increasing and its inverse function has the property of monotone increasing, it is often used as a threshold function of neural networks, It maps the variables between 0 and 1. The sigmoid function curve is shown in figure 2: (3) σ ( z ) = 1 1 + e − z

Figure 2.

Implementation process of mail classification algorithm based on deep neural network was shown in Figure 3.

Figure 3.

In order to verify the effect of the email classification algorithm based on DNN, in this paper we constructeda DNN with 2 hidden layers. The number of nodes in each hidden layer was 30. When the training set was trained, we set up 2000 batches, and each batch has 3 trained data. We used the famous SpamBase dataset as the data set, which was from the UCI machine learning library at the University of California, USA. The specific situation is shown in table I.

TABLE I.

We compared the two kinds of email filtering algorithms of DNN and naive Bayes with accuracy, which is the main evaluation standard of email filtering technology. The accuracy is defined as follows: (4) Accuracy= Number of correctly identified email Total number of emails

We did five groups of experiments in this paper.The selection case of training set and testing set in each experiment is shown in table II.

TABLE II.

The experimental results were shown in Figure 4.

Figure 4.

The experiment result showed that DNN was higher than naive Bayes in the accuracy of email classification when the proportion of the training set was 10%, 20%, 30%, 40% and 50% respectively, and DNN showed a good classification effect.

The application of email classification algorithm based on deep neural network is studied in this paper. The algorithm constructed multiple hidden layers and generated DNN classifiers through training. The experiment results showed that the accuracy of the algorithm is obviously higher than the naive Bayes algorithm.

With the development of science and technology, spam manifests in many forms and the damage of it is more serious, this puts forward higher requirements for the accuracy of spam recognition. The focus of next research will becombining various algorithms to further improve the effect of email classification.