Comparison of compression estimations under the penalty functions of different violent crimes on campus through deep learning and linear spatial autoregressive models

Backpropagation neural network (BPNN) is a feedforward neural network trained according to the error backpropagation algorithm, which has good applications in many fields. BPNN and feedforward neural network are similar in structure; they consist of the input layer, the hidden layer and the output layer. The hidden layer can be a single layer or multiple layers. The difference is that BPNN will also utilise the backpropagation algorithm after the forward propagation ends and obtain the loss function of the difference between the expected output and the actual output of the model. It uses the gradient descent method to adjust the node weights layer-by-layer to improve the fitting degree of the model. This algorithm has an excellent classification effect for those input objects with obvious characteristics. However, when the dimension of the input value is too large or complicated, the convergence speed will also be slowed down, and it is easy for the model to fall into the local minimum. Thus, considering this defect, the hidden layer of BPNN only retains one layer [17, 18, 19].

In addition to feature extraction and classification of images, the signal frequency domain features also show good classification performance. In terms of extracting frequency domain features, this algorithm can also exert its advantages as much as possible to reduce the impact of disadvantages; therefore, BPNN is an efficient classifier. The model structure of BPNN is shown in Figure 1:

Fig. 1

In the input layer of BPNN, each node corresponds to a feature. When outputting through the hidden layer node, the hidden layer node needs to be determined, as shown in Eq. (1): (1) e=m+n+α, 1≤α≤10 e = \sqrt {m + n}+ \alpha ,\quad 1 \le \alpha\le 10 where: e represents the number of nodes in the hidden layer, n represents the number of nodes in the input layer, m represents the number of nodes in the output layer and α represents a random number. During training, the activation function is also very important. The sigmoid function is expressed as follows [20]: (2) S(x)=11+e−x S\left(x \right) = {1 \over {1 + {e^{- x}}}}

Deep belief network (DBN) is a widely used algorithm in DL. It is a probabilistic generative model that establishes the joint distribution between observation data and labels; the discriminant model is only to predict the labels. This kind of network can be used for both supervised and unsupervised learning as a classifier. Among them, the restricted Boltzmann machine (RBM) is a component unit in DBN; the entire DBN is composed of multiple RBM stacks; the structure of DBN is shown in Figure 2 [21, 22]:

Fig. 2

As shown in Figure 2, in the DBN, there are n RBMs. When the input of the first RBM is the visible layer, the output is the hidden layer 1. The hidden layer 1 is used as the input layer of the next RBM, and the rest may be deduced by analogy. Then, a complete DBN is formed, in which the last RBM output is used as the DBN output. Among them, there is a connection between the visible layer and the hidden layer, but there is no connection between the units in the layer.

ARM is a stationary time-series model with a good effect on the prediction of economy, information and natural phenomena. This method does not require much data when predicting, and can complete the prediction task according to its variable sequence. However, ARM will also be limited during applications. For example, it must have autocorrelation, because autocorrelation coefficient is the key. Usually, autocorrelation is used to predict the economic phenomena associated with the previous period. For economic phenomena that are greatly affected by social factors, other variables need to be added, or other methods need to be combined to complete the prediction [23, 24].

Due to the lag in space, the parameter estimation results obtained by the least squares method (LSM) are inconsistent when estimating the parameters of the spatial ARM. Therefore, many scholars have utilised the maximum likelihood estimation method and the generalised matrix estimation method to estimate the parameters. These methods have good verification results in their respective fields. The linear spatial ARM is proposed based on the spatial ARM. Variable selection is an important component of the model and an important link in the analysis of specific data. Generally, the autoregressive variable selection problems are explored specifically for a determinate penalty function. However, due to the large number of penalty functions, different penalty functions have different penalty effects [25, 26].

Assuming that X^T = (X₁, X₂,⋯ ,X_n) is the regression matrix of n × p, a general spatial ARM can be established, as shown in Eq. (3): (3) Yn=ρWnYn+Xnβ+εn {Y_n} = \rho {W_n}{Y_n} + {X_n}\beta+ {\varepsilon_n} where ρ and β represent fixed parameters, W_n represents the spatial weight matrix of order n, ε_n represents the random error vector of n × 1, the mean is 0, σ²I_n represents the variance and I_n represents the unit matrix of n orders [27]. When estimating the parameters, the log-likelihood function can be analysed as in Eq. (4): (4) Ln(θ)=−n2log2π−n2logσ2+log|In−ρW|−12σ2εnT(θ)εn(θ) Ln\left(\theta\right) =- {n \over 2}\log 2\pi- {n \over 2}\log {\sigma^2} + \log \left| {{I_n} - \rho W} \right| - {1 \over {2{\sigma^2}}}\varepsilon_n^T\left(\theta\right){\varepsilon_n}\left(\theta\right)

On this basis, the interface likelihood method is used to find the estimates β^(ρ) \hat \beta \left(\rho\right) and σ^2(ρ) {\hat \sigma^2}\left(\rho\right) of β and σ², as shown in Eqs (5) and (6): (5) β^(ρ)=(XTX)−1XTA(ρ)Y \hat \beta \left(\rho\right) = {\left({{X^T}X} \right)^{- 1}}{X^T}A\left(\rho\right)Y (6) σ^2(ρ)=1n(A(ρ)Y−Xβ)T(A(ρ)Y−Xβ) {\hat \sigma^2}\left(\rho\right) = {1 \over n}{\left({A\left(\rho\right)Y - X\beta} \right)^T}\left({A\left(\rho\right)Y - X\beta} \right)

If Eqs (5) and (6) are brought into Eq. (4), Eq. (7) will be obtained: (7) Ln(ρ)=−n2(1+log(2π))−n2log(1n(A(ρ)Y)TM(A(ρ)Y))+log|A(ρ)| Ln\left(\rho\right) =- {n \over 2}\left({1 + \log \left({2\pi} \right)} \right) - {n \over 2}\log \left({{1 \over n}{{\left({A\left(\rho\right)Y} \right)}^T}M\left({A\left(\rho\right)Y} \right)} \right) + \log \left| {A\left(\rho\right)} \right|

While solving Eq. (7) and obtaining the maximum value, the estimated value ρ^ \hat \rho of the parameter ρ is obtained, which is then brought into Eqs (5) and (6), whose estimates β^(ρ^) \hat \beta \left({\hat \rho} \right) and σ^2(ρ^) {\hat \sigma^2}\left({\hat \rho} \right) are obtained. To further obtain the sparse estimation of the parameters, the penalty function is set as the objective function, as shown in Eq. (8): (8) Q(β)=‖A(ρ^)y−Xβ‖2+∑i=1Ppλi(|βi|) Q\left(\beta\right) = {\left\| {A\left({\hat \rho} \right)y - X\beta} \right\|^2} + \sum\limits_{i = 1}^Pp{\lambda_i}\left({\left| {{\beta_i}} \right|} \right)

The basic principle of the penalty function is to compress the smaller parameters to 0 as much as possible, thereby eliminating them finally. Therefore, not only can the parameter be estimated but also the selection of variables can be realised. Therefore, the following three penalty functions are introduced, that is, least absolute shrinkage and selection operator (LASSO), adaptive least absolute shrinkage and selection operator (ALASSO), and smoothly clipped absolute deviation (SCAD). The estimates of these functions are shown in Eqs (9–11): (9) β^lasso=argmin‖(A(ρ^)y−Xβ)T(A(ρ^)Y−Xβ)‖2+λ∑j=1p|βj| {\hat \beta_{lasso}} = \arg \min {\left\| {{{\left({A\left({\hat \rho} \right)y - X\beta} \right)}^T}\left({A\left({\hat \rho} \right)Y - X\beta} \right)} \right\|^2} + \lambda \sum\limits_{j = 1}^p\left| {{\beta_j}} \right| (10) β^alasso=argmin‖(A(ρ^)y−Xβ)T(A(ρ^)Y−Xβ)‖2+∑j=1ppλwj|βj| {\hat \beta_{alasso}} = \arg \min {\left\| {{{\left({A\left({\hat \rho} \right)y - X\beta} \right)}^T}\left({A\left({\hat \rho} \right)Y - X\beta} \right)} \right\|^2} + \sum\limits_{j = 1}^p{p_\lambda}{w_j}\left| {{\beta_j}} \right| (11) β^scad=argmin‖(A(ρ^)y−Xβ)T(A(ρ^)Y−Xβ)‖2+∑j=1ppλ|βj| {\hat \beta_{scad}} = \arg \min {\left\| {{{\left({A\left({\hat \rho} \right)y - X\beta} \right)}^T}\left({A\left({\hat \rho} \right)Y - X\beta} \right)} \right\|^2} + \sum\limits_{j = 1}^p{p_\lambda}\left| {{\beta_j}} \right|

Equation (9) is the estimator of LASSO penalty. This method has small variance and fast calculation speed, but it is biased for the parameter estimation. Equation (10) is the estimator of ALASSO penalty, where w represents the weight vector, which can make choices according to different data [28]. Therefore, the obtained estimator has asymptotic normality. Besides, some scholars also found in analysis that because the method is a convex function, it can obtain the global optimal solution. Equation (11) is an estimate of the SCAD penalty, where p_λ can be expressed as Eq. (12): (12) pλ|θ|={λ|θ|,|λ|≺θ(a2−1)λ2−(|θ|−aλ)22(a−1),λ≺|θ|≤aλ12(a+1)λ2,|θ|≻aλ {p_\lambda}\left| \theta\right| = \left\{{\matrix{{\lambda \left| \theta\right|,} \hfill & {\left| \lambda\right| \prec \theta} \hfill\cr{{{\left({{a^2} - 1} \right){\lambda ^2} - {{\left({\left| \theta\right| - a\lambda} \right)}^2}} \over {2\left({a - 1} \right)}},} \hfill & {\lambda\prec \left| \theta\right| \le a\lambda} \hfill\cr{{1 \over 2}\left({a + 1} \right){\lambda ^2},} \hfill & {\left| \theta\right| \succ a\lambda} \hfill\cr}} \right. (13) pλ'(θ)=λ(I(θ≺λ))+(aλ−θ)+I(θ≻λ)(a−1)λ p_\lambda^{'}\left(\theta\right) = \lambda \left({I\left({\theta\prec \lambda} \right)} \right) + {{\left({a\lambda- \theta} \right) + I\left({\theta\succ \lambda} \right)} \over {\left({a - 1} \right)\lambda}}

The subsequent orders of the ARMs established by different types of signals are different. At this moment, to ensure that the dimensions of the regression parameter vectors of all signals are the same, the smallest model lag order in a set of ARMs established by all signals in the dataset is used as a criterion for determining the parameter vector dimension.

After obtaining all the regression parameters of the ARM, these models are combined into parameter vectors; then the DBN model is utilised to learn and extract features. Since RBM nodes are binary, it can be seen that each node has only two states: 0 and 1. Besides, the nodes of each layer are not directly connected; therefore, the state probability of each node can be expressed by Eq. (14) to Eq. (17): (14) p(hj=1|v)=sig(cj+∑iviwij) p\left({{h_j} = 1\left| v \right.} \right) = sig\left({{c_j} + \sum\limits_i{v_i}{w_{ij}}} \right) (15) p(vi=1|h)=sig(bi+∑jhjwij) p\left({{v_i} = 1\left| h \right.} \right) = sig\left({{b_i} + \sum\limits_j{h_j}{w_{ij}}} \right) (16) p(hj=0|v)=1−p(hj=1|v) p\left({{h_j} = 0\left| v \right.} \right) = 1 - p\left({{h_j} = 1\left| v \right.} \right) (17) p(vi=0|h)=1−p(vi=1|h) p\left({{v_i} = 0\left| h \right.} \right) = 1 - p\left({{v_i} = 1\left| h \right.} \right) where v_i and h_j, respectively, represent the state of the i-th node in the visible layer and the j-th node in the hidden layer, and sig represents the activation function. Therefore, the energy function between the visible layer and the hidden layer can be expressed by Eq. (18): (18) E(v,h)=−∑ibivi−∑jcjhj−∑i,jvihjwij E\left({v,h} \right) =- \sum\limits_i{b_i}{v_i} - \sum\limits_j{c_j}{h_j} - \sum\limits_{i,j}{v_i}{h_j}{w_{ij}}

Therefore, on this basis, the possible probability of the combined allocation of the visible layer and the hidden layer is shown in Eq. (19): (19) p(v,h)=1z∑he−E(v,b)=e−E(v,h)∑v,he−E(v,h) p\left({v,h} \right) = {1 \over z}\sum\limits_h{e^{- E\left({v,b} \right)}} = {{{e^{- E\left({v,h} \right)}}} \over {\sum\limits_{v,h}{e^{- E\left({v,h} \right)}}}}

After summing all the possible hidden layer node states, the probability distribution of the visible layer unit state vector is obtained, as shown in Eq. (20): (20) p(v)=1z∑he−E(v,b)=∑he−E(v,h)∑v,he−E(v,h) p\left(v \right) = {1 \over z}\sum\limits_h{e^{- E\left({v,b} \right)}} = {{\sum\limits_h{e^{- E\left({v,h} \right)}}} \over {\sum\limits_{v,h}{e^{- E\left({v,h} \right)}}}}

When training RBM, it is necessary to adjust the weights and offsets, thereby reducing the structural energy and then reaching a stable state. The parameters are adjusted, as shown in Eq. (21): (21) Δwij=γ(〈vihj〉data−〈vihj〉model)Δbi=γ(〈vi〉data−〈vi〉model)Δcj=γ(〈hj〉data−〈hj〉model) \matrix{{\Delta {w_{ij}}} \hfill & {= \gamma \left({{{\left\langle {{v_i}{h_j}} \right\rangle}_{data}} - {{\left\langle {{v_i}{h_j}} \right\rangle}_{\bmod el}}} \right)} \hfill\cr{\Delta {b_i}} \hfill & {= \gamma \left({{{\left\langle {{v_i}} \right\rangle}_{data}} - {{\left\langle {{v_i}} \right\rangle}_{\bmod el}}} \right)} \hfill\cr{\Delta {c_j}} \hfill & {= \gamma \left({{{\left\langle {{h_j}} \right\rangle}_{data}} - {{\left\langle {{h_j}} \right\rangle}_{\bmod el}}} \right)} \hfill\cr} where γ represents the learning rate, 〈〉_data represents the expected p(h_j|v) of the data distribution and 〈〉 _{mod el} represents the expected ∑vp(v, h_j) of the model distribution.

On this basis, an output layer is added to the DBN structure as a classifier. At this moment, the weight and offset of the DBN network need to be used as the feature values of the structure, and the network state is continuously optimised through training [29]. After the DBN is trained, the number of nodes in the output layer is consistent with the number of signal types that need to be identified. This requires the utilisation of supervised learning of the BPNN algorithm to adjust the optimal parameters of each RBM, so as to optimise the entire network. BPNN can adjust the feedback layer-by-layer during the adjustment process; then, it connects the RBM to obtain the optimal parameters.

Here, the campus violence cases are taken as examples. The violent crime cases from 2006 to 2010 in a determinate area are selected, and the simulation analysis is conducted through different punishment functions. Finally, the parameter estimations are compared.

According to statistics, from 2015 to 2019 in this area, the local procuratorate has prosecuted 238 violent crimes in and around the campus, involving 340 people and covering nearly 100 schools, including universities, primary schools and high schools. The data statistics are shown in Figure 3:

Fig. 3

As shown in Figure 3, violent incidents occur on and off campus every year. According to the number of incidents and schools, the incidents involve various schools, and the frequency of occurrence is high, which threatens the safety of teachers and students. Since 2016, the number of cases and the number of crimes has declined year by year, which is also a happy fact.

In the analysis of violence cases, the model parameters are set. It is assumed that the total number of campus areas studied is R, and they are independent of each other. There are m individuals in each area, and they interact with each other. During the simulation, when different sample sizes are given, the parameters after the space will correspond to different simulation results, where C represents the average number of ‘0’ coefficients which are correctly estimated, I represents the four non-zero coefficient that are erroneously estimated as the average number of ‘0’, and MSE represents the mean square error of the estimator, which is used to measure how close the estimated value is to the true value. In the results given by the simulation, the MSE value is selected to be expanded 100 times for easier comparison.