Computer vision recognition and tracking algorithm based on convolutional neural network

The convolution layer is well known. In the direction of image processing, a convolution neural network has certain advantages over a depth neural network, and its use of local perception is stronger than the latter. In the convolution layer, we extract the input data to get its eigenvalues, and when these speech data are preprocessed, the extracted eigenvalues will be cut into rectangular blocks one by one. For the back-propagation algorithm, by calculating each convolution unit, the input data features pass through a convolution layer. Through the accumulation of convolution layers, more information on data features can be extracted more completely. When passing through more convolution layers, some complex features will be extracted from the basic data. Therefore, the calculation formula of convolution can be expressed by the following formula 1.

1 Y00=X00∗W00+X01∗W01+X10∗W10+X11∗W11

For example, in a simple convolution process, the initial output data X is a 4*4 matrix. After the first convolution of matrix W, the data is half 2*2. Finally, after another convolution, the result matrix Y is 3*3.

After these three layers of processing, it needs to be processed through softmax regression. In this process, it is to define the probability value that the initial sample does not match the same category, and obtain the probability that the input belongs to a certain category among different categories on the final output. Here, some processing of convolution is different from the category of signal operation. In this model, convolution processing is a linear weighted calculation. See the formula above (1).

The convolution layer contains many convolution kernels. The eigenvalues of different levels are learned by these convolution kernels, to obtain many output values. Of course, these convolution kernels are not directly connected with the whole characteristic graph. There is a layer of rules between them, which combine them. In the convolution core, its work is different from that of the full connection layer network. The convolution core in the convolution layer is connected to the areas where the features are input. Through such connection, the connection size of the whole network has been reduced to a certain extent. On the contrary, it can get enough structured features. Moreover, because the convolution core has a parameter sharing mechanism, the parameters of the training network have been reduced a lot.

Another name for the pooling layer is also called the full connection layer. In the pooling layer, it will not reduce the matrix depth of the whole network and can reduce the size of the matrix to a certain extent [15]. After processing in the convolution layer, a lot of information coincides to a certain extent, so there is a pooling layer. After the processing of the pooling layer, the number of nodes in the full connection layer can be reduced, which can achieve the effect of reducing the number of parameters in the whole neural network. After the processing of the pooling layer, the effect obtained is to accelerate the calculation speed. At the same time, it can also make the output data not cause overfitting problems. In terms of use, it is somewhat similar to the convolution layer. Here, there are two calculation methods to process the data characteristics obtained through the convolution layer, the maximum pooling layer, and the average pooling layer, The former is to take the maximum value of the data, which is also the most used pool layer structure. The latter obtains its average value through the relevant calculation area of specified characteristic data [17–21].

Single depth slice is 4× The block of 4 is sampled with the maximum value, and the result is 2×2. In this way, the single depth slice can be divided into 2×2, as shown in the figure above, take the maximum value of each small square. We can use formula (2) to express the output size of the pooling layer.

2 Size_Pool=Size_Patch/Size_Cpool

In the above formula (2), we can see that there is a division operation, which shows that the characteristics of the input pool layer should be divided by the size of the pooling layer. Size_Pool is the data feature size obtained by the above method, Size_Patch is the feature size output after the calculation of the convolution layer, Size_Cpool refers to the size of the pooling layer.

The processing of the pooling layer has achieved remarkable results in two aspects. One is to compress the data level of the whole neural network. At the same time, it will reduce the training parameters of the neural network and improve the constraint of the neural network. Based on these two, it will achieve a good effect, that is, it can maintain the invariance of the network, to improve the Luban of the whole network itself, In this way, the application in speech recognition can achieve an effect, that is, the speech recognition system will have certain fault tolerance in the translation caused by different wording techniques and some external environment noise.

After the cyclic calculation of the above two layers, for the data input into the network, after the cyclic calculation of the first two layers, for the input characteristic data, the relevant important data has been reorganised, and its key information can be better reflected. For the operation of the first two layers, they can be regarded as automatic calculation behaviour. After these two processes, the processing results are classified through the full connection layer. There is a rasterisation process, which reorganises the output results, and then obtains a feature vector, which also upgrades part of the feature information processed by the convolution layer and pooling layer to deeper dimensional space, and also classifies all the information. The tool of this classification is the full connection layer. For example, it can convolute a two-dimensional feature value through the above convolution, In this way, it is transformed into one-dimensional feature data so that the information can be reorganised globally. Softmax full connection is generally adopted, which is also the last checkpoint of a convolutional neural network. Its key role is to normalise the output results. The activation value obtained in this way is the speech feature calculated by a convolutional neural network.

After the first two layers of the convolutional neural network are calculated, if the output result has a flag, after the last softmax, the output result is a vector value with N dimensions. Each value in the vector has a corresponding probability, that is, it represents the occurrence probability of the first flag. Each flag corresponds to the probability one by one, and the sum of the probabilities of the N flags is 1. The expression (3) of softmax for the last level is as follows.

3 F(Zj)=ezj∑i=1Nezi$$F({Z_j}) = {{{e^{{z_j}}}} \over {\mathop \sum \limits_{i = 1}^N {e^{{z_i}}}}}$$

From the above formula, the function of this operation is to improve the nonlinear expression ability of the whole convolution model. The whole formula is actually a normalisation process. The meaning of the denominator part is the sum of N signs. The upper part is the result of the first sign. Obviously, the result of all signs will not be >1. When the corresponding probability of the result of a sign approaches 1, the result of this sign must be greater than that of other signs. In this way, the results of other signs must be infinitely close to each other. This is the normalisation process mentioned above.

The random gradient descent method is a simple and effective method. It randomly extracts one group from all samples, updates it according to the gradient after training, and then extracts one group and updates it once. When the sample size is large, you may not need to train all samples to get a network model within an acceptable range. Because only one group of samples is updated each time. The parameters required for the updating of the network model will be greatly reduced and the training speed of the network will be accelerated, but this will also cause the problem of uncertain optimisation direction, which will eventually result in local optimisation rather than global optimisation, thus reducing the accuracy of the network model.

The momentum gradient descent method draws on the ideas of physics. Imagine rolling the ball into a frictionless bowl. The accumulated momentum does not stop at the bottom of the bowl but pushes it forward, which produces momentum. The descent process of the above random gradient descent method is not directly to the best, but back and forth, which will waste a lot of network training time. In addition, the random gradient descent algorithm needs to set a learning rate before training. Setting an excessive learning rate will lead to excessive updating of parameters and large errors of gradient swing. Setting a small learning rate will increase the number of iterations of training, which will increase the training time of the model and waste resources and time. The momentum gradient descent method can set a larger learning rate and reduce the training times in the training process so that the convergence speed of the network model is faster and more stable, and there will be no back-and-forth descent problem. The algorithm mainly sets an impulse in the process of random gradient descent to accelerate the gradient descent. The momentum gradient descent algorithm can be expressed by the following formula: 4 Vdw=θVdw+(1−θ)dw 5 Vdb=θVdb+(1−θ)db 6 W=W−φVdw 7 b=b−φVdb

Where dw and db respectively represent the gradient value generated in this iteration, V_dw and V_db the gradient momentum accumulated in the previous iteration. The weighted average gradient of the two indexes can slow down the gradient decline and ϕ are super parameters introduced. A large number of experiments show that the effect is better.

Adaptive motion estimation is a combination of rmsprop and random gradient descent methods. The algorithm uses the square gradient of rmsprop to scale the learning rate of the network model, and uses the average value of the gradient movement rather than the gradient itself to calculate. In this way, it can set different learning rates for different parameters. It is an adaptive learning rate method. Adam algorithm is named for its use of an adaptive matrix. It adjusts the learning rate in the network model by estimating the first-order matrix and the second-order matrix. The fluctuation problem of the previous algorithm is improved, which significantly improves the convergence speed and stability of the network model. Adam algorithm can be expressed by the following formula: 8 Vdw=ϕ1Vdw+(1−ϕ1)dw 9 Vdb=ϕ1Vdb+(1−ϕ1)db 10 Sdw=ϕ2Sdw+(1−ϕ2)dw2 11 Sdb=ϕ2Sdb+(1−ϕ2)db2 S_dw and S_db in the above formula represent the weighted and biased second-order exponential weighted average gradient respectively. Due to the large difference between the moving average gradient value and the value after iteration, it is necessary to properly correct the value. The correction formula is: :Vdwc=Vdw/(1−ϕ1t),Vdbc=Vdb/(1−ϕ1t),Sdwc=Sdw/(1−ϕ2t),$$\matrix{ {:V_{dw}^c = {V_{dw}}/(1 - \phi _1^t),\quad V_{db}^c = {V_{db}}/(1 - \phi _1^t),\quad S_{dw}^c = {S_{dw}}/(1 - \phi _2^t),} \cr } $$

Sdbc=VSdb/(1−ϕ2t) Combining the modified weight and offset to update the equation, we can get a new formula: 12 W=W−φVdwcSdwc+ω 13 b=b−φVdbcSdbc+ω ϕ₁ in the above formula corresponds to the value of ϕ in the momentum gradient descent algorithm, and the same value is 0.9. The parameter ϕ2 corresponds to the specified value similar to ϕ₁, which is generally 0.999. The parameter ω is a smooth item, which is generally taken as 10−8. The learning rate φ needs to be fine-tuned in the network model training.

PSNR is the most commonly used objective evaluation method of image quality. It is usually compared with the original image after image compression and image restoration. It evaluates the image quality by comparing the difference in pixel values between the original image and the processed image. Its formula is as follows: 14 PSNR=10⋅log10(k21N∑i=1N(I(i)−I^(i)2))$$PSNR = 10 \cdot {\log _{10}}\left( {{{{k^2}} \over {{1 \over N}\mathop \sum \limits_{i = 1}^N (I(i) - \hat I{{(i)}^2})}}} \right)$$

Generally, k adopts 8-bit representation, and k is 255. According to the above formula, PSNR is only related to the error MSE between pixels. MSE is to calculate the mean square error between the real image I with N pixels and the reconstructed image. PSNR only focuses on the differences between image pixels, while people often pay attention to the visual sensory experience. However, to rationally compare with the experimental results in literature and works, it still has great advantages. Therefore, PSNR is still the most extensive evaluation standard in the field of image super-resolution. For colour images, the PSNR values of three RBG channels are generally calculated and then averaged, or the mean square error of three RGB channels can be calculated and then averaged.

Generally speaking, it is easier for human vision to extract the structure of images. Therefore, according to the comparison of image brightness, contrast and structure, a structural similarity index (SSM) is proposed to measure the structural similarity between images. For an image I with N pixels. Suppose the brightness of the image is γi and the contrast of the image is λi, respectively corresponding to the average value and deviation of the image feature intensity. among 15 γi=1/N∑i=1NI(i) 16 λi=(1N−1∑i=1NI(i−γi)2)1/2$${\lambda _i} = {\left( {{1 \over {N - 1}}\mathop \sum \limits_{i = 1}^N I{{(i - {\gamma _i})}^2}} \right)^{1/2}}$$

I(i) in the formula represents the characteristic intensity of the second pixel of image I. The comparison of pixel illumination and contrast of image Cl(I,I^) and Cc(I,I^) can be expressed by the following formula: 17 Cl(I,I^)=2γIγI^+c1γI2+γI^2+c1 18 Cc(I,I^)=2λIλI+c1λI2+λI2+c2

c1=(k1m)2 and c2=(k2m)2 in the above formula are constants whose denominator is 0. Among them, k₁ and k₂ are two constants far less than 1, usually k1=0.01 and k2=0.03. The normalised pixel value of the image can be expressed as ((I−γI)−λI). The structural similarity is measured by the correlation between the pixel value of the original image and the brightness and contrast of the image, which is equivalent to the correlation coefficient between image I and image Cs(I,I^). Therefore, the comparison function 8 between them can be expressed by the following formula: 19 λII^=1N−1∑i=1N⁡(I(i)−γI)(I^(i)−γI^)\begin{equation} {\lambda _{I\hat I}} = \frac{1}{{N - 1}}\mathop \sum \limits_{i = 1}^N \left( {I(i) - {\gamma _I}} \right)\left( {\hat I(i) - {\gamma _{\hat I}}} \right) \label{eq19} %(19) \end{equation} 20 Cs(I,I^)=(γII^+c3)γIγI^+c3$${C_s}(I,\hat I) = {{\left( {{\gamma _{I\hat I}} + {c_3}} \right)} \over {{\gamma _I}{\gamma _{\hat I}} + {c_3}}}$$

In the formula is the covariance between image I and image I^, and c₃ is also to prevent the constant with denominator 0. Finally, the SSIM expression can be obtained from the above formula: 21 SSIM(I,I^)=[Cl(I,I^)]α[Cc(I,I^)]β[Cs(I,I^)]δ$$SSIM(I,\hat I) = {\left[ {{C_l}(I,\hat I)} \right]^\alpha }{\left[ {{C_c}(I,\hat I)} \right]^\beta }{\left[ {{C_s}(I,\hat I)} \right]^\delta }$$

Among them, α, β, and δ are the control parameters for adjusting image elements. If these parameters are set to 1, the following formula can be obtained: 22 SSIM(II^)=(2γIγI^+c1)(λII^+c2)(γI2+γI^2+c1)(λI2+λI2+c2)$$SSIM(I\hat I) = {{\left( {2{\gamma _I}{\gamma _{\hat I}} + {c_1}} \right)\left( {{\lambda _{I\hat I}} + {c_2}} \right)} \over {\left( {\gamma _I^2 + \gamma _{\hat I}^2 + {c_1}} \right)\left( {\lambda _I^2 + \lambda _I^2 + {c_2}} \right)}}$$

When calculating the formula, a m×m size window will be taken from the image, and then calculated according to the movement of the window. Finally, the average value of the results will be taken as the global SSM. This evaluation method is to express the traditional brightness and contrast through structural information, use the mean value as the brightness estimation, covariance as the measure of structural similarity, and standard deviation as the contrast estimation. SSIM evaluates the reconstruction quality of an image from the perspective of human eye observation. Compared with PSNR, SSIM can better meet people’s needs in intuitive vision. Therefore, this image evaluation method has also been widely used