Mathematical model of transforming image elements to structured data based on BP neural network
Published Online: Dec 15, 2021
Page range: 257 - 266
Received: Jun 16, 2021
Accepted: Sep 24, 2021
DOI: https://doi.org/10.2478/amns.2021.1.00084
Keywords
© 2021 Wang et al., published by Sciendo
This work is licensed under the Creative Commons Attribution 4.0 International License.
A porcelain insulator is one of the most used insulators in power transmission lines, and thus it is essential to ensure its safe and stable operation. Porcelain insulators of overhead lines are subjected to mechanical forces such as wind load, ice coating, conductor weight etc., in addition to long-term resistance to electric field forces such as working voltage, transient overvoltage and lightning overvoltage. These factors reduce its mechanical properties and insulation properties, and form zero-value insulators’ [1]. These external environmental factors affect the regular operation of the power grid. In recent years, power grid companies have repeatedly encountered operation accidents of porcelain insulators in substations, which seriously threaten the safety of power grids.
At present, with the development of power big data, manual detection is gradually being eliminated due to its cost, accuracy and detection efficiency, which are difficult to adapt to the influx of massive image data. Infrared, ultraviolet, ultrasonic and other long-distance detection methods are widely used because of their timeliness, safety and high diagnostic efficiency. The electrical and thermal characteristics of degraded insulators are different from those of insulators under regular operation and thus become a vital detection index for diagnosing whether insulators are degraded or not. Infrared imaging technology is a non-contact, passive measurement technology. Through active thermal excitation, the specially shaped structure inside the object is expressed in a temperature difference as a detection method.
However, infrared images’ feature extraction and classification technology are demanding and critical in the automatic fault diagnosis system. Therefore, we must seek a fault diagnosis method with less input, fast diagnosis speed and higher accuracy. Some scholars have shown that the surface emissivity of the infrared imager has a particular influence on the detection of high-voltage electrical equipment, and the data should be set in the range of 0.85–0.95 [2]. This method has a high accuracy rate, but a large amount of input requires high detection conditions. Some scholars use temperature and image information as characteristic thresholds to judge the operating status of insulators. The calculation process of this algorithm is complicated, and the detection time is extended.
The author takes the three-umbrella porcelain insulator as the research object and takes the colour matrix of the centreline of the insulator as the characteristic vector. We use BP neural network to build a diagnostic model. The algorithm has the advantages of small input dimension, fast diagnosis speed, and high accuracy rate. This has a particular engineering significance for infrared detection of degraded insulators.
To obtain typical and representative infrared pictures of degraded insulators, the author conducted the following experiments. The article chooses the three-umbrella porcelain insulators that have been replaced on-site as the test samples. The resistance values of the sample deteriorated insulators were 7.5 MΩ, 19 MΩ and 150 MΩ. The experimental setup is shown in Figure 1.
Fig. 1
Schematic diagram of insulator infrared experimental device.
The temperature range is 18°C to 25°C under Class I pollution conditions, and when the relative humidity of the environment is <50%, we choose a low-value degraded insulator. They were located at positions 2, 5 and 7 for testing, and the remaining six films were all classic films [3]. We applied a 65.9 kV power frequency voltage to the insulator string at the high-voltage end, and the experiment was carried out for 2 h. We photographed it with an infrared imager every 0.5 h. After the insulator is entirely cooled at the end of the experiment, we replaced the remaining low-value insulators with different resistance values and repeated the above steps in sequence. The infrared imager model is Fluke Ti55 TF, and the emissivity is 0.9. We obtained 405 infrared insulator string images through experiments. We sliced the image of the insulator string to obtain 2835 single insulators as images to be processed in the experiment. We randomly select 2500 monoliths as training samples and 335 monoliths as test samples.
A neural network is a data processing system composed of many interconnected processing elements of neurons. Among them, the BP neural network algorithm is one of the leading classification algorithms for data mining. It can automatically learn and establish a large number of input-output mapping relationships [4]. During training, we continuously adjust the connection weights of the neural network and the size of the network through backpropagation, which minimises the error sum of squares. This is because the insulator infrared diagnosis problem is essentially a multi-input nonlinear problem and requires a faster training speed to adapt to the infrared data generated every moment in the system. Therefore, this article uses BP neural network to ensure that both accuracy and detection speed meet high requirements.
The choice of input variables is significant. It directly affects the training speed of the network and the speed of convergence. Appropriate input variables can make the fault diagnosis model contain the most abundant information and improve fault diagnosis accuracy. Since the acquired data is unstructured image data, it is necessary to extract features from the image as input for training [5]. Traditional image feature extraction methods include colour moments, colour histograms and colour sets. This paper proposes a diagnostic method that takes the colour matrix of the centreline of the insulator as the feature input parameter and compares it with the colour moment and colour histogram in the traditional feature parameter extraction method.
The colour histogram is a function of colour information. It represents the number of pixels with the same colour level in the image. We give preference to this method because it has the advantages of rapidity and insensitivity to image changes such as translation and rotation. Its formula is expressed as
Figure 2 shows the colour histogram of a single piece of insulator. The greyscale histograms under the three channels of R, G and B are shown in sequence from top to bottom. There is an individual difference in the colour histogram between the standard insulator and the faulty insulator. The colour of standard insulators under the R channel has a peak value near 0 and 30, while faulty insulators only have a greater probability of distribution near 0. The colour of standard insulators under the G channel occupies a large proportion in 0–10 and 150–200. The faulty insulator only has a peak near 0, and the distribution probability of 150–200 is almost 0. The changing trend of the two-colour distributions under the B channel is the same, but the distribution probability between 225 and 240 is still significantly different.
Fig. 2
Colour histogram of a single piece of degraded/standard insulator.
Stricker and Orengo propose the colour moment feature. It is a commonly used colour feature. It is widely used in the field of image processing. The advantage of this feature is that it has the lowest eigenvector dimension and lower computational complexity [6]. We can fully express the colour distribution of the image by using the first-order moment (mean), second-order distance (variance) and third-order moment (skewness) of colour information. In other words, colour moments are used to express colour characteristics. The mathematical model is as follows:
Fig. 3
Colour moment distribution diagram of a single piece of degraded/standard insulator.
The thermal image characteristics of porcelain insulators generally show that the temperature of the steel cap is higher than the temperature of the porcelain plate. The farther the porcelain disc is separated from the axis of the insulator string, the lower the temperature. The temperature directly affects the colour difference of infrared imaging. The colour value curve of the degraded insulator under our sampling line 1 is consistent with the overall trend of the standard insulator. The difference is only at the peak value; the B channel of the degraded insulator under-sampling line 2 has a more significant peak value in the 0–5 coordinate interval. This is because the part of the porcelain plate of the upper insulator is cut during the slicing, and there is no significant difference from the standard insulator. The difference between the normal insulator and the degraded insulator under our sampling line 3 is noticeable [8]. This is mainly reflected in the 0–15 coordinates. The R channel contains a more significant colour value component, and the G/B channel has a lower colour value component. This is because the heating part of the degraded insulator is mainly concentrated in the steel cap. At the same time, the infrared image of the higher temperature is red and white, and the steel cap of the standard insulator is mostly blue under the infrared image. Therefore, there is a big difference between the two. Comparing the three sampling lines, it can be seen that the centreline of the insulator contains rich information, the colour value curve changes perceptibly and the interference of the experimental background factors on the image information is effectively avoided. The experiment achieved data dimensionality reduction and background noise reduction based on ensuring the maximum amount of information. Therefore, this paper takes the colour matrix of the centreline of the insulator as the research object and extracts the pixel matrix under the RGB channel. The pixel matrix is used as the feature quantity to construct the fault diagnosis model of the BP neural network.
Since it is prone to overfitting and non-convergence, when designing a BP neural network, we should prioritise the topological structure of the ‘input layer-hidden layer-output layer.’ Based on this, this paper selects a neural network with a three-layer structure. The structure diagram is shown in Figure 4.
Fig. 4
BP neural network model.
The number of hidden layer nodes can be set precisely according to system requirements. However, if there are too few nodes, it becomes difficult for the network to establish complex mapping relationships, which will lead to poor network training effects, too many nodes and too long a learning time. This directly affects the convergence and efficiency of the network. After repeated trials, the results are shown in Table 1. Based on the colour moment, colour histogram and centreline colour matrix, this paper selects 10, 10 and 5 neurons as the number of hidden layers, respectively.
We first cut the collected infrared insulator string images into layers to form monolithic insulator sets. Then, the article randomly arranges the cut monolithic collections to avoid training contingency. Next, we use homomorphic filtering to remove noise interference and extract the pixel matrix under the three-channel RGB of the infrared image. After calculating the grey histogram under the three channels, the article considers the BP neural network model as the feature value [9]. The experimental results are shown in Figure 5 and Table 2.
Comparison of the accuracy of the number of neurons in different hidden layers
| Number of input neurons | Colour moment accuracy rate/% | Colour histogram accuracy rate/% | Centreline colour matrix accuracy rate/% |
|---|---|---|---|
| 5 | 86.87 | 83.88 | 93.37 |
| 10 | 87.82 | 87.58 | 93.37 |
| 15 | 87.70 | 85.37 | 93.13 |
| 20 | 87.76 | 82.69 | 93.13 |
| 30 | 87.88 | 85.67 | 92.84 |
Fig. 5
BP neural network prediction and actual value comparison under the colour histogram.
Diagnosis results based on the feature amount of the colour histogram
| Serial number | 1 | 2 | 3 | 4 | 5 | Average |
|---|---|---|---|---|---|---|
| Accuracy/% | 86.27 | 87.16 | 88.06 | 88.66 | 87.76 | 87.58 |
The X-axis indicates the predicted sample number, and the Y-axis indicates the sample prediction and actual results. 1 represents the detection as a fault state, and 0 represents the detection as a normal state. The asterisk represents the operating state of the experimental insulator, and the circle represents the test result after the diagnostic model. It can be seen from Figure 6 that the detection effect of this algorithm is relatively general. However, there are many false detections and missed detections. The reason for this situation may be that the colour histogram reflects the global colour distribution. When taking an infrared picture of the insulator string, it will be more or less affected by the background colour, making the extracted colour histogram challenging to train. In addition, due to the stochastic gradient descent method adopted by the BP neural network, each training result has an inevitable change [10]. But the overall fluctuation is within a specific range, so we train the experiment 5 times and take the average value as a reference. It can be seen from Table 2 that the average accuracy rate based on the colour histogram is 87.58%, and the detection effect is relatively average. Since the feature quantity selects the grey histogram under the RGB three-channel and the input quantity is 256*3=768, the training time is very long. This test took 590.160333 s. The pattern of the missed inspection is shown in Figure 6.
Fig. 6
Typical missed patterns under the colour histogram algorithm.
We will obtain the infrared insulator string image through cutting, random arrangement, standardisation, homomorphic filtering and other steps to obtain the monolithic insulator set to be processed. To calculate the original pixel matrix, we extract the colour moments under the RGB channel and use the first-order moment (mean), second-order moment (variance). Third-order moment (slope) values are used as eigenvalues to bring them into the BP neural network model. The experimental results are shown in Figure 7, Table 3.
Fig. 7
Comparison of BP neural network prediction results and actual values under colour moments.
Diagnosis results based on the colour moment feature quantity
| Serial number | 1 | 2 | 3 | 4 | 5 | Average |
|---|---|---|---|---|---|---|
| Accuracy/% | 88.06 | 86.87 | 89.25 | 88.36 | 86.57 | 87.82 |
As shown in Figure 7, the average accuracy rate based on the colour moment feature quantity is 87.82%, and the detection effect is average. More false detections of samples 315–340 are shown in Figure 8. The reason for our analysis is that the colour of some faulty insulators that have just started to heat up does not change significantly, and the colour moment still contains part of the shooting background content. This part of the pixel matrix becomes the main factor that interferes with the training effect. However, since the feature quantity selects the third-order colour moment under the RGB three channels, amounting to a total of nine input quantities, the training time is faster [11]. The training can be completed in 10.764295 s. Compared with the colour histogram, the accuracy of this method is almost the same, but the training time is immensely shortened. The pattern of the missed inspection is shown in Figure 8.
Fig. 8
Typical missed detection pattern under the colour moment algorithm.
We will obtain the infrared insulator string image through cutting, random arrangement, standardisation, homomorphic filtering and other steps to obtain the monolithic insulator set to be processed. Then, we calculate the original pixel matrix and extract the RGB of the centreline of the insulator as the eigenvalue and bring it into the neural network model. The experimental results are shown in Figure 9, Table 4.
Fig. 9
Comparison of BP neural network prediction results and actual values under the colour matrix of the insulator centre line.
Table 4 Diagnosis results based on the colour matrix of the centreline of the insulator
| Serial number | 1 | 2 | 3 | 4 | 5 | Average |
|---|---|---|---|---|---|---|
| Accuracy/% | 93.73 | 92.84 | 93.43 | 93.43 | 93.37 |
As shown in Figure 10, the average accuracy rate based on the colour matrix feature of the insulator centreline is 93.37%, which is better than the colour histogram and colour moment. There were almost no false detections in the test samples, and some of the faulty insulators were not heated because of the short energising time. The colour gap of the infrared image is small, and so it is not detected, which is also a consequence of the fact that this method only extracts the colour matrix of the centreline of the single insulator piece and is hardly interfered with by environmental factors and background colours [12]. Therefore, the detection efficiency of this method is relatively high. In addition, the training time of this method is shorter, and the time-consuming is 46.679750 s, which is only an increase of 30 s compared with the colour moment method. But the accuracy rate is greatly improved. The pattern of the missed inspection is shown in Figure 10.
Fig. 10
Typical missed detection pattern under the colour matrix of the centreline of the insulator.
This paper uses the representative infrared image data of insulators under different conditions obtained in the experiment to establish learning samples. 2835 sheets of monolithic insulator sets were obtained by layered cutting of pictures. We randomly select 2500 pictures as training samples and the remaining 335 pictures as test samples. The experiment extracts the colour histogram, the colour moment and the colour matrix of the insulator centre line as feature quantities for BP neural network training. It compares the training time and the prediction accuracy.
The diagnosis method based on the colour histogram shown in Table 5 has a large amount of input. It contains more comprehensive information, but because it reflects that the global colour distribution is easily interfered with by the background colour, the diagnosis effect is poor. The accuracy rate is 87.58%. Because the input matrix is too large, the training time is 590.160333 s. The diagnosis method based on colour moments has less input, the average accuracy is 87.82% and the training time is 10.764295 s. Compared with the colour histogram, both the training speed and the accuracy of diagnosis are better than the former. The diagnosis method based on the colour matrix of the centreline of the insulator effectively avoids the interference of experimental background factors on the image information. It achieves data dimensionality reduction and background noise reduction based on ensuring the maximum amount of information. The accuracy of the obtained prediction results reached 93.37%, and the training time was only 46.679750 s.
Comprehensive comparison of three diagnostic methods
| Feature amount | Training time/s-1 | Forecast accuracy rate/% | Overview |
|---|---|---|---|
| Colour histogram | 590.160333 | 87.58 | Short training time, average prediction accuracy |
| Colour moment | 10.764295 | 87.82 | Longest training time, average prediction accuracy |
| Centreline colour matrix | 46.679 | 750, 93.37 | Short training time and high prediction accuracy |
The article uses the pixel matrix under the RGB channel as the feature quantity to construct a BP neural network fault diagnosis model, which takes the colour matrix of the centreline of the insulator as the research object. The centreline of the insulator contains rich information, and the colour value curve changes significantly. Compared with traditional fault feature extraction methods, the interference of experimental background factors on image information is minimised. The experiment achieved data dimensionality reduction and background noise reduction based on ensuring the maximum amount of information. Experimental results show that this method can effectively determine the operating status of insulators. Furthermore, it has a shorter training time and higher accuracy, which provides an efficient and reliable diagnostic method for monitoring and analysing a single piece of porcelain suspension insulator.