Research on Multi-Sensor Fusion Firefighting Manpower Status Monitoring Technology
Published Online: Mar 15, 2024
Page range: 99 - 106
DOI: https://doi.org/10.2478/ijanmc-2023-0070
Keywords
© 2023 Wanying Dang et al., published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Firefighting safety is a matter of life and death, and the safety of firefighters is of paramount importance. According to the information of firefighters in recent years, the average annual death of about 30 people, to protect the lives of firefighters, in order to improve the safety of firefighters and the efficiency of performing tasks multi-sensor information fusion technology is applied to the firefighters’ equipment in order to realize the comprehensive monitoring and analysis of the fire environment [1]. Multi-sensor data fusion can more accurately and graphically describe the observation object [2], providing new ideas for the safety and security of firefighters’ single soldiers. Taking the research project of Xi’an University of Technology as an example, the decision-making fusion of multi-sensor data needs to be transmitted to the command center. Therefore, the first step is to complete the multi-sensor data fusion for firefighters.
The firefighting single soldier is the main body of firefighting work [3], the use of multi-sensor detection and fusion of the target, break through the limitations of single-sensor measurement, to avoid the blind spot of single-sensor information, and to improve the accuracy of measurement. We need to solve the following problems: First of all, to monitor and protect the firefighting singletons we must get their physiological data, and pulse and ECG are the basic indicators of vital signs monitoring, reflecting the firefighting singletons’ physical condition and movement status; Then, only physiological sensors can not determine the firefighting singletons’ status in special cases, and we need to combine with the firefighting singletons’ posture data to improve the accuracy of the results; At last, a single sensor can not guarantee the accuracy and robustness of information fusion, multi-sensor information fusion can ensure the accuracy and practicality of information fusion, through multi-level, multi-space information complementary and optimized combination of processing, multi-sensor data fusion ultimately produces a consistent interpretation of the firefighting soldier[4]. In this paper, a multi-sensor data fusion decision-making algorithm based on firefighting singletons is designed,, so as to provide scientific technical support and guidance for the practical application of firefighting singletons.
The basic principle of multi-sensor data fusion is just like the process of comprehensive information processing in human brain, it makes full use of multi-sensor resources, the complementary and redundant information of various sensors in space and time is optimized according to a certain optimization criterion, which produces a consistent interpretation and description of the observation environment. The research results of this paper include the following, the acquisition of pulse and ECG time and frequency domain indicators in physiological information, and simulation of firefighting single-armed operation intensity experiments to obtain pulse, ECG, and posture data, to extract posture features and identify and classify postures; the acquisition of the relevant physiological dataset and posture dataset to analyze the above indicators of the correlation between safety and security, in order to determine the feature indicators based on the final safety decision-making model.
In the Physiology and Sports Research Laboratory of a university in the U.S.A., a special fireproof shirt was designed in which a physiological state monitor-PSM was embedded[5].The purpose of this study was to test that while performing various activities related to firefighting, firefighters can accurately detect the heart rate HR and the respiratory rate RR by wearing the device, which allows for real-time analysis of firefighters’ health status information to monitor the physiological status of the firefighters[6]. In 2015, a wearable smart system was proposed by the French National Institute of Higher Arts and Textile Industries. In 2020, Literature [7] provides an overview of existing technologies and solutions for smart textiles and sensing garments for physiological monitoring and suggests that this can be achieved by applying textile technology to sensing garments in which the sensors are fully embedded in the fabric.
Shufeng Yang et al [8] designed and developed a firefighter vital signs detection system for real-time firefighting singletons in the exercise state with information on respiration, heart rate, body temperature, activity intensity, and posture. In February 2016, a smart wearable system was proposed to protect firefighting singletons and to deliver real-time information on the physiological status of firefighting singletons. The proposed smart wearable system includes heart rate and temperature information detection, gas concentration sensors, and acceleration sensors, which are combined to judge and estimate the overall physical condition of the firefighter. The development of the domestic wearable sensor system in recent years is still relatively rapid, commanders need to effectively judge the on-site environment according to the equipment of the firefighter’s single soldier, do a good job of surveying, and in the process of making judgments, you can use the relevant equipment to enhance the scientific nature of the command with the support of intelligent technology to ensure the safety of the on-site firefighters [9].
The firefighting single soldier safety and security system includes wearable sensor terminal, remote information transmission and intelligent control platform. The composition of this firefighting manpower safety and security system is shown in Figure 1.
Figure 1.
Firefighting man-portable safety and security system
Wearable sensor terminal is the carrier of this system, mainly based on a variety of wearable sensors, can collect firefighting soldiers for physiological information, movement status, location information, and at the same time has the function of remote information interaction. Remote information transmission is the data interaction between the wearable sensor terminal and the control platform based on wireless communication, the combat firefighters through the sensor terminal in the form of data flow through the form of radio uploaded to the control platform, so that the control center personnel can understand and analyze the scene situation in a timely manner. The control platform will release the fire emergency plan and all kinds of instructions through wireless communication, realizing real-time interaction between firefighters and the command center.
The control platform realizes data collection, organization and analysis. Through the information interaction between combat firefighters and the control platform, it realizes the management of firefighting command and firefighting rescue. The control platform can not only monitor the scene in real time, but also has the functions of firefighting operation planning, professional and technical navigation and daily personnel management [10].
As shown in Figure 2, it is a flowchart of the design of the safety and security of the firefighting single soldier consisting of three components, namely the equipment needed for data acquisition, the processing of the different equipment after data acquisition and the fusion of multi-sensor data. The three components are described in detail next.
Figure 2.
Design flowchart for firefighting manpower safety
The data acquisition and processing process is categorized into physiological and postural approaches, and the data the purpose of data preprocessing is to fill in missing values, smooth or remove abnormal values, and correct data inconsistencies [11]. The reason for feature extraction in physiological data acquisition process is the individual variability of firefighting soldiers and the different physiological functions of the human body, and then through the threshold value to the firefighting soldier’s state of the division, and finally get the results of the current state; posture data acquisition process of feature recognition refers to the extraction of firefighting soldier’s posture or behavior of the special information, and through the posture recognition model to the firefighting soldier’s posture of the classification, and finally get the results of the current posture, the result of the current posture.
Due to the good accuracy and high sensitivity of the posture sensor, the amount of data collected in a short period of time is relatively large, so this paper chooses the support vector machine algorithm that can solve the problem of small samples, and after pre-processing the original more than 30,000 groups of posture samples, only two thousand groups of posture samples can have a good classification effect on the two categories of non-falling and falling postures. And we choose to fix the posture sensor at the position of waist which has higher accuracy [12]. In this paper for the firefighting soldier posture recognition is mainly to improve the accuracy of judging the state of the firefighting soldier, all the use of simple binary classification with which the combination of posture features are also more obvious, the use of small samples and a simple SVM algorithm model for posture classification to give a result with an accuracy of up to 99.06%.
In multi-sensor data fusion, by collecting multi-level and comprehensive information about the target and combining it with appropriate fusion methods, the final information obtained is far more comprehensive and reliable than that obtained by using a single sensor [13]. A BP neural network and an improved BP neural network are used for multi-sensor data fusion, and the states of firefighting soldiers are categorized into two major categories: “normal state” and “event state”, and the “event state” is categorized in detail. The “event state” is also categorized in detail. The “ordinary state” is the safety state, and the “event state” is divided into abnormal state and warning state, i.e., safety, abnormality and warning.
The bp neural network model determines the input and output levels of a contract according to the known and predicted data types. The data signals are transmitted from the input layer to the hidden layer centered in the neural network, where the data is learned and processed. The processed data signals are transmitted to the output layer. If the results do not match expectations, errors spread somewhere and hidden weights and thresholds return to expectations [14].
Figure 3.
Schematic diagram of BP neural network and security
The problem of the traditional BP algorithm is that the BP network uses local optimization algorithm, and the objective function surface of the multilayer network structure is generally more complex, which leads to local convergence of the network, and the optimal solution of the problem cannot be obtained. Therefore, two improved methods are proposed to solve the problems of traditional BP: the improved fireworks algorithm can solve the defect of local optimization, and the fireworks algorithm is simple and has a wide search range.
The fireworks algorithm contains three operations: explosion, variation and selection, each firework will explode and produce several sparks, some sparks have a certain probability to produce special sparks, and the sparks will be the variation sparks. Sparks are distributed randomly and evenly around the spark according to the amplitude of the current spark, while mutant sparks are generated in a normal distribution around the current spark. Each spark in the fireworks algorithm has two properties: amplitude and quantity. The brighter the fireworks (the better the target), the smaller the amplitude [15]; The brighter the spark from the explosion (the better the target), the greater the number of sparks produced by the explosion.X1,X2,…,Xi is the input value of the neural network, Y1,Y2,…Yj is the output value of the neural network, the number of neurons in the input layer is I, ωih is the weight of the connection between the input layer and the hidden layer, ωho is the connection weights from the hidden layer to the output layer [16]. f1 is the hidden layer activation function and b1 is the hidden layer threshold. Firework is an improvement on the BP neural network back propagation.
The number of sparks in the explosion Eq (1) and (2):
The formula (3) and (4) for the amplitude size of a spark:
Among them, Smax is the maximum number of sparks generated by the explosion, Amax is the maximum amplitude, ξ is a small real number,a is the minimum spark generation ratio, b is the maximum spark generation ratio, r is a random number between -1 and 1 [17].
In this paper, using the loss (Loss) and accuracy (Accuracy) of the algorithm model to evaluate. The calculation formula is as follows:
The flowchart of the improved Firework-BP algorithm is shown in Figure 4.
Figure 4.
Flowchart of the improved Firework-BP algorithm
Posture data collection: the sensor device needs to be worn comfortably and cannot affect the freedom of movement of the firefighting soldiers during operation. Due to the special nature of the firefighting soldiers’ work and the uniqueness of the firefighting soldiers’ movements during work, the position and number of wearing posture sensors should be constrained under the circumstance of guaranteeing the accuracy. Twenty subjects were selected to conduct the following experiments respectively. The BWT61CL posture sensor was fixed in the center of the back side of the subject’s waist, and after many experiments, it was found that wearing it on the waist of the human body was the most ideal [18], and the posture data collection was divided into three forms: standing, free movement and falling. Each subject’s data collection time for each group was 15s, four groups for each standing and activity, eight groups for the falling state, and standing and free activity were categorized as the upright state. One sample of the posture sensor contained nine data of angle x, y, z (unit: g), acceleration x, y, z (unit: °/s) and angular velocity x, y, z (unit: °).
Physiological data collection: There is a gap between the physiological state data of firefighting soldiers collected in the laboratory and the state data of real firefighting operations. In order to ensure the objectivity of the data testing, 20 male graduate students, whose height is not less than 170cm, whose weight is not more than 20% of the standard weight for men and not less than 10% of the standard weight for men, and who are in good health, were selected; all participants were provided with a written informed consent form, and was informed of the specific contents of the experiment and the precautions to be taken when collecting data, to ensure that the data collected could be effectively used for the experiment. The data were collected in a manner that ensured that the data could be used effectively in the experiments. Physiological data were collected using the ErgoLAB pulse PPG sensor and the ErgoLAB pulse ECG sensor. The pulse PPG sensor is worn with a black ear clip attached to the earlobe and a strap secured to the arm; the ECG sensor is worn with three electrode pads attached to the three electrodes of the sensor and a strap secured to the subject. In order to increase the situational variability and realism of the data, the types of data collected during the firefighting manoeuvre were categorized into low-risk, medium-risk, and medium-high-risk in order to collect a wider range of physiological state reference values.
The pulse PPG sensor is shown here, and the light intensity it detects changes pulsatilly with the heartbeat. This light intensity change signal is converted into an electrical signal, which reflects the change of the blood flow after amplification. The signals analyzed in the experiments were the R-R period, which indicates the heart rate or interbeat period, and the P-P period, which reflects the heart rate cycle of sinus rhythm, but it is difficult to accurately detect the P-P period, so the R-R period was used in the actual detection. The R-R interval is usually equal to the P-P interval.
The electrocardiographic ECG sensor, shown in the figure, detects signals as the heart is stimulated sequentially by the pacemaker, atria, and ventricles during each cardiac cycle. Along with the bioelectric changes, various forms of potential change patterns are mapped from the body surface by means of a cardiac tracer. The human heart rate (R-R spacing) fluctuates under normal conditions, and the R-R spacing generally varies by tens of milliseconds. Which responds to the changing pattern of the heart’s rhythm, and is a commonly used indicator for the study of cardiac autonomic function.
The attitude sensor can recognize the attitude of all parts of the limbs and torso of the firefighting soldier, integrating a high-precision three-axis gyroscope, three-axis accelerometer and three-axis Euler angle, using a high-performance microprocessor and advanced dynamics solving and Kalman dynamic filtering algorithms [18].The output frequency in the attitude sensor is 100Hz and the baud rate is 115200bps.The X-axis, Y-axis and Z-axis in the attitude sensor correspond to roll angle, pitch angle and yaw angle respectively.
Experiments have been conducted on the data fusion decision based on the firefighting single soldier in the fireground operation of multi-sensors on the BP neural network algorithm and Firework-BP neural network algorithm. In this paper, using the loss and accuracy of the algorithm model to evaluate.
The Loss and Accuracy plots of the BP neural network for data fusion are shown in Figure 5 and Figure 6.
Figure 5.
Loss plots for BP data fusion
Figure 6.
Accuracy plots for BP data fusion
The Loss and Accuracy plots of the Firework-BP neural network for data fusion are shown in Figure 7 and Figure 8.
Figure 7.
Loss plots for Firework-BP data fusion
Figure 8.
Accuracy plots for Firework-BP data fusion
As can be seen from the above figure, the number of iterations for both models is 800, the loss of the validation set after data fusion of the improved Firework-BP neural network model is smaller than that of the BP neural network model, and the accuracy of the validation set after data fusion of the improved Firework-BP neural network model is higher than that of the BP neural network model, which proves that the Firewor-kBP neural network algorithm proposed in this paper has a higher accuracy of multi-sensor data fusion. The loss decreases the fastest when the iteration is almost 100 times, the accuracy of the BP neural network model increases the fastest in the 100th and 300th times, and the accuracy of the Firework-BP neural network model increases the fastest in the 80th and 200th times, and it can be clearly seen that the stability of the improved Firework-BP neural network model is better by comparing the accuracy graphs of the two models. In this paper, the accuracy of the algorithm model is also better when the sudden failure problem of one of the sensors is considered, and the robustness of the algorithm model is better.
Firefighting singletons are the main body of firefighting work, fire safety and security system effectively protects the safety of firefighting singletons when they are operating in the fire scene, using multi-sensors for target detection and fusion, verifying the algorithm’s role of enhancing firefighters’ safety and task efficiency through experiments and data analysis, and providing scientific technical support and guidance for the practical application of firefighting singletons.
In this paper, multi-sensor data fusion based on Firework-BP neural network algorithm is used to judge the physical state of firefighting singletons, which not only improves the low accuracy of the judgment of a single class of sensors, but also effectively improves the stability of the BP neural network algorithm. The vital signs of firefighters are categorized into three types: safety, abnormality and warning, alert firefighters with warning vital signs, remove firefighters from the fire scene if necessary, minimizing the risk factor for single firefighters, and rescue and resuscitate firefighters whose vital signs are in danger, and the state of firefighting singletons is predicted according to the trained model, which can effectively improve the decision-making on the safety and security of firefighting singletons through multi-sensor data fusion and improve the stability of the system. The accuracy of the improved Firework-BP data fusion algorithm model is improved by 3.64%.It is verified that the improved model proposed in this thesis is effective and reliable, and can be widely applied in the actual operation of protecting firefighting singletons, which can effectively improve the personal safety of firefighting singletons, Contribute valuable ideas and effective methods for future research in the area of protection of the firefighting unit.