Recent Advances in PCG Signal Analysis using AI: A Review

In healthy young adults, a PCG signal is characterized by the periodic occurrence of S1 and S2 Heart Sounds at repeated intervals of time as shown in Figure 1. Sometimes, normal Heart Sounds are accompanied by S3 and S4 sounds in abnormal subjects. The systolic time period ranges from starting of S1 to starting of S2. The diastolic time period ranges from the start of S2 to the start of S1; sometimes, it may contain S3 and S4 in case of abnormal Heart Sounds. The time duration of the diastolic period is around 240 ms, whereas the systolic period is 360 ms. One complete human cardiac cycle is composed of a systolic period and a diastolic period.

Figure 1:

Feature learning [1, 27] and categorization become very handy when dealing with Heart Sound signal analysis. However, many developments are made toward Heart Sound segmentation, feature extraction, cardiac disease identification, and categorization. Nevertheless, an adaptive model for denoising, preprocessing, and filtering background noises would still significantly help further processing in this domain, especially when working with Heart Sound captured in real-time, like diagnosis centers and labs. Feature learning has been done on captured Heart Sound signals using transformation methods and to analyze Heart Sounds with computing features such as the following:

Acoustic features: MFCCs, Mel, Chroma, Contrast, and Tonnetz

The Figures 1(a) and 1(b) show various steps involved in PCG signal analysis. The features of human Heart Sounds are extracted through computation, and after that, the categorization is done depending on various deep-learning methods.

Figure 1(a):

Figure 1(b):

The study was designed to keep the analysis and diagnosis part of the valvular Heart Sound. There exist different Heart Sound analysis methods but screening time, portability, and cost are the important parameters when selecting the proper one.

The study carrying part is carried out on selected volunteers of different age groups and genders. Formal consent was taken from the volunteers followed by the application of the real-time developed analysis method.

In the data analysis part, artificial intelligence-based intelligent systems have been used. Different Python based developed deep-learning algorithms are chosen and implemented for the analysis.

The motivation behind this study of PCG signal and its analysis is mainly divided into two parts: –

Challenges in PCG signal analysis.

The study would like to highlight a few recent significant works done by different researchers. The research focuses on identifying Heart Sound and their analysis. In this work, the signal categorization of the PCG signal is essential.

As per the literature study, many workers are researching in this field. Many researchers have described different algorithms and categorization methods for cardiac sounds. Randhawa and Singh [1] surveyed methods for automatically analyzing and classifying cardiac sounds. Gupta et al. [2] worked on identifying S1 & S2 Heart Sounds using a Stacked Auto Encoder with the combinatory feature for categorizing fundamental Heart Sounds. Then, the CNN approach is also used. This method has restrictions on analyzing and detecting S3 Heart Sounds. Lecun et al. [3] proposed an innovative algorithm to detect heart and lung sounds (HLS) based on the variational mode decomposition (VMD) of PCG signal. It is helpful to detect the third Heart Sound. However, this algorithm has limitations in real-time environment applications. Roy and Roy [4] proposed a new method of extracting PCG signals from noise through sparse recovery analysis. Barma et al. [5] did focus on the detection of second Heart Sound, which engages with computing the time duration and the energy of normalized instantaneous frequencies of A2s (Aortic valve) and P2s (Pulmonary valve), but it could not categorize cardiac sounds.

Tavel [6] and Muduli et al. [7] did work on Heart Sound analysis using the discrete wavelet transform (DWT) algorithm. It has specific restrictions in real-time analysis. Roy et al. [8] made PCG signal analysis using Shannon Energy Envelop and DWT method, but it cannot classify the Heart Sounds efficiently. Mishra et al. [9] made heart defect analysis and classification using Pattern Recognition Technique, but the method is based on echo imaging. Mishra et al. [10] made PCG signal analysis using classification by feature extraction but not in the real-time analytical process. El-Segaier et al. [11] made Heart Sound analysis and its categorization using a fuzzy classifier-based soft computing method. The laboratory study was done on a readily available online Heart Sound repository. It was an offline study and not validated with human volunteers. Nygaard et al. [12] studied the classification of PCG signals. Anju and Sanjay Kumar [13] did a segmentation and categorization of PCG signals for Heart Sound analysis. There is no significant work on developing real-time screening of valvular diseases. Buchanna et al. [152] studied the classification of EEG signals using fractal analysis and support vector regression. Ye et al. [151] analyzed multimodal medical images using semantic segmentation maps generated through deep learning. Many researchers studied PCG signal analysis; some of the corresponding algorithms are given below in Table 1.

The full review paper is structured as mentioned. Section 2 provides a background of different types of Heart Sound and their characteristics. Section 3 provides details of real-time PCG signal analysis. Section 4 explains the other methods used in Heart Sound analysis. Section 5 describes the deep-learning and machine-learning methods adopted in Heart Sound analysis. Finally, Section 6 summarizes the conclusions and future scope.

Different types of valvular Heart Sound disorders are explained below:

AS – Aortic valve could not open properly.

Figure 3 (a) depicts the block diagram of sensors used in the Heart Sound signal analysis. It also gives us an idea about how filters play a pivotal role in PCG signal analysis. Generally, low-pass, high-pass, and bandpass filters are used in the said process. New technologies like adaptive noise cancellers (ANCs) and adaptive line enhancers (ALEs) are used in the modern-day era to meet the requirements in a better-enhanced manner in terms of efficiency and stability.

Figure 3(a):

PCG signal in the chest generates pressure waves that the stethoscope diaphragm has picked up. The electret microphone fitted inside the chest piece and the stethoscope tube convert Heart Sound signal to electrical signal. This electrical signal is boosted by an audio amplifier based on a MAX 9812 IC chip of gain 20 and fed to the notch filter to remove electrical interferences. The processed signal is fed to an analog tunable bandpass filter with a 35–470 Hz spectrum. The Heart Sound signal typically belongs to 35–470 Hz for normal and abnormal sounds. Finally, the processed signal is converted to digital form in a USB sound card connected to a Raspberry Pi 4B computer. The Heart Sound samples are picked up in real-time by a stethoscope sensor and saved in WAV. format for further processing through Raspberry Pi 4B with developed deep-learning models, as illustrated in Figure 3 (b).

Figure 3(b):

Figures 4 and 5 illustrate the denoising methods used in the Heart Sound analysis and diagnosis. Table 3 highlights the various sensors used in the denoising algorithms adopted for the Heart Sound classification.

Figure 4:

Figure 5:

ANC is composed of two input signals from two separate sources. The first signal is noisy, and another contains ambient noise only. These two signals are subtracted at the subtractor circuit, producing a noise-free desired signal.

In ALEs, digital adaptive filters can change their characteristics depending on the input signal change. Adaptive filters generally use the least mean square (LMS) algorithm to work. It can be considered one of the better tools for the Heart Sound denoising process. Figure 5 shows an ALE where a delay block is also appended with the adaptive filter to obtain a noise-free Heart Sound signal.

Generally, Heart Sound banks are used in real-time valvular Heart Sound analysis. The following are the Heart Sound bank repositories: (a)

2016 PhysioNet/Computing in Cardiology (CinC) Challenge – Normal and Abnormal Heart Sound [76]

General preprocessing methods used for Heart Sound analysis are:

Normalization.

4.1.1

Preprocessing of Signal [40, 83]

The cardiac sounds are preprocessed first to generate the required Heart Sound data, as shown in Figure 6. Under the preprocessing stage, Heart Sound normalization occurs, followed by filtering and feature extraction. A typical normalization process is shown in Figure 7.

Figure 6:

Figure 7:

Normalization is expressed as shown in Eq. (1): (1) x(n)=xr(n)−xr(min)/xr(max)−xr(min) {\rm{x}}({\rm{n}}) = {\rm{xr}}({\rm{n}}) - {\rm{xr}}(\min)/{\rm{xr}}(\max) - {\rm{xr}}(\min) where xr(n) denotes the recorded PCG signal, and x(n) symbolizes the normalized Heart Sound signal [47, 51, 87]. At the same time, xr(min) and xr(max) are the lowest and highest values of the Heart Sound, respectively.

In filtering operation, second-order high-pass and low-pass Butterworth filters with 15 Hz and 700 Hz cut-off frequencies are used to filter the normalized PCG signal and remove unwanted background noise. Feature learning is used for the filtered signal as expressed in the literature [41, 45, 88]. The four commonly used envelop-energy-based algorithms employed in the production of features are: (1) Hilbert envelope, (2) Homomorphic environment map, (3) Wavelet envelope, and (4) Power spectral density envelope.

4.1.3

Feature Extraction

The segmented Heart Sound [89, 95, 97] goes to the feature extraction block where new features are extracted, which results in a reduced Heart Sound data size and eventually classifications based on these extracted features using CNN-based deep-learning [90, 96] methods. Figure 8 describes the reduction in data size of the PCG signal by the feature learning method.

Figure 8:

4.1.4

Signal Segmentation using Shannon Energy

The method based on the Shannon energy of the Heart Sound [125, 126] is considered for extracting the Heart Sound signals as shown in Figure 9. Shannon energy fetches the medium-intensity cardiac sound and removes the noise effect. It is the best way to extract the envelope of any cardiac sound signal compared to other methods, and the following terminologies are used to compute the different parameters: (a)

Shannon Energy: It is the average of the spectrum energy in the Heart Sound signal.

(b)

Shannon Entropy: It is the amount of information in the Heart Sound signal.

(c)

Absolute Value: The positive values of Heart Sound signal.

(d)

Energy (square) value: It is related to the square of the amplitude of the PCG signal.

Figure 9:

4.1.5

Feature Extraction and Classification

To differentiate between various kinds of cardiac sounds [127, 128] as normal or abnormal and then identify the abnormal sound as an MR or MS case, the following stages are considered in this method: (i)

Averaging energy envelop

(ii)

Finding averaged envelop peaks

(iii)

Fixing threshold (S1 and S2 duration computation)

(iv)

Murmurs detection and analysis

Figure 10 (a) illustrates the PCG of a typical cardiac sound, and Figure 10 (b) denotes the Shannon energy envelope of a typical cardiac sound. Figure 11 depicts the flowchart of the said algorithm used.

Figure 10(a):

Figure 10(b):

Figure 11:

The Shannon energy method can be a handful tool for identifying different types of Heart Sounds. Figure 10 gives a description of the feature extraction technique using the Shannon energy envelop method. In the flowchart, the preprocessed Heart Sound is checked for the threshold amplitude value (set by the user) and a cycle duration of less than 0.9 s. If it satisfies the condition, then it is a normal sound. Otherwise, the algorithm proceeds to the ascertainment of the S1 and S2 peak locations and the extraction of murmur. The murmur sound undergoes DWT. The 5^th level of DWT is selected in the frequency domain using a Fast Fourier transform. If the center frequency is lying at 200–250 Hz, it is an MR; otherwise, it is an MS.

In this method [40, 15, 25], Heart Sound signals are decomposed to a desired number of levels (n). The variance of detail coefficients is produced at the required levels. The computation has been made separately to normal, AS, AR, MS, and MR types of PCG signals. The schematic diagram of PCG signal analysis has been described in Figure 12.

Figure 12:

Different abnormal Heart Sounds are considered and analyzed. All Heart Sounds are taken from a standard cardiac repository available online. Figure 13 illustrates the time-domain representation of normal PCG signal, AS, AR, MS, and MR in the time domain.

Figure 13:

Figure 14 (a) illustrates a generalized block diagram of the DWT. In this Transform, the input signal initially passes through two types of filters: high-pass and low-pass. The high-pass filter output is known as the detail coefficient, and the low-pass filter output is represented as the approximation coefficient. The low-pass filter output is further passed through two filters, thus generating another set of detail and approximation coefficients. This process continues up to the required level n.

Figure 14(a):

Figure 14 (b) describes the flowchart of DWT where a signal S is at first divided into CA1 (approximation coefficient at level 1) and CD1 (detail coefficient at level 1). CA1 is further divided into CA2 and CD2, and this process continues until the desired level is reached. In this way, different PCG signals can be classified using DWT, as illustrated in Figure 15.

Figure 14(b):

Figure 15:

Various types of PCG signals are analyzed and plotted in Figure 15 using DWT. The Heart Sounds are plotted against the amplitude vs. time domain of the Heart Sound samples. Systolic murmurs and diastolic murmurs can be easily observed from the said characteristics plot.

The DWT [8, 24, 40] is a conversion method to analyze the temporal and spectral features of non-stationary signals like sound. The PCG signals are convenient for identifying the different types of valvular heart diseases. ECG readings are not good enough to find information related to the heart valves. This paper uses DWT to categorize normal and various abnormal Heart Sounds without getting into the hectic segmentation method. The proposed method is quickly adopted using a stethoscope without the help of ECG readings. In this paper, Heart Sounds are acquired through data acquisition and decomposed into n number of levels using DWT. Then variance of detail coefficients is calculated. The variance of detail coefficients at the required number of levels will form a feature set to classify normal Heart Sound, abnormal Heart Sound, and cardiac murmurs. Using DWT, any Heart Sound is represented in terms of Detailed Coefficients and Approximate Coefficients. Figures 16 (a) and (b) highlight the description of various Heart Sounds [123, 124] using time-domain and frequency-domain analysis that uses DWT.

Figure 16:

In Harmonic-Distribution [43, 44, 1], the amplitude and phase may be ignored due to negligible values. Harmonic-Distribution is produced initially by transforming the time-domain Heart Sound into the frequency domain and then by taking out the real and imaginary components separately.

Stages for computing the Harmonic-Distribution of a recorded cardiac sound signal:

Load the cardiac sound using Python in the time domain.

For understanding, a harmonics order of up to 40 is selected. The following distributions provide the Harmonic-Distribution of Amplitude for normal and abnormal Heart Sound as well as for Cardiac Murmurs [119, 120].

Figures 17 (a–c) highlight various Harmonic-Distributions of normal, abnormal, and murmur Heart Sound.

Figure 17(a,b):

Figure 17(c):

Figure 18 illustrates the classification of various types of cardiac sounds. Murmurs have a smaller amplitude in distribution, while abnormal sounds have a higher amplitude in distribution. This method can identify and classify multiple types of Heart Sounds accurately.

Figure 18:

Figure 25 compares the deep-learning-based classification [39, 50] method used for valvular Heart Sound analysis with the traditional machine-learning methods. It highlights low-level features used for deep learning and handcrafted high-level features used for conventional machine learning [85, 86].

Figure 25:

It is the first Convolutional Neural Network model developed in deep learning. It has five layers comprising three convolutions and two fully connected layers. This network also includes two max-pooling layers and one Softmax layer, as illustrated in Figure 26.

Figure 26:

It is the second Convolutional Neural Network model developed in deep learning. It has eight layers comprising five convolution layers and three fully connected layers. This network also contains three max-pooling layers and one Softmax layer, as depicted in Figure 27.

Figure 27:

It is the third Convolutional Neural Network model developed in deep learning. It contains 16 layers comprising 13 convolution layers, and 3 fully connected layers, as illustrated in Figure 28.

Figure 28:

It is the fourth Convolutional Neural Network model developed in the field of deep learning. It has 19 layers comprising 16 convolution layers, and 3 fully connected layers, as illustrated in Figure 29.

Figure 29:

It is the fifth Convolutional Neural Network model developed in the field of deep learning. It has 121 layers comprising 120 convolution layers and 1 fully connected layer, as illustrated in Figure 30.

Figure 30:

It is the seventh Convolutional Neural Network model developed in deep learning. It is named after squeeze and expand layers. Generally, it is 18 layers deep, containing fire blocks. Fire blocks comprise the desired number of squeeze and expand filters, as illustrated in Figure 31. Squeeze layers have 1 × 1 filters, whereas the expanded layer combines 1 × 1 and 3 × 3 filters.

Figure 31:

The mobile net is compatible with mobile operating systems and can be embedded in mobile devices. The modified CNN-based mobile network [6, 15] is used in this research. It is based on the idea of depth-wise separable convolution, where convolution takes place in two steps, namely those of depth-wise and point-wise convolutions. Depth-wise separable convolution takes less complexity of the model as the computations become lower in the mobile network. Figure 32 shows a modified mobile network model module, where depth-wise convolution uses a 3 × 3 kernel, batch normalization, and ReLU activation function, followed by a 1 × 1 normal convolution [9, 18], batch normalization, and ReLU activation function.

Figure 32:

It is a sparsely connected network that runs deep. It contains multiple numbers of inception modules connected in series. The inception module comprises the max pool and various filter sizes of 1, 3, and 5 at the same layer. Thus, multiple convolutions occur at the same layer, followed by a concatenation operation, as shown in Figure 33.

Figure 33:

It is an intense network that uses skip connections. As the number of layers increases, an issue called a vanishing gradient occurs. This problem can be resolved by residual blocks used in the residual network, as illustrated in Figure 34.

Figure 34: