The fidelity of compressed and interpolated medical images

Due to the amount of medical image data being produced and transferred over networks, employing lossy compression has been accepted by worldwide regulatory bodies. As expected, increasing the degree of compression leads to decreasing image fidelity. The extent of allowable irreversible compression is dependent on the imaging modality and the nature of the image pathology as well as anatomy. Interpolation, which often causes image distortion, has been extensively used to rescale images during radiological diagnosis. This work attempts to assess the quality of medical images after the application of lossy compression followed by rescaling. This research proposes a full-reference objective measure of quality for medical images that considers their deterministic and statistical properties. Statistical features are acquired from the frequency domain of the signal and are combined with elements of the structural similarity index (SSIM). The aim is to construct a model that is specialized for medical images and that could serve as a predictor of quality.


Introduction
Remote medical record access has gained popularity in the healthcare industry as it enables remote diagnosis, quicker access to a specialist, remote monitoring and makes the communication between physicians regarding a patients' medical history faster. There has been a lot of advances in computerized diagnosis methods. A classification system for grading cancer malignancy is one objective of such modern computerized diagnosis Jeleń, Lipiński, Detyna & Jeleń, 2011). Moreover, optimized machine learning techniques are shown to be very useful in the detection of coronary artery disease (CAD) (Abdar et al, 2019).
Transforming proprietary medical imaging software into web and mobile enabled platforms is a recent example of progress in medical imaging technology. The speed limitations of existing networks and access time along with the explosive growth of image modality technologies with extremely high volume outputs have combined to make the issue of irreversible medical data coding one of the key considerations in the design of future picture archival and communication systems (PACS). Irreversible image coding may include compression, interpolation, bit depth reduction and other image manipulation techniques. The benefit of using irreversible coding is improved system performance acquired by reducing the time to access and display image data to the user. The trade-off of using irreversible coding is reduced image quality. Table 1. Recommended compression ratios for medical images in UK, Canada and Germany (Strintzis, 1998).
A medical image undergoes several irreversible operations before it is displayed to the user. These operations may include extracting the image content from the DICOM format, transforming the original 16-bit image to an 8-bit image by means of the window levelling operation, compressing the image for faster transmission using lossless or lossy algorithms and manually resizing the image using an interpolation algorithm. The above operations cause a loss of image quality and may introduce undesired artifacts. For the purpose of displaying medical images in browsers and mobile enabled platforms, all of the above operations are often applied to a medical image before it is displayed to the user. There exists a body of literature focusing on the effects of one lossy operation applied to medical images (Cosman, Gray & Olshen, 1994); however, the effects of multiple irreversible coding have not been largely studied. This work is concerned with the assessment of image quality after it has been degraded by two operations -lossy compression and interpolation -which irreversibly alter image quality and may have an impact upon diagnosis.

Lossy compression and resizing of medical image
Lossy image compression often results in the distortion of the images and therefore creates the risk of losing or altering relevant diagnostic information. Irreversible medical image compression is one of the key considerations in the design of PACS systems (Wang, Bovik, Sheikh, & Simoncelli, 2004& Wang & Li, 2010. As expected, increasing the degree of compression of an image leads to decreasing fidelity. The extent of allowable irreversible compression is dependent on the imaging modality and the nature of the image pathology and anatomy. Image compression could result in distortions of the visual quality of images, and therefore it introduces the risk of losing or altering important diagnostic features. Due to the concerns that lossy compression could lead to an invalid diagnosis, a lot of effort has been put into studying the effects of compression and the adoption of lossy compression for medical images. In particular, the use of JPEG and JPEG2000 compression methods for various modalities and anatomical regions has been extensively researched (European Society of Radiology, 2011). The reason for this special attention given to JPEG and JPEG2000 compression algorithms is that these are the compression formats allowed in the digital imaging and communications in medicine (DICOM) format, which is the standard for the transmission and storage of medical images. As a result, recommendations for allowable compression ratios for medical images have been adapted and published by several radiological societies around the world (European Society of Radiology, 2011). The published recommendations all suggest that the use of irreversible image compression is possible without the loss of relevant clinical features required for diagnosis. However, the level of appropriate irreversible compression depends on the modality of the image, anatomy, and the nature of the pathology. Table 1 shows compression ratio recommendations based on body part and image modality for Canada, UK and Germany.
Interpolation is extensively used by radiologists to rescale images during radiological diagnosis, for treatment purposes, surgical operations or radiation treatment (Lehmann, Gonner & Spitzer, 1999;Meijering, 2000). Besides image rescaling, interpolation of sample data is necessary in a number of digital image processing operations including sub pixel translation, rotation, elastic deformation and warping (Meijering, 2000). These operations are performed during image reconstruction and registration for the purpose of radiological diagnosis, computer aided diagnosis (CAD), computer assisted surgery (CAS) and in picture archiving and communication systems (PACS) (Lehmann, Gonner & Spitzer, 1999). For example, in computed tomography (CT) or magnetic resonance imaging (MRI), interpolation is used to approximate the discrete functions to be back projected during image reconstruction. Moreover, in modern X-ray imaging systems, such as digital subtraction angiography (DSA), interpolation techniques are employed during image registration in order to enable the alignment of the given radiograph and the mask image (Thévenaz, Blu & Unser, 2000).
Interpolation in the context of this work refers to producing a larger version of an image, a high-resolution image, by adding new pixels to the existing image. The interpolated pixels are obtained by convolution of a linear interpolation filter. Convolution involves linearly combining the known pixels with some weighted functions that satisfy certain properties and are known as convolution kernels. Even for the same image, different interpolation techniques can produce images that differ significantly. Interpolation is only an approximation and therefore an image will always undergo some loss of quality when interpolation is performed. The most common artifacts resulting from this estimation include blurring, edge distortion, ringing and aliasing. Figure 1 illustrates interpolation effects on the natural standard Lena image using the most common interpolation methods. For natural images, the degradations introduced by interpolation may impact their visual quality; moreover, in the case of medical images, interpolation may also have an impact on their diagnostic quality.
The effects of resizing a medical image may have an impact on diagnosis.
Blurring that appears as out of focus regions could cause very small structures to vanish. Distortions due to aliasing could result in the loss of structural information, i.e. they could cause changes in texture. Another example of interpolation effects that could lead to misdiagnosis is the ringing artifact appearing as "oscillations" and is attenuated near edges (Thévenaz, Blu, & Unser, 2000). It is important to mention "pixelated" images -the result of the "blockiness" effect, which is associated with poor quality due to the loss of detail, especially near edges. Figure 2 shows the original and a magnified region of a brain CT image using the bilinear interpolation technique with a scaling factor of 4:1. The visible artifacts in the magnified image include blurring and distortions around edges.
Most interpolation techniques were designed for general sets and therefore they do not necessarily correspond well to human visual perception. Although a large amount of research on interpolation methods exists, evaluations of the effects of these techniques for the specific purpose of medical imaging are still lacking (Lehmann, Gonner & Spitzer, 1999). The performance of a general interpolation technique is related to the support and the approximation order of the convolution kernel. In practice, however, the best choice of an interpolation method is a trade-off between quality and computation time.

Objective image quality assessment
Many objective image quality metrics have been proposed in the last decade. Due to the wide variety of image types and applications, image quality assessment is not (yet) fully automatic and subjective approaches remain predominant (Naït-Ali & Cavaro-Ménard, 2008). How do we measure diagnostic quality? It is the pathological condition that determines the information that must be retained in any given medical data. In order to avoid misdiagnosis, it is absolutely necessary to correctly identify and measure degradations caused by irreversible operations that an image undergoes before it is displayed to the user. In order to do so, we need reliable quality assessment methods. This work focuses on assessing the quality of a medical image that was resized after lossy compression had been applied to it.
The most reliable comparison of interpolation algorithms would be to run time-consuming subjective tests involving a number of radiologists assessing the quality of each interpolated image. A faster way to measure the performance of interpolation techniques would be to use an objective quality measure, which would automatically predict image quality using a numerical algorithm. Fullreference image quality measures require a reference image which is assumed to be of original quality and a degraded image of the same resolution as the reference image. Objective full reference quality assessment for compressed images has been the subject of research for decades. In the case of magnified images obtained using interpolation, automatic comparison is impossible since there is no one-to-one mapping between the original image and the interpolated image. The original image has lower resolution than its magnified version and therefore cannot be used as a reference image. Thus, it is not clear how to measure the performance of interpolation algorithms using a formula such as the mean square error (MSE) or the structural similarity index (SSIM). Apart from this, there is no objective quality model that has yet been established specifically for medical images.
In order to overcome the lack of the possibility of direct pixel referencing, some workarounds have been suggested in the literature (Lehmann, Gonner & Spitzer 1999;Meijering, Niessen & Viergever 2001). Some attempts to make this comparison possible include creating a lower resolution image from the original image (using some downsizing techniques), interpolating the lower-resolution image and comparing it to the original image. Another approach may involve acquiring the same image at different resolutions. The high-resolution image is regarded as the reference image. The low-resolution image is interpolated to obtain the same resolution as the reference image. With this approach, objective comparisons could be performed directly at the higher resolution. The problem with this approach, however, is the difference of the signal-to-noise ratio and movement artifacts (a very serious problem for medical images) between the two acquired images. Although the signal-to-noise ratio can be adjusted, it is unclear whether this approach would produce reliable results.

Image quality assessment methods
A number of full reference image quality assessment algorithms are proposed in the literature. A lengthy review of objective image quality was performed by George and Livingston (George & Livingston, 2013), where the most common image quality measures -MSE and SSIM (Wang & Bovik, 2009) -are discussed. According to Marmolin (Marmolin, 1986): "MSE is not very valid as a quality criterion for pictures reproduced for human viewing and the improved measures could be derived by weighting the error in accordance with assumed properties of the visual system." MSE is shown to have poor correlation with visual quality; however, it should not be assumed that any other objective quality measure must be more accurate. Moreover, it cannot be assumed that an objective quality measure that shows satisfactory performance for natural images would guarantee an acceptable diagnostic quality for medical images. The SSIM index and other image quality measures show better performance than MSE for natural image and video content in consumer electronics applications based on subjective tests (Strintzis, 1998;Thévenaz, Blu & Unser, 2000). Moreover, SSIM shows that it corresponds better with subjective radiologists' responses than MSE (Kowalik-Urbaniak, 2014;Kowalik-Urbaniak et al, 2014). In spite of its rather poor performance, MSE has been commonly used in medical image quality assessment.
MSE is related to the L 2 distance between two image functions. The MSE between the distorted image g and the original image f is given by: (1) The SSIM index was introduced by Wang and Bovik (Wang & Bovik, 2009). It assumes that the human visual system (HVS) is highly sensitive to structural image distortions such as JPEG blockiness, "salt-and-pepper" noise, the ringing effect or blurring. However, the HVS automatically adjusts to non-structural distortions such as luminance/spatial shift and contrast change. An important assumption of the SSIM index is that images are significantly structured with strong neighbouring pixel dependencies, which are ignored by the MSE. The SSIM index combines three components of HVS to measure the difference/ similarity between two images: luminance -l(f,g), contrast -c(f,g) and structure s(f,g). The (local) SSIM is given by: (2) where μ is the mean, σ f 2 is the variance and σ f g is the covariance. The parameters C 1 ,C 2 and C 3 , represent stability constants. Local SSIM index is computed over m×n pixel neighbourhoods. The final SSIM quality score is averaged over all local measurements. Since SSIM measures the similarity between f and g, the closer f and g are to each other, the closer SSIM(f,g) is to the value 1. If f and g are identical, then SSIM(f,g)=1. The SSIM behaves as follows: and SSIM(f,g) = 1⟺f = g. (4) The parameters C 1 ,C 2 and C 3 , represent stability constants of a relatively small magnitude. These stability constants are introduced in order to avoid numerical "blowups", which would occur if a very small denominator is encountered.
In the special case C 3 = C 2 /2, a simplified, two-term version of the SSIM For natural images, the default values for these parameters are proposed in (Kowalik-Urbaniak et al, 2015). It is important to mention that smaller values of these stability constants increase the sensitivity of the SSIM index to small image textures (i.e. noise). In the literature (Kowalik-Urbaniak, 2014), a range of stability constants was proposed for the optimal assessment of the diagnostic quality of compressed medical images. A helpful feature that the SSIM index provides is a meaningful quality map. This map reveals local image quality variation over space, which may give insights to explain the differences between compression algorithms and develop better image compression algorithms for medical images.

Methodology
In this study, the quality of sixty interpolated compressed and uncompressed neuro-and body CT images (512 x 512 pixels) was evaluated objectively using MSE and SSIM, and Qual ind -a proposed measure that is based on the deterministic and statistical information of the signal and is defined in the next section of this paper. The compression ratios used are 2:1 to 35:1 for both JPEG and JPEG2000. The procedure involved magnification at two factors -2:1 and 4:1. The magnified images were then interpolated again back to their original size (512 x 512 pixels) using the same interpolation technique. The resulting https://doi.org/10.37705/TechTrans/e2020031 images were compared to their original counterparts using MSE, SSIM and the proposed measure of quality Qual ind . The scaling procedure was performed in such a way that the scale using a given interpolation algorithm along X and Y directions such that the aspect ratio was conserved. The original image size was restored using the same interpolation algorithm.

A modified image fidelity measure
How do we quantitatively measure the loss of information due to degradation resulting from compression and/or interpolation? Let's begin with the energy of a discrete signal f[n], which is given by: By Parseval's theorem, the Fourier transform can be used to measure the energy of a signal. The energy spectral density of the signal f is defined as: The energy spectral density is thus the energy per unit frequency of the signal at the given frequency k. Suppose f is a reference image and g is a distorted image. The ratio, where N-p represents the number of high frequency coefficients of the signal that were considered in the computation), measures the amount of degradations in the distorted image as compared to the given reference image. This ratio is always non negative. S gg and S ff are the energy spectral densities of the distorted image and the reference image, respectively. The modified measure of image quality Qual ind is computed using the elements of SSIM index and energy spectral density of high frequency coefficients of the signal. It is defined as follows: We will consider only the high frequency content of the signal. Based on the idea presented in the literature (Cheeseman, Kowalik-Urbaniak & Vrscay, 2016), the large DC coefficient is ignored.
This measure is an attempt to capture the loss of diagnostic information of compressed and interpolated medical images by employing deterministic and statistical information of the signal. The statistical properties are acquired from the frequency domain (high-frequency content) of the signal and are combined with elements of SSIM.

Observations and results
It is expected that with higher compression ratio, the visual quality of images that were compressed and then scaled gets worse in comparison to their original counterparts. It also seems natural to expect that applying two lossy operations to an image would result in a worse image quality than when one of the lossy operations is applied. However, a counterintuitive behaviour of the quality scores has been observed. Surprisingly, according to MSE and SSIM, at some compression ratios, where two irreversible operations (JPEG compression and https://doi.org/10.37705/TechTrans/e2020031 scaling) are applied, the quality of the resulting image is slightly better than in the case where a single lossy operation was applied (JPEG compression). Table  2 shows the scores for MSE and SSIM using JPEG and JPEG2000 compressions. Figure 3 illustrates the trend of the SSIM scores as the JPEG compression ratio increases. The two curves in the left plot cross, revealing that at some compression ratio, the quality of the image according to SSIM gets better if two lossy operations are applied, namely compression and scaling, than in the case when the images had only been compressed. The proposed measure Qual ind follows the intuition correctly as indicated in Table 2. The right plot of Figure 3 shows the correct and expected behaviourof the proposed Qual ind measure.
How do we explain such inconsistency in measuring image quality? The quality scores seem correct if we only consider the visual aspects of the image since the image might have gained visual quality due to the smoothing effect or the reduction of blockiness in JPEG caused by interpolation. However, the image cannot have more details than its compressed (and not interpolated) counterpart due to the loss of information caused by interpolation. This oddity is not observed for JPEG2000 compressed images since this wavelet-based compression technique already causes a smoothing effect. This effect is exemplified after interpolation is applied causing degradations in visual quality, which are correctly captured by MSE and SSIM.
The Qual ind was designed to capture the amount of information that is retained after lossy operations are applied to a medical image. The component of Qual ind that measures the amount of degradations in the distorted image, namely the ratio S s S s gg ff > @ > @ , is observed to correctly capture the information loss that Fig. 3. Illustration of observed trends of the behaviour of SSIM and Qual ind as the compression ratio increases for CT body and brain images. The curve labelled 'Compressed' represents images that were compressed using JPEG, whereas the curve labelled "Compressed-interpolated" corresponds to the images that were compressed using JPEG and then scaled. The plots represent the observed trends and are not based on real data Table 2. SSIM, MSE and Qual ind scores for JPEG body CT images for a given compression ratio.
The images were scaled using the cubic interpolation technique https://doi.org/10.37705/TechTrans/e2020031 has occurred. Using Qual ind we observe that the quality of images is properly captured (Table 2), i.e. interpolation lowers the quality of compressed images as the amount of compression increases.

Conclusions and further work
This research proposes a full-reference objective measure of quality for interpolated images, which considers deterministic and statistical knowledge about the image. The statistical properties are acquired from the frequency domain (high-frequency content) of the signal and are combined with elements of SSIM.
This work is an attempt to capture the loss of diagnostic information in compressed and interpolated medical images. The inspiring observation that triggered further investigation was the unexpected quality scores obtained using SSIM and MSE in the case where two lossy operations were applied to an image. For some compression ratios, SSIM and MSE indicated better quality for images that had been compressed and interpolated as compared to images to which only lossy compression had been applied. Although the quality scores for JPEG show relatively small differences when the two cases of degradations are compared (Table 2), it is important to note the tendency of such behaviour. The proposed measure Qual ind shows to correctly correspond to the loss of information that occurs when compression followed by interpolation have been applied to an image. The proposed measure could serve as a starting point in understanding how the quality of medical images is affected when more than one lossy operation is performed. Further work would involve validation of the proposed image fidelity measure based on subjective radiological assessments using a modified receiver operating characteristic (ROC) analysis (Kowalik-Urbaniak, 2014;, 2015. The aim is to construct a model that could serve as a predictor of the quality of medical images after more than one lossy operation is applied for a given image modality and anatomical region.