Several factors impact on colour images and they do not only affect visual perception of the image. They also hinder the identification and distinction of image features that are relevant for different applications such as segmentation or pattern recognition. Noise is one of the most common of these factors and it can significantly affect visual quality of images, as well as the performance of most image processing tasks. It is the result of errors in the image acquisition process.
In several cases, images are taken under not suitable conditions: low light, too much clarity or poor weather conditions. A deficient quality equipment can hamper image acquisition because of transmissions errors, problems with networked cables, signal disturbances, troubles with sensors, etc. Therefore, pixel intensity values do not reflect true colors of the real scene we are shooting. For these reasons, lots of methods have been developed in order to recover lost image information and to enhance image details. Color image
Analogously,
The initial approach is usually to consider it as a two-steps process: first smoothing and later sharpening, or the other way around. However, this approach usually leads to many problems. On the one hand, if we first apply a smoothing technique, then we could be losing information that cannot be recovered in the succeeding sharpening step. On the other hand, if we first apply a sharpening method over a noisy image, we will amplify the noise present in it. The ideal way to address this problem is to consider a method that was able to sharp image details and edges while removing noise. Nevertheless, this is not a simple task given the opposite nature of these two operations.
Many methods for both sharpening and smoothing have been proposed in the literature, but if we restrict ourselves to methods that consider both of them simultaneously, the state-of-the-art is not so extensive. In this work we will also survey several methods of two-steps approaches in order to intensify the features of an image and to reduce the existing noise of the image. We will also review techniques that address both goals simultaneously.
In this way, the paper is organized as follows: Section 2 presents a brief review about image smoothing. In Section 3 we revisit some well-known techniques within enhancement and sharpening field. In Section 4.1 we introduce two-steps methods for smoothing and later sharpening and, alternatively, for sharpening and later smoothing. A comparison of both approaches will be shown. This will motivate the need of techniques that simultaneously address both processes, that will be exposed in Section 4.2. Finally, in Section 5 we compare the results given by the aforementioned methods.
Image smoothing techniques have the goal of preserving image quality. In other words, to remove noise without losing the principal features of the image. However, there are several types of noise. The main three types are: impulsive, additive, and multiplicative.
There are different sources of noise and plenty of denoising methods for each kind of noise. The most common one is probably the so-called
Another interesting case is
The elimination of this type of noise is known as
First approaches for Gaussian noise smoothing were based on linear strategies. These methods, such as the
Within nonlinear methods, a wide class of them uses averaging to take advantage of the zero-mean property of the Gaussian noise. This class includes the well-known
Another non-linear method respectful with image structure is the
A well-known nonlinear filter is the
To overcome the shortcomings of this kind of filters, linear and non-linear methods are combined in order to exploit the benefits of each of them for denoising colour images respecting details. In [15] graph theory is used to propose
The filters introduced in [17] give detection rules based on differences between the
The
Guo et al [8] presented an adaptive PM filter able to segment the noisy image into two different regions, inner ones and borders. Then diffusion is applied by adapting it depending on the region we are considering.
In [2], Dabov et al introduced collaborative filtering strategies which are probably the ones that provide the most impressive results within the block matching based denoising. The method presented there is called
Methods based on
Wavelet representation has become very popular within smoothing of images field [29]. It consists on decomposing an image signal into multiple scales, which represent its different frequency components. There are plenty of wavelets families, such as the ones of Haar, Daubechies, Coiflet, Symlet, Meyer, Morlet or the Mexican Hat, among others. In these methods, smoothing is applied in the image by using a threshold for removing detail coefficients. In this way, a hard scale-dependent threshold is proposed in [38]. Statistical modeling can be performed instead of thresholding to operate over wavelet coefficients to suppress noise [31, 43]. Wavelet transformation also works for data regularization as it is proposed in [9].
In Figures 1 and 2 we can see the performance of some of the smoothing filters reported in this section. They have been applied to classical
Image enhancement process consists of a collection of techniques whose purpose is to improve image visual appearance and to highlight or recover certain details of the image for conducting an appropriate analysis by a human or a machine.
During the acquisition process, several factors can influence on the quality of the image such as illumination conditions, ambient pressure or temperature fluctuations. In order to enhance the image, we try to convert it for getting details that are obscured, or to sharpen certain features of interest. There is a large number of applications of these techniques that include medical image analysis, remote sensing, high definition television, microscopic imaging, etc. The existence of such a variety implies that there will also be very different goals within image enhancement, according to each particular application. In some cases, the purpose is to enhance the contrast, in others, to emphasize details and/or borders of the image. We will refer to this last process as
In this section we present a brief overview about the principal sharpening techniques. They can be classified into two different groups depending on the image domain:
Spatial domain techniques for sharpening an image are based on manipulations of pixel values. One of the ways to improve it is by augmenting the contrast among different parts of the image.
There are several methods for image sharpening in the spatial domain. One of the most well-known is
The application of this technique in colour images is not a simple task. Histogram equalization is a non-linear process and involves intensity values of the image and not the colour components. For these reasons, channel splitting and equalizing each channel separately is not the proper way for equalization of contrast. So, the first step is to convert the colour space of the image from RGB into other colour space which separates intensity values from colour components such as HSV, YCbCr or Lab, and apply to the equalization over the H, Y or L channel respectively. In Figure 5 we can see the result of apply HE over the R, G and B channel separately and over the channel L in the Lab space. There are other approaches that generalize histogram equalization to colour spaces. Among the most well-known is 3D histogram [60].
There are lots of works seeking to improve HE techniques such as
With the
The aforementioned methods do not use spatial information neighbours of a given pixel. They are confined to use the intensity values of all pixels of the image. Local histogram equalization based methods were introduced in order to adapt these techniques by using local information. In this way, the
We can see in Figure 6 the result of applying BPDFHE and CLAHE methods to Lenna image. As it is indicated above, with this last method we improve the performance through a local approach that allows us to extract more information of the image structure.
Another well-known technique within spatial domain sharpening is the
In the
Frequency domain techniques are based on the use of transformations like the Discrete Fourier (or Cosine) Transform or Wavelet Transforms. We remind that each one of these methods is not unique and, in fact, they compile a family of methods that are in essence the same, but each one with slight differences respect to the others. They work as follows: First, we apply one of these transformation methods, after we process the transform under one of these methods and, finally, the inverse transformation of the processed image gives us the result.
This approach has a wide advantage, the facility of distinguish different regions in an image. Higher frequencies are related to edges or details and lower correspond to smooth areas of the image. This easy separation allows to process the image appropriately depending on the goal. However, this also comprises that we are processing details of different regions at the same time in a indistinguishable way. This also happens with smooth regions.
Wavelet theory has become a potent image processing tool in the last years, this technique provide us image spatial and frequency information. An enhancement of the image can be obtained by adding high-pass or substracting low-pass filtered versions from the image [29, 30]. One of the early works on contrast sharpening in the wavelet domain is reported in [26], where a parametrised hyperbolic function is applied to the gradient of the wavelet coefficients. Since then, lots of works have been developed in the wavelet domain. For instance, Loza et al. proposed a non-linear enhancement method based on the local dispersion of the wavelet coefficients [25]. This algorithm enhances the contrast in images adaptively, based on local statistics of the wavelet coefficients of the image.
A contrast enhancement technique using a scaling of the internal noise of a dark image in the
In Figure 7 we can see the output of the UM and CLAHE methods for Parrot image. We also can see enlarged images of detail regions of them, where we can appreciate the sharpening effect over edges. This is an example of sharpening technique as opposite to the examples showed in Figure 6, which were methods more tied to contrast enhancement. They can be compared in Figure 7, where we can see an example of contrast enhancement, using CLAHE, versus sharpening using UM.
In this section we discuss about techniques that jointly considered smoothing and sharpening. The first idea we come up is to process the image in two different steps: first, by implementing one operation and then, over the processed image, carrying on the second process. Here, the order in which we carry the operations can greatly change the output. If we sharpen before smoothing, we can increase the relevance of image noise, which will complicate the smoothing task. If, by contrast, we smooth before sharpening, we may loss information in the smoothing process that the sharpen method could not recover. In general, the second approach usually provides better outcomes, however, it is still not an optimal solution. For that reason, techniques that were able to combine simultaneously both smoothing and sharpness have been suggested in the last few years.
Two-step methods for smoothing and sharpening consist on the sequential application of two methods, one of each type. In Figure 8 we can compare two-step methods based on BF for smoothing and CLAHE for sharpening. In the first case, we start with BF, and in the second one with CLAHE. This last method is applied to blurred Lenna and Parrot images in Figure 9.
We have seen the result of smoothing an image and subsequently apply a sharpening technique over the denoised image. In the first step, we lost a lot of information about the image, and then the second step was not good enough to recover the lost information. To overcome this drawback, we can first apply a sharpening, and in a second step we smooth the image. Results of both approaches can be seen in Figures 8 and 9.
Another example of a unified two-step method for both smoothing and sharpening over low light colour images is proposed in [21]. There two different steps are applied too. BM3D filter is combined with a structural filter for smoothing. Afterwards, a luminance adaptive contrast is applied in order to sharp the details of the smoothed image.
Although smoothing and sharpening are apparently opposite operations, the necessity of using both techniques at the same time is ever increasing. Both of them have been extensively studied and the techniques developed for each process are very different. However, this does not happen if we talk about doing both operations at the same time. The state of the art in terms of methods that are able to sharp details while removing noise is still relatively reduced. In this section we present some of these techniques.
Two smooth and sharpening techniques, such as PM and CLAHE, have been combined simultaneously by means of a synchronization algorithm [4], where we can appreciate the improvement respect the corresponding two-step methods based on them. The method draw on the advantage of these original models and combine it for constructing a good tool for medical images, more concretely for magnetic resonance.
As we saw in Section 4.1, PM is based in a non-linear forward diffusion process geared by a diffusion variable that permits to control the smoothing effects over the image. In this way, it is tempting to use backward diffusion in order to obtain a sharpened image. However, backward diffusion is unstable and an ill-posed problem. Nevertheless, Gilboa et al. show that it is possible to combine forward and backward nonlinear diffusion processes for getting the
Nevertheless, in the same way that te backward diffusion process, the FAB diffusion is unstable and ill-posed. In order to overcome this drawback Vadim and Yehoshua proposed the use of
In [3], the authors proposed to combine BM3D with a transform-domain sharpening technique, applied to blocks, in order to sharpen while noise is being removed. We will refer to this method as (BM3DSharp).
We can also find fuzzy based methods with this double purpose. Russo proposed, in [51, 52], a fuzzy neural network technique that consists on a multiple-output processing system that adopts fuzzy networks in order to combine sharpening and smoothing. In particular, three fuzzy networks are combined; the first and third one smooth the image and the second one is responsible of the sharpening. The aforementioned methods can be compared in Figures 10, 11, and 12.
As we have mentioned UM has the disadvantages of increasing the noise in homogeneous regions and of not being able to sharpen all details due to its use of a fixed sharpening strength. With the purpose of overcome this drawback and to remove the noise at the same time that edges are sharpened, Kim et al. have developed an adaptive unsharp mask, called
In [66], an
To overcome this problem, an
In a few words, the guided filter is a linear translation-variant filter in which each pixel is replaced by a linear transform of a guidance image (input image or another one). Saini et al. proposed a modification of the ABF that firstly considers a segmentation of the image in clusters with similar structure [58]. This clustering is based on features that describe the local structure of the image. After a segmentation, each pixel is processed with a weighted mean that uses bilateral weights of the corresponding cluster.
Wavelet based methods have also been proposed for dealing with smoothing and sharpening simultaneously. In [24] the image on the HSV space is transformed into the wavelet domain by
In [10], the authors apply smoothing and sharpening process on images in the
A combined method based on the graph Laplacian operator is performed in [23] where the output image is the solution of a minimization problem of a function with two different terms: one is a standard sparse coding formulation for image smoothing and the other one allows to sharpen the image thanks to the Laplacian operator.
In this paper, the main techniques for removing white Gaussian noise in colour images have been revisited. Also, we have reviewed the typical techniques for colour images smoothing and sharpening, both in spatial and in frequency domain.
Both operations have an opposite nature, the aim of smoothing an image is to remove the noise. However, the aim of sharpening is somehow the opposite, since it tries to emphasize details. These techniques are responsible for making more visible variations and details or edges of the images. We have seen that the application of both techniques in two steps, one after the other, produce wrong results because of loosing some relevant information or sharpening the noise.
The reduced number of approaches that simultaneously respond to both goals lies on the difficulty of combining these apparently contradictory process. We have reported the most remarkable of these methods.