Otwarty dostęp

Development of Blind Deblurring Based on Deep Learning

 oraz    | 21 maj 2023

Zacytuj

Figure 1

Different fuzzy types
Different fuzzy types

Figure 2

Different CNN for image processing. (a) U-net or Codec network. (b) Multiscale or cascade refinement network. (c) Extended convolution network. (d) Scale recursive network (SRN).
Different CNN for image processing. (a) U-net or Codec network. (b) Multiscale or cascade refinement network. (c) Extended convolution network. (d) Scale recursive network (SRN).

Figure 3

(a) The original remaining network building blocks. (b) Building blocks of the modified network by NAH et al.
(a) The original remaining network building blocks. (b) Building blocks of the modified network by NAH et al.

Figure 4

Visual comparison of image deblurring results of GoPro test set [13]. Patches blurred by key points are displayed in (b), while patches magnified from deblurring results are displayed in (c) – (h).
Visual comparison of image deblurring results of GoPro test set [13]. Patches blurred by key points are displayed in (b), while patches magnified from deblurring results are displayed in (c) – (h).

Blind deblurring algorithm based on deep learning

Method Applicable Scenario Mechanism Advantage Limitations
Spatial variation RNN[22] Motion blur, dynamic scene blur The deblurring process is formulated through the wireless impulse response model Weights can be learned from another network and different weights can be learned for different fuzzy systems Large regional and spatial change structures need to be involved at the same time’
SRN[20] Motion blur New multiscale cyclic network structure The number of trainable parameters is reduced and the training efficiency is improved Limited to fixed data sets and training periods
DMPHN[21] Motion blur End to end CNN hierarchical model similar to spatial pyramid matching The required filter is small and can be inferred quickly Requires large GPU memory
DPSR[32] LR blurred image A new SISR degradation model is designed The deep plug and play framework can deal with any fuzzy kernel For most real images, it does not match the degradation model
BIE-RVD[33] Motion blur Automatic coding structure of spatiotemporal video screen based on end-to-end differentiable structure High accuracy and fast network running speed The task of training is complex and difficult
DDMS[34] Motion blur A full convolution structure with filtering transformation and characteristic modulation is constructed Real time filtering completely eliminates multi-scale processing and large filters Real time filtering completely eliminates multi-scale processing and large filters
deblurGAN[27]DeblurGANV2[28] Motion blurMotion blur The generated countermeasure network based on perceptual loss [9] (perceptual loss) constraint is used for deblurring The restored image is more similar to the target image in semantics and closer to people's subjective evaluation of image quality The influence of different feature layers in the perceptual network on the perceptual loss is not considered, so that the restored image details are still smooth.
Deepdeblur[18] dynamic scene blur End to end multiscale convolution network Without estimating the fuzzy kernel, multi-scale CNN can restore clear images directly and flexibly The multi-scale stacked sub network results in large amount of parameters, large consumption of video memory and great difficulty in training
SRN-deblur[20] Blur of dynamic scene End to end multiscale cyclic network Multi-scale structure and parameter sharing alleviate the problem of large amount of parameters, and the learning ability is more stable The edge is too smooth and there are artifacts
DMPHN[21] Motion blur The deep-seated multi-facet network based on spatial pyramid matching processes fuzzy images through fine hierarchical representation. It can solve the problem of performance saturation and run faster than multi-scale method It can solve the problem of performance saturation and run faster than multi-scale method
MPRnet[35] Deblurring, rain removing and noise removing A multi-stage progressive image restoration It can output accurate spatial details and context information. The network structure is simple and the effect is good The deblurring effect under the dark light line is not good
MIMO-Net[29] Motion blur Single encoder multiple input single decoder multiple output Increase the network feeling field and make the training less difficult The spatial details are lost and the texture is not clear enough

Comparison of characteristics of mainstream data sets

Data Set Construction Method Advantages and Disadvantages
Levin etc. Algorithm simulation fuzzy kernel Easy to obtain; It is easy to obtain without considering local fuzziness;
Kupy etc. Simulated trajectory Easy to obtain; Only the motion in two-dimensional space is simulated, and the real three-dimensional space is not considered
Kohler etc. The motion track is captured by 6D camera The motion trajectories in three-dimensional space are collected; Lens distortion, depth of field variation, etc. are not considered
GOPRO etc. Take the average value for continuous shooting by high-speed camera Closer to the real fuzzy situation; The acquisition process is troublesome and the data scene is single
Lai etc. Real acquisition Completely real fuzzy pictures; There is no corresponding clear image, which is often used as a test set
eISSN:
2470-8038
Język:
Angielski
Częstotliwość wydawania:
4 razy w roku
Dziedziny czasopisma:
Computer Sciences, other