Impact of Low Resolution on Image Recognition with Deep Neural Networks: An Experimental Study

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y. and Zheng, X. (2016). TensorFlow: Large-scale machine learning on heterogeneous distributed systems, arXiv: 1603.04467.Search in Google Scholar

Bevilacqua, M., Roumy, A., Guillemot, C. and Alberi-Morel, M.L. (2012). Low-complexity single-image super-resolution based on nonnegative neighbor embedding, British Machine Vision Conference (BMVC), Guildford, UK.10.5244/C.26.135Search in Google Scholar

Chang, H., Yeung, D.-Y. and Xiong, Y. (2004). Super-resolution through neighbor embedding, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, Washington, DC, USA, Vol. 1, pp. I-I.Search in Google Scholar

da Costa, G.B.P., Contato, W.A., Nazare, T.S., Neto, J.E. and Ponti, M. (2016). An empirical study on the effects of different types of noise in image classification tasks, arXiv: 1609.02781.Search in Google Scholar

Dodge, S. and Karam, L. (2016). Understanding how image quality affects deep neural networks, 8th International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal, pp. 1-6.10.1109/QoMEX.2016.7498955Search in Google Scholar

Dong, C., Loy, C.C., He, K. and Tang, X. (2014). Learning a deep convolutional network for image super-resolution, European Conference on Computer Vision, Zurich, Switzerland, pp. 184-199.10.1007/978-3-319-10593-2_13Search in Google Scholar

Dong, C., Loy, C.C. and Tang, X. (2016). Accelerating the super-resolution convolutional neural network, European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 391-407.10.1007/978-3-319-46475-6_25Search in Google Scholar

Dutta, A., Veldhuis, R.N. and Spreeuwers, L.J. (2012). The impact of image quality on the performance of face recognition, 33rd WIC Symposium on Information Theory in the Benelux, Boekelo, The Netherlands, pp. 141-148.Search in Google Scholar

Freeman, W.T., Jones, T.R. and Pasztor, E.C. (2002). Example-based super-resolution, IEEE Computer Graphics and Applications 22(2): 56-65.10.1109/38.988747Search in Google Scholar

Goodfellow, I., Bengio, Y. and Courville, A. (2016). Deep Learning, MIT Press, Cambridge, MA.Search in Google Scholar

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y. (2014). Generative adversarial nets, in Z. Ghahramani et al. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc., Red Hook, NY, pp. 2672-2680.Search in Google Scholar

He, K., Zhang, X., Ren, S. and Sun, J. (2016). Deep residual learning for image recognition, Proceedings of the IEEE10.1109/CVPR.2016.90Search in Google Scholar

Huang, J.-B., Singh, A. and Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 5197-5206.10.1109/CVPR.2015.7299156Search in Google Scholar

Johnson, J., Karpathy, A. and Fei-Fei, L. (2016). DenseCap: Fully convolutional localization networks for dense captioning, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, VN, USA, pp. 4565-4574.10.1109/CVPR.2016.494Search in Google Scholar

Karahan, S., Yildirum, M.K., Kirtac, K., Rende, F.S., Butun, G. and Ekenel, H.K. (2016). How image degradations affect deep CNN-based face recognition, International Conference of the Biometrics Special Interest Group (BIOSIG), Darmstadt, Germany, pp. 1-5.10.1109/BIOSIG.2016.7736924Search in Google Scholar

Kim, J., Kwon Lee, J. and Mu Lee, K. (2016). Accurate image super-resolution using very deep convolutional networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 1646-1654.10.1109/CVPR.2016.182Search in Google Scholar

Kingma, D. and Ba, J. (2014). Adam: A method for stochastic optimization, arXiv: 1412.6980.Search in Google Scholar

Koziarski, M. and Cyganek, B. (2017). Image recognition with deep neural networks in presence of noise-dealing with and taking advantage of distortions, Integrated Computer- Aided Engineering 24(4): 337-349.10.3233/ICA-170551Search in Google Scholar

Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks, Neural Information Processing Systems, Lake Tahoe, CA, USA, pp. 1097-1105.Search in Google Scholar

Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. and Shi, W. (2016). Photo-realistic single image super-resolution using a generative adversarial network, arXiv: 1609.04802.10.1109/CVPR.2017.19Search in Google Scholar

Lim, B., Son, S., Kim, H., Nah, S. and Lee, K.M. (2017). Enhanced deep residual networks for single image super-resolution, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop, Honolulu, HI, USA, Vol. 1, p. 3.10.1109/CVPRW.2017.151Search in Google Scholar

Long, J., Shelhamer, E. and Darrell, T. (2015). Fully convolutional networks for semantic segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 3431-3440.10.1109/CVPR.2015.7298965Search in Google Scholar

Mao, X., Shen, C. and Yang, Y.-B. (2016). Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections, Neural Information Processing Systems, Barcelona, Spain, pp. 2802-2810.Search in Google Scholar

Martin, D., Fowlkes, C., Tal, D. and Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, 8th IEEE International Conference on Computer Vision, ICCV 2001, Vancouver, Canada, Vol. 2, pp. 416-423.Search in Google Scholar

Peng, X., Hoffman, J., Stella, X.Y. and Saenko, K. (2016). Fine-to-coarse knowledge transfer for low-res image classification, IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, pp. 3683-3687.10.1109/ICIP.2016.7533047Search in Google Scholar

Ronneberger, O., Fischer, P. and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, pp. 234-241.10.1007/978-3-319-24574-4_28Search in Google Scholar

Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C. and Fei-Fei, L. (2015). ImageNet large scale visual recognition challenge, International Journal of Computer Vision 115(3): 211-252.10.1007/s11263-015-0816-ySearch in Google Scholar

Sanchez, A., Moreno, A.B., Velez, D. and V´elez, J.F. (2016). Analyzing the influence of contrast in large-scale recognition of natural images, Integrated Computer-Aided Engineering 23(3): 221-235.10.3233/ICA-160516Search in Google Scholar

Schmidhuber, J. (2015). Deep learning in neural networks: An overview, Neural Networks 61(1): 85-117.10.1016/j.neunet.2014.09.00325462637Search in Google Scholar

Schulter, S., Leistner, C. and Bischof, H. (2015). Fast and accurate image upscaling with super-resolution forests, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 3791-3799.10.1109/CVPR.2015.7299003Search in Google Scholar

Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D. and Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 1874-1883.10.1109/CVPR.2016.207Search in Google Scholar

Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition, arXiv: 1409.1556.Search in Google Scholar

Sun, Y., Wang, X. and Tang, X. (2013). Deep convolutional network cascade for facial point detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, pp. 3476-3483.10.1109/CVPR.2013.446Search in Google Scholar

Tai, Y., Yang, J. and Liu, X. (2017). Image super-resolution via deep recursive residual network, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp. 2790-2798.10.1109/CVPR.2017.298Search in Google Scholar

Timofte, R., De Smet, V. and Van Gool, L. (2014). A+: Adjusted anchored neighborhood regression for fast super-resolution, Asian Conference on Computer Vision, Singapore, Singapore, pp. 111-126.10.1007/978-3-319-16817-3_8Search in Google Scholar

Vasiljevic, I., Chakrabarti, A. and Shakhnarovich, G. (2016). Examining the impact of blur on recognition by convolutional networks, arXiv: 1611.05760.Search in Google Scholar

Yang, J., Wright, J., Huang, T.S. and Ma, Y. (2010). Image super-resolution via sparse representation, IEEE Transactions on Image Processing 19(11): 2861-2873.10.1109/TIP.2010.205062520483687Search in Google Scholar

Zeyde, R., Elad, M. and Protter, M. (2010). On single image scale-up using sparse-representations, International Conference on Curves and Surfaces, Paris, France, pp. 711-730.10.1007/978-3-642-27413-8_47Search in Google Scholar

Zou, W.W. and Yuen, P.C. (2012). Very low resolution face recognition problem, IEEE Transactions on Image Processing 21(1): 327-340.10.1109/TIP.2011.216242321775262Search in Google Scholar