<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>With the development and improvement of deep learning technology, its application and practice in modal data (image, speech, and text) has been achieved tremendously. In this paper, based on the neural network modality class model in deep learning, we analyze its adaptation to the visual question and answer system, propose a visual question and answer model based on the gated attention mechanism, and construct a question, and answer prediction mechanism adapted to the recurrent neural network transfer model. To address the problem of low accuracy of the model on complex problems, the inference network module is built using visual inference so that the model can extract complex problem features to improve the inference capability of the model. By predicting answers through semantic information about text and visual elements in images, correlations across modalities, and inference, advances in natural language processing and computational vision have led to improved answer accuracy in deep learning-based visual quiz models. Multiple sets of experiments show that models with deep learning inference capabilities answer complex questions with significantly higher accuracy than other existing methods.</p>
</abstract>

With the development and improvement of deep learning technology, its application and practice in modal data (image, speech, and text) has been achieved tremendously. In this paper, based on the neural network modality class model in deep learning, we analyze its adaptation to the visual question and answer system, propose a visual question and answer model based on the gated attention mechanism, and construct a question, and answer prediction mechanism adapted to the recurrent neural network transfer model. To address the problem of low accuracy of the model on complex problems, the inference network module is built using visual inference so that the model can extract complex problem features to improve the inference capability of the model. By predicting answers through semantic information about text and visual elements in images, correlations across modalities, and inference, advances in natural language processing and computational vision have led to improved answer accuracy in deep learning-based visual quiz models. Multiple sets of experiments show that models with deep learning inference capabilities answer complex questions with significantly higher accuracy than other existing methods.

Research and implementation of visual question and answer system based on deep learning

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

With the development and improvement of deep learning technology, its application and practice in modal data (image, speech, and text) has been achieved...

Research and implementation of visual question and answer system based on deep learning

Published Online: Jun 02, 2023

Page range: -

Received: Jul 02, 2022

Accepted: Oct 19, 2022

DOI: https://doi.org/10.2478/amns.2023.1.00182

Keywords
Deep learning, Modal data, Recurrent neural networks, Visual inference networks, Question-and-answer prediction models

© 2023 Kunming Wu, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Research and implementation of visual question and answer system based on deep learning

Published Online: Jun 02, 2023

Page range: -

Received: Jul 02, 2022

Accepted: Oct 19, 2022

DOI: https://doi.org/10.2478/amns.2023.1.00182

KeywordsDeep learning, Modal data, Recurrent neural networks, Visual inference networks, Question-and-answer prediction models

© 2023 Kunming Wu, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Deep learning, Modal data, Recurrent neural networks, Visual inference networks, Question-and-answer prediction models