<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Based on ELAN multimodal discourse analysis software, this paper constructs a multimodal Russian translation model based on the machine translation model with visual grammar and multimodal discourse analysis as the theoretical basis. To address the issue of missing semantics caused by insufficient input information at the source of real-time translation, the model uses images as auxiliary modalities. The real-time Russian translation model is constructed using the wait-k strategy and the concept of multimodal self-attention. Experiments and analysis are carried out on the Multi30k training set, and the generalization ability and translation effect of the model are finally evaluated with the test set. The results show that by applying multimodal discourse analysis to Russian translation, the three translation evaluation indexes of BLEU, METEOR, and TER are improved by 1.3, 1.0, and 1.4 percentage points, respectively, and the phenomenon of phantom translation is effectively reduced.</p>
</abstract>

Based on ELAN multimodal discourse analysis software, this paper constructs a multimodal Russian translation model based on the machine translation model with visual grammar and multimodal discourse analysis as the theoretical basis. To address the issue of missing semantics caused by insufficient input information at the source of real-time translation, the model uses images as auxiliary modalities. The real-time Russian translation model is constructed using the wait-k strategy and the concept of multimodal self-attention. Experiments and analysis are carried out on the Multi30k training set, and the generalization ability and translation effect of the model are finally evaluated with the test set. The results show that by applying multimodal discourse analysis to Russian translation, the three translation evaluation indexes of BLEU, METEOR, and TER are improved by 1.3, 1.0, and 1.4 percentage points, respectively, and the phenomenon of phantom translation is effectively reduced.

Effective application of multimodal discourse analysis in Russian translation

Management and Law of Jilin Normal University

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

{"article-title":"Effective application of multimodal discourse analysis in Russian translation"}

Based on ELAN multimodal discourse analysis software, this paper constructs a multimodal Russian translation model based on the machine translation model with...

Effective application of multimodal discourse analysis in Russian translation

Published Online: Jun 03, 2024

Page range: -

Received: Feb 13, 2024

Accepted: Apr 14, 2024

DOI: https://doi.org/10.2478/amns-2024-1318

Keywords
Multimodal discourse, Visual grammar, Self-attention, Wait-k strategy, Multi30k training set

© 2024 Yanan Wu, et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Effective application of multimodal discourse analysis in Russian translation

Published Online: Jun 03, 2024

Page range: -

Received: Feb 13, 2024

Accepted: Apr 14, 2024

DOI: https://doi.org/10.2478/amns-2024-1318

KeywordsMultimodal discourse, Visual grammar, Self-attention, Wait-k strategy, Multi30k training set

© 2024 Yanan Wu, et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Keywords
Multimodal discourse, Visual grammar, Self-attention, Wait-k strategy, Multi30k training set