Open Access

Performance Comparison of Statistical vs. Neural-Based Translation System on Low-Resource Languages


Cite

Figure 1:

Architecture of SMT system. SMT, statistical machine translation.
Architecture of SMT system. SMT, statistical machine translation.

Figure 2:

Encoder–decoder model with attention [3].
Encoder–decoder model with attention [3].

Figure 3:

Transformer model [21].
Transformer model [21].

Figure 4:

GAN model in the case of NMT use. GAN, generative adversarial network.
GAN model in the case of NMT use. GAN, generative adversarial network.

Figure 5:

Heat map representation of attention visualization.
Heat map representation of attention visualization.

Figure 6:

Masked values are represented with zeros.
Masked values are represented with zeros.

Figure 7:

Graphical representation of masking operation; the colored right half is the masked part.
Graphical representation of masking operation; the colored right half is the masked part.

Figure 8:

BLEU score generated by NMT and SMT for Eng.–Beng. language pairs. MT, machine translation; SGD, stochastic gradient descent; SMT, statistical machine translation.
BLEU score generated by NMT and SMT for Eng.–Beng. language pairs. MT, machine translation; SGD, stochastic gradient descent; SMT, statistical machine translation.

Figure 9:

BLEU score generated by NMT and SMT for Eng.–Hindi language pairs. MT, machine translation; SGD, stochastic gradient descent; SMT, statistical machine translation.
BLEU score generated by NMT and SMT for Eng.–Hindi language pairs. MT, machine translation; SGD, stochastic gradient descent; SMT, statistical machine translation.

Figure 10:

BLEU with minimum n-gram having maximum score. SGD, stochastic gradient descent.
BLEU with minimum n-gram having maximum score. SGD, stochastic gradient descent.

Attention-based NMT outperforms SMT for the Bengali–Hindi language pair (Das et al. [32])

Translation model BLEU score Iterations
Attention-based NMT model 20.41 25
MOSES (SMT) 14.35 -

NMT outperformed SMT with transfer learning, ensemble, and further processing of data (Zopth et al.)

Language SBMT NMT Transfer Final
Hausa 23.7 16.8 21.3 24.0
Turkish 20.4 11.4 17.0 18.7
Uzbek 17.9 10.7 14.4 16.8
Urdu 17.9 5.2 13.8 14.5

NMT system with transformer model and BPE outperformed phrase-based SMT for English–Hindi and Hindi–English language pairs (Haque et al. [33])

MT model BLEU METEOR TER
Eng.Hindi-PBSMT 28.8 30.2 53.4
Eng.Hindi-NMT 36.6 33.5 46.3
Hindi–Eng.PBSMT 34.1 36.6 50.0
Hindi–Eng.NMT 39.9 38.5 42.0

English–Hindi translation using different optimizers

Language pair Optimizer BLEU-4 score NMT model No. of epochs
Eng.–Hindi Adam 12.25 NMT with attention 14
Eng.–Hindi SGD 11.50 NMT with attention 14
Eng.–Hindi 16.64 MOSES

English–Bengali translation BLEU scores using different optimizers

Language pairs Optimizer BLEU-4 score MT model No. of epochs
Eng.–Beng. Adam 10.78 NMT with attention 12
Eng.–Beng. SGD 11.17 NMT with attention 12
Eng.–Beng. 14.58 MOSES

BLEU-1, 2, and 3 scores are summarized for Eng.–Beng. and Eng.–Hindi language pairs using Adam- and SGD-Optimizers

BLEU Eng.–Beng.-NMT (Adam-Optimizer) Eng.–Beng.-NMT (SGD-Optimizer) Eng.–Hindi (NMT-Adam) Eng.–Hindi (NMT-SGD)
BLEU-1 14.15 13.91 15.77 14.18
BLEU-2 12.65 13.11 14.12 13.33
BLEU-3 11.83 12.17 13.95 12.19

For various low-resource corpus SMT outperformed NMT (Ahmadnia et al. [17])

Corpus SMT NMT NMT* NMT**
Gnome 20.54 15.49 17.26 18.76
KDE4 15.64 13.36 14.29 15.71
Subtitles 18.82 18.62 19.51 22.54
Ubuntu 16.76 14.27 15.14 15.87
Tanzil 17.69 15.14 16.53 17.72
Overall 17.06 15.25 16.67 17.32
eISSN:
1178-5608
Language:
English
Publication timeframe:
Volume Open
Journal Subjects:
Engineering, Introductions and Overviews, other