<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Today, technology allows us to produce extensive multimodal systems which are totally under human control. These systems are equipped with multimodal interfaces, which enable more natural and more efficient interaction between man and machine. End users can take advantage of natural modalities (e.g. audio, eye gaze, speech, gestures, etc.) to communicate or exchange information with applications. In this work, we assume that a number of these modalities are available to the user. In this paper, we present a prototype of a multimodal architecture, and show how modality selection and fission algorithms are implemented in such a system. We use a pattern technique to divide a complex command into elementary subtasks and select suitable modalities for each of them. We integrate a context-based method using a Bayesian network to resolve ambiguous or uncertain situations.</p></abstract>

Today, technology allows us to produce extensive multimodal systems which are totally under human control. These systems are equipped with multimodal interfaces, which enable more natural and more efficient interaction between man and machine. End users can take advantage of natural modalities (e.g. audio, eye gaze, speech, gestures, etc.) to communicate or exchange information with applications. In this work, we assume that a number of these modalities are available to the user. In this paper, we present a prototype of a multimodal architecture, and show how modality selection and fission algorithms are implemented in such a system. We use a pattern technique to divide a complex command into elementary subtasks and select suitable modalities for each of them. We integrate a context-based method using a Bayesian network to resolve ambiguous or uncertain situations.

Prototyping using a Pattern Technique and a Context-Based Bayesian Network in Multimodal Systems

Université du Québec, École de technologie supérieure

Université de Versailles-Saint-Quentin-en-Yvelines

College of computers and information technology Taif University

International Journal on Smart Sensing and Intelligent Systems

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Today, technology allows us to produce extensive multimodal systems which are totally under human control. These systems are equipped with multimodal...

Prototyping using a Pattern Technique and a Context-Based Bayesian Network in Multimodal Systems

Publié en ligne: 01 sept. 2015

Pages: 1667 - 1686

Reçu: 03 mai 2015

Accepté: 21 juil. 2015

DOI: https://doi.org/10.21307/ijssis-2017-824

© 2015 Atef Zaguia et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.