<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>In this paper, we propose a system for natural and intuitive interaction with the robot. Its purpose is to allow a person with no specialized knowledge or training in robot programming to program a robotic arm. We utilize data from the RGB-D camera to segment the scene and detect objects. We also estimate the configuration of the operator’s hand and the position of the visual marker to determine the intentions of the operator and the actions of the robot. To this end, we utilize trained neural networks and operations on the input point clouds. Also, voice commands are used to define or trigger the execution of the motion. Finally, we performed a set of experiments to show the properties of the proposed system.</p>
</abstract>

In this paper, we propose a system for natural and intuitive interaction with the robot. Its purpose is to allow a person with no specialized knowledge or training in robot programming to program a robotic arm. We utilize data from the RGB-D camera to segment the scene and detect objects. We also estimate the configuration of the operator’s hand and the position of the visual marker to determine the intentions of the operator and the actions of the robot. To this end, we utilize trained neural networks and operations on the input point clouds. Also, voice commands are used to define or trigger the execution of the motion. Finally, we performed a set of experiments to show the properties of the proposed system.

Multimodal Robot Programming Interface Based on RGB-D Perception and Neural Scene Understanding Modules

Journal of Automation, Mobile Robotics and Intelligent Systems

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

In this paper, we propose a system for natural and intuitive interaction with the robot. Its purpose is to allow a person with no specialized knowledge or...

Multimodal Robot Programming Interface Based on RGB-D Perception and Neural Scene Understanding Modules

Publicado en línea: 04 mar 2024

Páginas: 29 - 37

Recibido: 14 ene 2023

Aceptado: 24 may 2023

DOI: https://doi.org/10.14313/jamris/3-2023/20

Palabras clave
Human-robot interface, Robot programming, 3D perception

© 2023 Bartłomiej Kulecki, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Multimodal Robot Programming Interface Based on RGB-D Perception and Neural Scene Understanding Modules

Publicado en línea: 04 mar 2024

Páginas: 29 - 37

Recibido: 14 ene 2023

Aceptado: 24 may 2023

DOI: https://doi.org/10.14313/jamris/3-2023/20

Palabras claveHuman-robot interface, Robot programming, 3D perception

© 2023 Bartłomiej Kulecki, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Palabras clave
Human-robot interface, Robot programming, 3D perception