<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>In this paper, we propose a system for natural and intuitive interaction with the robot. Its purpose is to allow a person with no specialized knowledge or training in robot programming to program a robotic arm. We utilize data from the RGB-D camera to segment the scene and detect objects. We also estimate the configuration of the operator’s hand and the position of the visual marker to determine the intentions of the operator and the actions of the robot. To this end, we utilize trained neural networks and operations on the input point clouds. Also, voice commands are used to define or trigger the execution of the motion. Finally, we performed a set of experiments to show the properties of the proposed system.</p>
</abstract>

In this paper, we propose a system for natural and intuitive interaction with the robot. Its purpose is to allow a person with no specialized knowledge or training in robot programming to program a robotic arm. We utilize data from the RGB-D camera to segment the scene and detect objects. We also estimate the configuration of the operator’s hand and the position of the visual marker to determine the intentions of the operator and the actions of the robot. To this end, we utilize trained neural networks and operations on the input point clouds. Also, voice commands are used to define or trigger the execution of the motion. Finally, we performed a set of experiments to show the properties of the proposed system.

Multimodal Robot Programming Interface Based on RGB-D Perception and Neural Scene Understanding Modules

Journal of Automation, Mobile Robotics and Intelligent Systems

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

In this paper, we propose a system for natural and intuitive interaction with the robot. Its purpose is to allow a person with no specialized knowledge or...

Multimodal Robot Programming Interface Based on RGB-D Perception and Neural Scene Understanding Modules

Data publikacji: 04 mar 2024

Zakres stron: 29 - 37

Otrzymano: 14 sty 2023

Przyjęty: 24 maj 2023

DOI: https://doi.org/10.14313/jamris/3-2023/20

Słowa kluczowe
Human-robot interface, Robot programming, 3D perception

© 2023 Bartłomiej Kulecki, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Multimodal Robot Programming Interface Based on RGB-D Perception and Neural Scene Understanding Modules

Data publikacji: 04 mar 2024

Zakres stron: 29 - 37

Otrzymano: 14 sty 2023

Przyjęty: 24 maj 2023

DOI: https://doi.org/10.14313/jamris/3-2023/20

Słowa kluczoweHuman-robot interface, Robot programming, 3D perception

© 2023 Bartłomiej Kulecki, published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Słowa kluczowe
Human-robot interface, Robot programming, 3D perception