Implementation of Voice Command-based Object Detection System for Wheeled Humanoid Robot Using Deep Learning

Abstract

The demand for service robots is continuously increasing. Robots that can interact with humans are required not only in factories but also in everyday life. This paper proposes a Human-Robot Interaction (HRI) system based on HUMIC, a wheeled humanoid robot with a Kinect V2 and four mecanum wheels. In the proposed system, when a person issues a voice command to an object to find, the HUMIC detects an object and moves in front of the object. For this purpose, deep learning models, i.e., wav2vec 2.0 and YOLO v3, are used for speech recognition and object detection. Furthermore, the proposed system is configured to select one object by interacting with a human when there is more than one object to find and prevents the object from disappearing from the robot’s view while moving. The experiments are conducted in a real-world environment. Because this study focuses on HRI, the experiments are conducted based on a scenario in which HUMIC detects an object in an environment without obstacles. The scenario was attempted nine times and succeeded eight times to achieve an 88% success rate.