Mobile manipulator control based on voice and visual signal

Abstract

Abstract: In this paper, two kinds of methods are used to control the mobile manipulator in terms of voice signal control approach andʳvisual signal control approach. In voice signal control approach, the mobile manipulator can perform some actions according to the operator’sʳspeaking voice input through a microphone. For example, the robot can be commanded to go forward or backward in specified distance,ʳturn left or right with specified angle, even can follow up “8 ” shape trajectory and catch a ball. In visual approach, the mobile robot can accomplishʳsome actions according to the operator’s gestures of two hands using Kinect sensor information. It gets the depth image of two hands inʳspatial coordinates and calculates the relative distance of them to take some actions by an onboard computer. An embedded single board ʳ computer is installed in mobile robot Pioneer2 DX for enhancing remote control and running program's ability.