Visual servoing for a user's mouth with effective intention reading in a wheelchair-based robotic arm

Abstract

Intention reading is one of the essential functions of human-friendly rehabilitation robots. First, the overall structure of a new wheelchair-based robotic arm system, KARES II, and its human-robot interaction technologies are presented. Among those technologies, we concentrate on visual servoing that allows this robotic arm to operate autonomously via visual feedback. Effective intention reading, such as recognizing the positive and negative meaning of the user, is performed on the basis of changes of the facial expression around lips that is strongly related to the user's intention while serving the user. For the efficient visual information processing, log-polar mapped images are used to control the stereo camera head that is located in the end-effector of the robotic arm. The visual servoing with effective intention reading was successfully applied to serve a beverage for the user.