Robot methods for human-robot spatial language interaction

Abstract

This thesis investigates perception and human-robot interaction methods for a robot designed to perform a fetch task.Natural spatial language is studied to direct a mobile robot to navigate in an indoor environment, detect objects, and use them as reference landmarks in finding a target object.The perception focus is on Kinect-based furniture recognition which allows the robot to use furniture items as landmarks in the spatial language description.A two-step process is proposed to recognize furniture objects.Furniture samples are first classified using geometric features by a linguistic model; the second step uses color and texture for further discrimination into specific furniture items by a probability graphical model (PGM); both extrinsic and intrinsic confidence values are computed.Orientation is also captured to support intrinsic reference frames of furniture such as chairs and couches.A robot behavior model is proposed to improve recognition by changing the viewing perspective when the recognition confidence is low.Eight furniture items are used in experiments to test algorithms for furniture recognition, orientation detection and robot behavior.Human-robot interaction is further investigated through the translation of a processed fetch description into robot commands that execute the fetch task and use the furniture recognition when specified in the description.The approach utilizes natural language processing methods designed to tag and chunk raw descriptions (developed elsewhere).The processed spatial description is then used for translation into robot commands.A simulation experiment is presented to evaluate the method.The results show good performance of the perception and human-robot interaction algorithms.