Integrated planning of robotic and computer vision based spatial reasoning tasks

Abstract

This paper describes research that is designed to integrate computer vision with complex robot planning in order to achieve a system capable of autonomously reasoning in spatially reconfigurable environments. In its basic form, the computer vision based spatial reasoning system provides the capability to view objects of known structure and kinematic design and to reason about their spatial locations and orientations relative to a robotic manipulator. For the case of the specific robotic task panel to be discussed, the spatial reasoning system has knowledge of the structure and allowable motions of each of the substructures such as the movable doors, a drawer and a circular latch. This model based knowledge is used to direct an intelligent robot path planner toward accomplishing a final goal in which one of the substructures is manipulated or toward an intermediate goal such as repositioning the visual sensor for the purpose of obtaining better viewpoints so that more accurate visual reasoning can take place.