Constructing Semantic World Models from Partial Views

摘要

of objects in the scene using a black-box object detector. From a single Kinect RGB-D image, however, objects may be occluded or erroneously classified. The bottom left depicts a rendered image, with detections superimposed in red; three objects are missing due to occlusion, and two objects have been misidentified (second and fourth from left). The semantic attributes that result in our representation are very sparse (bottom right; dot location is measured 2-D pose, color represents type), but requires aggregation and association across many partial views in order to achieve estimates such as those in figure 2. Abstract—Autonomous mobile-manipulation robots need to sense and interact with objects to accomplish high-level tasks such as preparing meals and searching for objects. Behavior in these tasks is typically guided by goals supplied to tasklevel planners, which in turn assume a representation of the world in terms of objects. In this work, we explore the use of attribute-level perception to estimate high-level representations of the world. We run a black-box object detector in each range image, getting a set of detections of objects, labeled by their types and poses. We provide a formal description of a 1-D version of the problem, then develop three different solution approaches based on tracking, clustering, and a combination of the two. We evaluate the approaches empirically on data gathered by a robot moving around a table with objects on it, using a Kinect sensor to detect the objects from multiple viewpoints. We find that each of the methods performs better than using raw data, and that different methods perform best in different operational regimes. I.