Using affordances for assembly: Towards a complete Craft Assembly System

摘要

When crafting a homemade object, such as in DIY (do-it-yourself) projects, a human is able to, from a goal object in mind, assemble a craft with the available objects in the scene without having a set of instructions. Taking inspiration from this, we propose a robotic system capable of performing such task, that we define as a Craft Assembly Task. In this paper, we show the preliminary version of our proposed system, focusing on the first step, where it needs to choose, from the available objects, which ones should be used as the components of a given assembly. The possible candidates are evaluated based on the visual likeness, using shape matching and dimensions comparison as the main criteria, and on functionality, using affordance matching. The desired final assembly is given as an input to the system in a 3D CAD model, from which the system extracts the shape, dimension and affordance labels from each component, then using a framework of neural networks, it detects the available objects in the scene and evaluate their affordances. After finding candidates with the corresponding affordances, their point clouds are used to evaluate their shapes and dimensions by using a RANSAC algorithm.