Deep Multi-state Object Pose Estimation for Augmented Reality Assembly

Abstract

Neural network machine learning approaches are widely used for object classification or detection problems with significant success. A similar problem with specific constraints and challenges is object state estimation, dealing with objects that consist of several removable or adjustable parts. A system that can detect the current state of such objects from camera images can be of great importance for Augmented Reality (AR) or robotic assembly and maintenance applications. In this work, we present a CNN that is able to detect and regress the pose of an object in multiple states. We then show how the output of this network can be used in an automatically generated AR scenario that provides step-by-step guidance to the user in assembling an object consisting of multiple components.