CNN-based Joint State Estimation During Robotic Interaction with Articulated Objects

Abstract

In this paper, we investigate the problem of state estimation of rotational articulated objects during robotic interaction. We estimate the position of a joint axis and the current rotation of an object from a pair of RGB-D images registered by the depth camera mounted on the robot. However, the camera mounted on the robot has a limited view due to occlusions of the robot's arm. Moreover, some configurations of objects are difficult to register by typical RGB-D sensors. Thus, the model-based methods fail in these cases. To deal with this problem, we propose a CNN-based architecture that gradually estimates the parameters and the state of the rotational joint. To meet real-time requirements on the real robot, we propose a fast inference on 2D images without directly operating on the 3D model of the object. The proposed method is trained and verified on the RBO dataset that contains RGB-D sequences of manipulated articulated objects.