Kinematic Structures Estimation on the RGB-D Images

Abstract

In this paper, we propose a system which detects and estimates the kinematic structures of objects in the indoor environment. We are interested in specific types of objects like doors, sliding doors, and drawers which are common in the human environment and very important taking into account the full autonomy of mobile robots. We assume that the mobile robot is equipped with an RGB-D camera. We utilize a Convolutional Neural Network-based (CNN-based) object detector to locate the articulated objects on the input image created from a pair of RGB-D images. Taking into account strong prior knowledge about the articulated object, we detect the segments on the image which belong to the articulated object. Then, the optimization-based procedure finds the 3D pose and configuration of the joint detected on the scene. We train and verify the method on the images from the Kinect sensor. The performance of the proposed method shows that we can estimate articulated objects in the indoor environment using typical sensors available on the mobile robot.