6D Object Pose Tracking with Optical Flow Network for Robotic Manipulation

Abstract

In this paper, we design a novel 6-DOF object pose tracking framework for robotic manipulation tasks. The framework takes a RGB-D video stream as input observations and outputs a 6D pose estimate corresponding to each frame for the interested object to be tracked. The novelty lies in the pose change estimation where we leverage a segmentation network and an optical flow network to estimate the pose change between previous and current frames. The final 6D pose estimate is the multiplication of the 6D pose matrix in previous frame and the pose change. Unlike most tracking networks, our pose tracking model does not require any object 3D model as auxiliary input. We take two consecutive frames as the input and estimate their optical flow map by using a pre-trained optical flow network. Our framework is a keypoint based estimation method. The estimated flow map can extract the temporal motion information that can be used to generate keypoint candidates. Then an iterative keypoint refinement scheme is used to validate the selected keypoints. Our experimental results show that our framework can outperform some existing works or achieve comparable results in three selected datasets.