A method for constraint-based six degree-of-freedom haptic interaction with streaming point clouds

Abstract

This paper presents a constraint-based method for haptic interaction between an arbitrary voxelized polygon tool and a streaming point cloud derived from a depth sensor. Using the presented method, a user can interact with both dynamic as well as static point cloud representations of real objects captured in real-time. Every depth image frame is filtered and surface normals are calculated in real-time. For movement of the virtual tool, a `quasi-static' simulation is used. The innovations of this work include the extension of haptic rendering from streaming point clouds to six degrees of freedom. This is appropriate for co-robotic tasks where haptic feedback to the user is combined with remote control of a robot.