Image-based predictive display for high d.o.f. uncalibrated tele-manipulation using affine and intensity subspace models

Abstract

The addition of immediate but estimated visual feedback, called predictive display, improves tele-manipulaton performance when the real video feedback is delayed. Current systems typically rely upon a previously calibrated camera and manipulator. We present a method where the motor-visual calibration is estimated on-line from motor commands and returned video images only. Predicted visual feedback is presented in two forms. As soon as a basic model has been estimated, a wire frame drawing of the predicted current pose is overlaid on the delayed video feedback. After some time when a rich model has been estimated, predicted intensity images are synthesized and these replace the delayed real video. In an intermediate situation where blurry synthesized images can be computed, the wireframe is overlaid on the synthesized images to show precisely the pose of the object. Experiments with a Utah/MIT robot hand and PUMA robot arm are shown.