A low-latency 60 Hz stereo vision system for real-time visual control

Abstract

The author presents an image acquisition and processing system designed for real-time six-dimensional object tracking with the precise temporal semantics and low latency required for use within the feedback loops of robotic applications. To achieve real-time performance, the system must extract from the image only the information needed to accomplish the task. Successful task performance depends on an integrated system. rather than on a collection of components. The hardware design takes the form of two large complementary VME-compatible cards: TRIAX, capable of simultaneously acquiring and processing two high-resolution video streams, and JIFFE, capable of 20 MFLOPS for the three-dimensional calculations. Algorithms are described that enable the system to track a rectangular solid, intended to track objects held by a robot hand. The algorithms hinge on scanning only small line segments of the image, followed by techniques for reconstructing object location from sparse edge points. The system tracks a cardboard box.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>