Vision system for mobile robots for tracking moving targets, based on robot motion and stereo vision information

Abstract

This paper presents a vision-tracking for mobile robots, which tracks a moving target based on robot motion and stereo vision information. The proposed system controls pan and tilt actuators attached to a stereo camera, using the data from a gyroscope, robot wheel encoders, pan and tilt actuator encoders, and the stereo camera. Using this proposed system, the stereo camera always faces the moving target. The developed system calculates the angles of the pan and tilt actuators by estimating the relative position of the target with respect to the position of the robot. The developed system estimates the target position using the robot motion information and the stereo vision information. The movement of the robot is modeled as the transformation of the frame, which consists of a rotation and a translation. The developed system calculates the rotation using 3-axis gyroscope data and the translation using robot wheel encoder data. The proposed system measures the position of the target relative to the robot, combining the encoder data of pan and tilt actuators and the disparity map of the stereo vision. The inevitable mismatch of the data, which occurs from the asynchrony of the multiple sensors, is prevented by the proposed system, which compensates for the communication latency and the computation time. The experimental results show that the developed system achieves excellent tracking performance in several motion scenarios, including combinations of straights and curves and climbing of slopes.