End-to-End Deep Learning for Robotic Following

摘要

Robotic following has many practical applications and involves several cognitive sub-tasks including object identification, localization, tracking, path planning, and motion control. We propose a system that uses a single deep neural network (DNN), with video camera pixels as the only input, to handle all the cognitive perception and visuomotor control functions needed to perform robotic following behaviors. Our approach combines 1) end-to-end deep learning for inferring motion control outputs from visual inputs, 2) multi-task learning for simultaneously producing multiple outputs with the same DNN, and 3) spatio-temporal deep learning for perceiving motion across multiple video frames. We use simulations of truck platooning to quantitatively show that spatio-temporal DNNs increase driving accuracy and improve machine perception of scene kinematics. Experiments conducted with mobile robots in a laboratory test track show real-time embedded systems performance and indicate that an end-to-end deep learning robot is able to follow its leader for long periods of time, while keeping within lanes and avoiding obstacles.