PRED18: Dataset and Further Experiments with DAVIS Event Camera in\n Predator-Prey Robot Chasing

Abstract

Machine vision systems using convolutional neural networks (CNNs) for robotic\napplications are increasingly being developed. Conventional vision CNNs are\ndriven by camera frames at constant sample rate, thus achieving a fixed latency\nand power consumption tradeoff. This paper describes further work on the first\nexperiments of a closed-loop robotic system integrating a CNN together with a\nDynamic and Active Pixel Vision Sensor (DAVIS) in a predator/prey scenario. The\nDAVIS, mounted on the predator Summit XL robot, produces frames at a fixed 15\nHz frame-rate and Dynamic Vision Sensor (DVS) histograms containing 5k ON and\nOFF events at a variable frame-rate ranging from 15-500 Hz depending on the\nrobot speeds. In contrast to conventional frame-based systems, the latency and\nprocessing cost depends on the rate of change of the image. The CNN is trained\noffline on the 1.25h labeled dataset to recognize the position and size of the\nprey robot, in the field of view of the predator. During inference, combining\nthe ten output classes of the CNN allows extracting the analog position vector\nof the prey relative to the predator with a mean 8.7% error in angular\nestimation. The system is compatible with conventional deep learning\ntechnology, but achieves a variable latency-power tradeoff that adapts\nautomatically to the dynamics. Finally, investigations on the robustness of the\nalgorithm, a human performance comparison and a deconvolution analysis are also\nexplored.\n