Bio-inspired vision mimetics toward next-generation collision-avoidance automation

Abstract

The current “deep learning + large-scale data + strong supervised labeling” technology framework of collision avoidance for ground robots and aerial drones is becoming saturated. Its development gradually faces challenges from real open-scene applications, including small data, weak annotation, and cross-scene. Inspired by the neural structure and processes underlying human cognition (eg, human visual, auditory, and tactile systems) and the knowledge learned from daily driving tasks, a high-level cognitive system is developed for integrating collision sensing and collision avoidance. This bio-inspired cognitive approach has the advantages of good robustness, high self-adaptability, and low computation consumption in practical driving scenes. Vision is the primary sense for motion perception. It provides key information for navigation, course control, gaze stabilization, and other motion-related activities.1Fu Q. Hu C. Peng J. et al.A robust collision perception visual neural network with specific selectivity to darker objects.IEEE Trans. Cybern. 2020; 50: 5074-5088Crossref PubMed Scopus (18) Google Scholar Biological visual neural networks have evolved over millions of years and are working efficiently in nature. These visual neural networks can be ideal models for designing artificial vision systems, especially for collision avoidance. As a visual neural structure, the lobula giant movement detector (LGMD) neuron in locusts shows a strong ability to detect moving objects.2Yue S. Rind F.C. Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement.IEEE Trans. Neural Network. 2006; 17: 705-716Crossref PubMed Scopus (98) Google Scholar This LGMD neuron consists of a four-layer neural network in which the first layer detects the changes in luminance, the second and third layers process the interaction between excitation and inhibition responses, and the last layer outputs a spike (peak response). When the locust eyes perceive an approaching object, the LGMD neuron will be stimulated to output a spike signal to warn the locust of potential danger at distance. The high sensitivity of LGMD neurons in perceiving objects in motion3Xu J. Park S.H. Zhang X. A temporally irreversible visual attention model based on motion sensitive neurons.IEEE Trans. Ind. Inf. 2020; 16: 855-865Crossref Scopus (10) Google Scholar is suitable to promote vehicles to detect the approaching object. To take advantage of bio-inspired vision in collision detection, in this commentary, an innovative visual neural network that possesses similar collision selectivity of LGMD neurons is reported. The modeling of LGMD is shown in Figure 1A. The visual stimuli are perceived from the photoreceptor (P) layer to the excitation (E) and inhibition (I) layers, then the visual signals are decomposed into ON/OFF channels and fused separately in the summation (S) layer, and the grouping (G) layer is further aggregated for the output. The visual signals input to ON channels indicate the increased brightness in its navigation, and vice versa. When an obstacle approaches, visual stimulation will activate the LGMD neuron to detect potential collisions. The locust will then update its behavior to avoid collisions. This commentary discusses the impact of two human factors on driving: central fixation bias and binocular vision. So far, only a few studies have been conducted thoroughly on modeling human factors in driving and investigating its potential in ground robotics and aerial drones. Central fixation bias, which is a key factor in driving tasks as evidenced in Xu et al.,4Xu J. Guo K. Sun P.Z. Driving performance under violations of traffic rules: novice vs. experienced drivers.IEEE Trans. Intell. Veh. 2022; 7: 908-917Crossref Scopus (8) Google Scholar particularly for experienced drivers, is not thoroughly reflected in previous research. It is reported that human vision exerts a tendency of central fixation bias to actively