Analog and digital implementations of retinal processing for robot navigation systems

Abstract

This thesis presents both analog and digital implementations of visual processing inspired by the biological retina.The aim is to use this processing for a tracking task to allow robotic navigation within a chase scenario.Although the full extent of retinal processing is not yet fully understood due to its enormous complexity, its interesting aspects are brought into the context of electronic engineering and computer science: in analog design, signal processing and machine learning.First, the basic functioning of the biological retina is explained as well as its analogies with the Dynamic Vision Sensor (DVS) or "Silicon Retina", which is the sensor used for the task.This sensor is the fundamental hardware on which this entire doctorate is based on.Its activity-dependent spiking output is used to mimic the photoreceptors' response to light changes, at high speeds and with large dynamic range.The sensor's output data is then processed either in the jAER JAVA framework or on FPGA, where neural circuits of the Retinal Ganglion Cells (RGC) are mimicked in order to extract useful information.Personal work done in this area has proved that it is indeed possible to use an ensemble of Object Motion Sensitive (OMS) RGCs in order to detect object motion and track it, under the condition of little or no ego-motion.The current OMS algorithm fails for self-motion of the DVS as the entire and more complex cellular interactions of RGCs is not re-created and the entire processing relies on a single cell-type.To overcome this problem, the approach of machine learning was adopted as a consequence.Specifically targeting a predator/prey scenario, a Convolutional Neural Network was setup in order to provide steering directions to the predator robot and allow it to move, in an arena, following its prey.A relatively small network was used and fed with DVS histograms created at an activity-dependent rate.The convolution kernels learned, i.e. the pattern that the network looks for in the scene, still resemble the receptive fields of the biological retina.However, the empirical knowledge surrounding machine learning algorithms does not yet allow to fully make a detailed comparison.Practical on-field tests were performed and surprisingly successful results were obtained.Finally, to further improve the basic technology on which the algorithmic work relies on, another, more sensitive, silicon retina was designed.A more sensitive sensor means more details, and therefore information, picked up in a scene.This thesis presents its design and characterization according to newly set and more precise standards, defining Signal-to-Noise Ratios for example.The contrast detected can be down to 0.95% for negative logarithmic changes in light intensity with this sensor.The high gain comes at the cost of a reduced intra-scene dynamic range; therefore, an adaptation mechanism is also present in the sensor in order to match the scene's median illumination.Another interesting application for using this sensor, in the context of calcium imaging in neurons, to detect their activity, is also discussed and preliminary results are shown.