Event-based Neuromorphic Stereo Vision

Depth perception from stereo vision is a challenging problem, affecting both biological and artificial vision systems. It involves finding correspondences in the different views captured by the visual sensors. While animals solve this problem effortlessly, the field of machine vision has struggled for many years to find algorithms and strategies that can be as efficient and robust as those used in biological nervous systems. One major difference between biological visual systems and conventional machine vision systems, which is partly responsible for this large gap in performance, lies in the vision sensors themselves. Biological vision systems carry out self-timed and continuous sensing of the visual scene. Conversely, the use of frame-based cameras has been prevalent in the field of machine vision, which capture static images at regular time intervals. Machine stereo vision algorithms have been optimized to extract depth information from pairs of static images, typically acquired at frequencies of a few tens of Hertz. As a result, the most common approaches suffer from a trade-off between latency and computational cost, which is caused by the processing of redundant data from subsequent images. In recent times, however, a new class of event-based vision sensor has become available. These sensors are called silicon retinas because they are based on models of the mammalian visual retina. The silicon retina used in this work produces continuous streams of spikes (or events), that only encode the changes caused by movement in a scene, thus providing a form of visual output that is sparse and excludes redundant information. These new sensors now offer the possibility to implement efficient frame-less machine vision algorithms that are much more closely related to those of real biological systems.