Early Image Detection Using Event Based Vision

Image recognition is a basic yet necessary task for real time applications. Although there is massive advancement in deep learning models for image recognition, they are all designed for normal images produced by frame-based cameras which are time and power hungry. This work aims to explore the possibility of using event based cameras (EBC) for performing image recognition based on asynchronous events produced in response to changes per pixel rather than waiting for the full-frame image, potentially leading to higher recognition speed and lower power consumption. First, we collect and label raw event data of the MNIST testing dataset using CeleX, a 1 Mega Pixel event-based sensor. Then we evaluate the raw data on a convolutional neural network (CNN) model that is pre-trained using the original full frame image dataset, to calculate the time difference between waiting for a full frame image against asynchronous events. The results demonstrate that using EBC allows us to achieve early image recognition which provides major benefits for real time processing in terms of time and power.