HFirst: A Temporal Approach to Object Recognition

Abstract

This paper introduces a spiking hierarchical model for object recognition which utilizes the precise timing information inherently present in the output of biologically inspired asynchronous Address Event Representation (AER) vision sensors. The asynchronous nature of these systems frees computation and communication from the rigid predetermined timing enforced by system clocks in conventional systems. Freedom from rigid timing constraints opens the possibility of using true timing to our advantage in computation. We show not only how timing can be used in object recognition, but also how it can in fact simplify computation. Specifically, we rely on a simple temporal-winner-take-all rather than more computationally intensive synchronous operations typically used in biologically inspired neural networks for object recognition. This approach to visual computation represents a major paradigm shift from conventional clocked systems and can find application in other sensory modalities and computational tasks. We showcase effectiveness of the approach by achieving the highest reported accuracy to date (97.5\%$\pm$3.5\%) for a previously published four class card pip recognition task and an accuracy of 84.9\%$\pm$1.9\% for a new more difficult 36 class character recognition task.

13 pages, 10 figures

References in corpus (2)

• Bioinspired Visual Motion Estimation
• Fast Neuromimetic Object Recognition using FPGA Outperforms GPU Implementations

Cited by in corpus (40)

• Event-based Vision: A Survey
• A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)
• Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars
• S4NN: temporal backpropagation for spiking neural networks with one spike per neuron
• Graph-based Spatial-temporal Feature Learning for Neuromorphic Vision Sensing
• Focus Is All You Need: Loss Functions For Event-based Vision
• Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural Networks
• BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning
• Spiking Deep Residual Network
• Is Neuromorphic MNIST neuromorphic? Analyzing the discriminative power of neuromorphic datasets in the time domain
• Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities
• Design Space Exploration of Hardware Spiking Neurons for Embedded Artificial Intelligence
• Inceptive Event Time-Surfaces for Object Classification Using Neuromorphic Cameras
• Event-based Camera Pose Tracking using a Generative Event Model
• Spatiotemporal Filtering for Event-Based Action Recognition
• DART: Distribution Aware Retinal Transform for Event-based Cameras
• Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams
• EvGNN: An Event-driven Graph Neural Network Accelerator for Edge Vision
• A Tandem Learning Rule for Effective Training and Rapid Inference of Deep Spiking Neural Networks
• EvRepSL: Event-Stream Representation via Self-Supervised Learning for Event-Based Vision
• Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differences
• Temporal State Machines: Using temporal memory to stitch time-based graph computations
• Memory-Efficient Graph Convolutional Networks for Object Classification and Detection with Event Cameras
• Automotive Object Detection via Learning Sparse Events by Spiking Neurons
• Temporal Pulses Driven Spiking Neural Network for Fast Object Recognition in Autonomous Driving
• Neuromorphic Imaging with Super-Resolution
• N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras
• A Spike Learning System for Event-driven Object Recognition
• Events-to-Video: Bringing Modern Computer Vision to Event Cameras
• Learning to Exploit Multiple Vision Modalities by Using Grafted Networks
• Event Probability Mask (EPM) and Event Denoising Convolutional Neural Network (EDnCNN) for Neuromorphic Cameras
• Covariant spatio-temporal receptive fields for spiking neural networks
• Spatiotemporal Feature Learning for Event-Based Vision
• Unsupervised AER Object Recognition Based on Multiscale Spatio-Temporal Features and Spiking Neurons
• Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks
• Shapes Characterization on Address Event Representation Using Histograms of Oriented Events and an Extended LBP Approach
• Investigation of event-based memory surfaces for high-speed tracking, unsupervised feature extraction and object recognition
• High Speed Event Camera TRacking
• Asynchronous "Events" are Better For Motion Estimation
• Spiking Neural Network based Region Proposal Networks for Neuromorphic Vision Sensors