Applications of Spiking Neural Networks in Visual Place Recognition
arXiv:2311.13186 · doi:10.1109/TRO.2024.3508053
Abstract
In robotics, Spiking Neural Networks (SNNs) are increasingly recognized for their largely-unrealized potential energy efficiency and low latency particularly when implemented on neuromorphic hardware. Our paper highlights three advancements for SNNs in Visual Place Recognition (VPR). Firstly, we propose Modular SNNs, where each SNN represents a set of non-overlapping geographically distinct places, enabling scalable networks for large environments. Secondly, we present Ensembles of Modular SNNs, where multiple networks represent the same place, significantly enhancing accuracy compared to single-network models. Each of our Modular SNN modules is compact, comprising only 1500 neurons and 474k synapses, making them ideally suited for ensembling due to their small size. Lastly, we investigate the role of sequence matching in SNN-based VPR, a technique where consecutive images are used to refine place recognition. We demonstrate competitive performance of our method on a range of datasets, including higher responsiveness to ensembling compared to conventional VPR techniques and higher R@1 improvements with sequence matching than VPR techniques with comparable baseline performance. Our contributions highlight the viability of SNNs for VPR, offering scalable and robust solutions, and paving the way for their application in various energy-sensitive robotic tasks.
arXiv comment20 pages, 10 figures, IEEE Transactions on Robotics (TRO)
References in corpus (27)
- Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
- Event-based Vision: A Survey
- Ensemble deep learning: A review
- Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
- Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures
- A Survey on Ensemble Learning under the Era of Deep Learning
- SegMatch: Segment based loop-closure for 3D point clouds
- The Revisiting Problem in Simultaneous Localization and Mapping: A Survey on Visual Loop Closure Detection
- EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association
- Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence
- Where is your place, Visual Place Recognition?
- Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
- Multi-Process Fusion: Visual Place Recognition Using Multiple Image Processing Methods
- A Review of Modularization Techniques in Artificial Neural Networks
- Event-based visual place recognition with ensembles of temporal windows
- Visual Place Recognition: A Tutorial
- A Hybrid Compact Neural Architecture for Visual Place Recognition
- Delta Descriptors: Change-Based Place Representation for Robust Visual Localization
- Neuromorphic control for optic-flow-based landings of MAVs using the Loihi processor
- Gridbot: An autonomous robot controlled by a Spiking Neural Network mimicking the brain's navigational system
- The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware
- Probabilistic Visual Place Recognition for Hierarchical Localization
- Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments
- Improving Condition- and Environment-Invariant Place Recognition with Semantic Place Categorization
- Neuromorphic Visual Scene Understanding with Resonator Networks
- Intelligent Reference Curation for Visual Place Recognition via Bayesian Selective Fusion
- Boosting Performance of a Baseline Visual Place Recognition Technique by Predicting the Maximally Complementary Technique