Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities

Abstract

This paper introduces a framework of gesture recognition operating on the output of an event based camera using the computational resources of a mobile phone. We will introduce a new development around the concept of time-surfaces modified and adapted to run on the limited computational resources of a mobile platform. We also introduce a new method to remove dynamically backgrounds that makes full use of the high temporal resolution of event-based cameras. We assess the performances of the framework by operating on several dynamic scenarios in uncontrolled lighting conditions indoors and outdoors. We also introduce a new publicly available event-based dataset for gesture recognition selected through a clinical process to allow human-machine interactions for the visually-impaired and the elderly. We finally report comparisons with prior works that tackled event-based gesture recognition reporting comparable if not superior results if taking into account the limited computational and memory constraints of the used hardware.

arXiv commentDraft version; final version published in Frontiers in Neuroscience (open access)

References in corpus (6)

• Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
• SLAYER: Spike Layer Error Reassignment in Time
• HFirst: A Temporal Approach to Object Recognition
• Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
• Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities
• A Real-time Hand Gesture Recognition and Human-Computer Interaction System

Cited by in corpus (7)

• Is Neuromorphic MNIST neuromorphic? Analyzing the discriminative power of neuromorphic datasets in the time domain
• Event-based Gesture Recognition with Dynamic Background Suppression using Smartphone Computational Capabilities
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision
• A Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless Networks
• Precise spiking motifs in neurobiological and neuromorphic data
• Fourier-based Action Recognition for Wildlife Behavior Quantification with Event Cameras
• EHWGesture -- A dataset for multimodal understanding of clinical gestures