298 citations · 853 across the 29 of their papers we have counts for
31 papers
Towards a physically realistic computationally efficient DVS pixel model
Rui Graca, Tobi Delbruck
Dynamic Vision Sensor (DVS) event camera models are important tools for predicting camera response, optimizing biases, and generating realistic simulated datasets. Existing DVS mod…
Modulating State Space Model with SlowFast Framework for Compute-Efficient Ultra Low-Latency Speech Enhancement
Longbiao Cheng, Ashutosh Pandey, Buye Xu +3
Deep learning-based speech enhancement (SE) methods often face significant computational challenges when needing to meet low-latency requirements because of the increased number of…
Steering Prediction via a Multi-Sensor System for Autonomous Racing
Zhuyun Zhou, Zongwei Wu, Florian Bolli +5
Autonomous racing has rapidly gained research attention. Traditionally, racing cars rely on 2D LiDAR as their primary visual system. In this work, we explore the integration of an…
Dynamic Gated Recurrent Neural Network for Compute-efficient Speech Enhancement
Longbiao Cheng, Ashutosh Pandey, Buye Xu +2
This paper introduces a new Dynamic Gated Recurrent Neural Network (DG-RNN) for compute-efficient speech enhancement models running on resource-constrained hardware platforms. It l…
SciDVS: A Scientific Event Camera with 1.7% Temporal Contrast Sensitivity at 0.7 lux
Rui Graca, Sheng Zhou, Brian McReynolds +1
This paper reports a Dynamic Vision Sensor (DVS) event camera that is 6x more sensitive at 14x lower illumination than existing commercial and prototype cameras. Event cameras outp…
Hardware Neural Control of CartPole and F1TENTH Race Car
Marcin Paluch, Florian Bolli, Xiang Deng +3
Nonlinear model predictive control (NMPC) has proven to be an effective control method, but it is expensive to compute. This work demonstrates the use of hardware FPGA neural netwo…
DeltaKWS: A 65nm 36nJ/Decision Bio-inspired Temporal-Sparsity-Aware Digital Keyword Spotting IC with 0.6V Near-Threshold SRAM
Qinyu Chen, Kwantae Kim, Chang Gao +4
This paper introduces DeltaKWS, to the best of our knowledge, the first RNN-enabled fine-grained temporal sparsity-aware KWS IC for voice-controlled devices. The 65 nm prototype…
Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training
Xi Chen, Chang Gao, Zuowen Wang +4
Recurrent Neural Networks (RNNs) are useful in temporal sequence tasks. However, training RNNs involves dense matrix multiplications which require hardware that can support a large…
Optimal biasing and physical limits of DVS event noise
Rui Graca, Brian McReynolds, Tobi Delbruck
Under dim lighting conditions, the output of Dynamic Vision Sensor (DVS) event cameras is strongly affected by noise. Photon and electron shot-noise cause a high rate of non-inform…
Exploiting Alternating DVS Shot Noise Event Pair Statistics to Reduce Background Activity
Brian McReynolds, Rui Graca, Tobi Delbruck
Dynamic Vision Sensors (DVS) record "events" corresponding to pixel-level brightness changes, resulting in data-efficient representation of a dynamic visual scene. As DVS expand in…
Shining light on the DVS pixel: A tutorial and discussion about biasing and optimization
Rui Graça, Brian McReynolds, Tobi Delbruck
The operation of the DVS event camera is controlled by the user through adjusting different bias parameters. These biases affect the response of the camera by controlling - among o…
Measuring diameters and velocities of artificial raindrops with a neuromorphic dynamic vision sensor disdrometer
Jan Steiner, Kire Micev, Asude Aydin +2
Hydrometers that can measure size and velocity distributions of precipitation are needed for research and corrections of rainfall estimates from weather radars and microwave links.…
A 23 W Keyword Spotting IC with Ring-Oscillator-Based Time-Domain Feature Extraction
Kwantae Kim, Chang Gao, Rui Graça +4
This article presents the first keyword spotting (KWS) IC which uses a ring-oscillator-based time-domain processing technique for its analog feature extractor (FEx). Its extensive…
Utility and Feasibility of a Center Surround Event Camera
Tobi Delbruck, Chenghan Li, Rui Graca +1
Standard dynamic vision sensor (DVS) event cameras output a stream of spatially-independent log-intensity brightness change events so they cannot suppress spatial redundancy. Nearl…
Bio-inspired Polarization Event Camera
Germain Haessig, Damien Joubert, Justin Haque +4
The stomatopod (mantis shrimp) visual system has recently provided a blueprint for the design of paradigm-shifting polarization and multispectral imaging sensors, enabling solution…
Unraveling the paradox of intensity-dependent DVS pixel noise
Rui Graca, Tobi Delbruck
Dynamic vision sensor (DVS) event camera output is affected by noise, particularly in dim lighting conditions. A theory explaining how photon and electron noise affect DVS output e…
Feedback control of event cameras
Tobi Delbruck, Rui Graca, Marcin Paluch
Dynamic vision sensor event cameras produce a variable data rate stream of brightness change events. Event production at the pixel level is controlled by threshold, bandwidth, and…
EdgeDRNN: Recurrent Neural Network Accelerator for Edge Inference
Chang Gao, Antonio Rios-Navarro, Xi Chen +2
Low-latency, low-power portable recurrent neural network (RNN) accelerators offer powerful inference capabilities for real-time applications such as IoT, robotics, and human-machin…
v2e: From Video Frames to Realistic DVS Events
Yuhuang Hu, Shih-Chii Liu, Tobi Delbruck
To help meet the increasing need for dynamic vision sensor (DVS) event camera data, this paper proposes the v2e toolbox that generates realistic synthetic DVS events from intensity…
DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction
Yuhuang Hu, Jonathan Binas, Daniel Neil +2
Neuromorphic event cameras are useful for dynamic vision problems under difficult lighting conditions. To enable studies of using event cameras in automobile driving applications,…
Data-Driven Neuromorphic DRAM-based CNN and RNN Accelerators
Tobi Delbruck, Shih-Chii Liu
The energy consumed by running large deep neural networks (DNNs) on hardware accelerators is dominated by the need for lots of fast memory to store both states and weights. This la…
Learning to Exploit Multiple Vision Modalities by Using Grafted Networks
Yuhuang Hu, Tobi Delbruck, Shih-Chii Liu
Novel vision sensors such as thermal, hyperspectral, polarization, and event cameras provide information that is not available from conventional intensity cameras. An obstacle to u…
Dynamic Vision Sensor integration on FPGA-based CNN accelerators for high-speed visual classification
Alejandro Linares-Barranco, Antonio Rios-Navarro, Ricardo Tapiador-Morales +1
Deep-learning is a cutting edge theory that is being applied to many fields. For vision applications the Convolutional Neural Networks (CNN) are demanding significant accuracy for…
EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras
Anton Mitrokhin, Chengxi Ye, Cornelia Fermuller +2
We present the first event-based learning approach for motion segmentation in indoor scenes and the first event-based dataset - EV-IMO - which includes accurate pixel-wise motion m…