Publications (80)
SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams
Zhuoheng Gao, Yihao Li, Jiyao Zhang +7
Conventional frame-based cameras often struggle with stereo depth estimation in rapidly changing scenes. In contrast, bio-inspired spike cameras emit asynchronous events at microse…
Differential Coding for Training-Free ANN-to-SNN Conversion
Zihan Huang, Wei Fang, Tong Bu +6
Spiking Neural Networks (SNNs) exhibit significant potential due to their low energy consumption. Converting Artificial Neural Networks (ANNs) to SNNs is an efficient way to achiev…
TDFormer: A Top-Down Attention-Controlled Spiking Transformer
Zizheng Zhu, Yingchao Yu, Zeqi Zheng +2
Traditional spiking neural networks (SNNs) can be viewed as a combination of multiple subnetworks with each running for one time step, where the parameters are shared, and the memb…
Deep Residual Learning in Spiking Neural Networks
Wei Fang, Zhaofei Yu, Yanqi Chen +3
Deep Spiking Neural Networks (SNNs) present optimization difficulties for gradient-based approaches due to discrete binary activation and complex spatial-temporal dynamics. Conside…
Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks
Xian Zhong, Shengwang Hu, Wenxuan Liu +4
Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biological interpretability. Their rich spatio-temporal information proc…
One Forward is Enough for Neural Network Training via Likelihood Ratio Method
Jinyang Jiang, Zeliang Zhang, Chenliang Xu +2
While backpropagation (BP) is the mainstream approach for gradient computation in neural network training, its heavy reliance on the chain rule of differentiation constrains the de…
AS-FedBridge: Pseudo-Spike Bridge Distillation for Heterogeneous ANN-SNN Federated Learning
Shengyang Li, Yiting Dong, Liuyang Song +5
Federated learning enables collaborative model training across distributed edge devices while strictly preserving data privacy. To facilitate practical deployment on resource-const…
Towards the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes
Zhaofei Yu, Jian K. Liu, Shanshan Jia +4
Neuroprosthesis, as one type of precision medicine device, is aiming for manipulating neuronal signals of the brain in a closed-loop fashion, together with receiving stimulus from…
GemS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer
Zecheng Hao, Shenghao Xie, Kang Chen +3
Spiking Neural Networks (SNNs) offer superior energy efficiency over Artificial Neural Networks (ANNs). However, they encounter significant deficiencies in training and inference m…
Rethinking SNN Online Training and Deployment: Gradient-Coherent Learning via Hybrid-Driven LIF Model
Zecheng Hao, Yifan Huang, Zijie Xu +4
Spiking Neural Networks (SNNs) are considered to have enormous potential in the future development of Artificial Intelligence due to their brain-inspired and energy-efficient prope…
HAD: Hierarchical Asymmetric Distillation to Bridge Spatio-Temporal Gaps in Event-Based Object Tracking
Yao Deng, Xian Zhong, Wenxuan Liu +3
RGB cameras excel at capturing rich texture details with high spatial resolution, whereas event cameras offer exceptional temporal resolution and a high dynamic range (HDR). Levera…
Rethinking High-speed Image Reconstruction Framework with Spike Camera
Kang Chen, Yajing Zheng, Tiejun Huang +1
Spike cameras, as innovative neuromorphic devices, generate continuous spike streams to capture high-speed scenes with lower bandwidth and higher dynamic range than traditional RGB…
Unleashing Temporal Capacity of Spiking Neural Networks through Spatiotemporal Separation
Yiting Dong, Zhaofei Yu, Jianhao Ding +2
Spiking Neural Networks (SNNs) are considered naturally suited for temporal processing, with membrane potential propagation widely regarded as the core temporal modeling mechanism.…
SpikeMM: Flexi-Magnification of High-Speed Micro-Motions
Baoyue Zhang, Yajing Zheng, Shiyan Chen +4
The amplification of high-speed micro-motions holds significant promise, with applications spanning fault detection in fast-paced industrial environments to refining precision in m…
SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks
Xinyu Shi, Zecheng Hao, Zhaofei Yu
The remarkable success of Vision Transformers in Artificial Neural Networks (ANNs) has led to a growing interest in incorporating the self-attention mechanism and transformer-based…
SpikeCV: Open a Continuous Computer Vision Era
Yajing Zheng, Jiyuan Zhang, Rui Zhao +5
SpikeCV is a new open-source computer vision platform for the spike camera, which is a neuromorphic visual sensor that has developed rapidly in recent years. In the spike camera, e…
CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning
Zijie Xu, Xinyu Shi, Yiting Dong +2
Spiking Neural Networks (SNNs) offer low-latency and energy-efficient decision-making on neuromorphic hardware by mimicking the event-driven dynamics of biological neurons. However…
Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation
Ziluo Ding, Rui Zhao, Jiyuan Zhang +4
Event camera has offered promising alternative for visual perception, especially in high speed and high dynamic range scenes. Recently, many deep learning methods have shown great…
SpiLiFormer: Enhancing Spiking Transformers with Lateral Inhibition
Zeqi Zheng, Yanchen Huang, Yingchao Yu +4
Spiking Neural Networks (SNNs) based on Transformers have garnered significant attention due to their superior performance and high energy efficiency. However, the spiking attentio…
A Unified Framework for Soft Threshold Pruning
Yanqi Chen, Zhengyu Ma, Wei Fang +3
Soft threshold pruning is among the cutting-edge pruning methods with state-of-the-art performance. However, previous methods either perform aimless searching on the threshold sche…
Error Amplification Limits ANN-to-SNN Conversion in Continuous Control
Zijie Xu, Zihan Huang, Yiting Dong +3
Spiking Neural Networks (SNNs) can achieve competitive performance by converting already existing well-trained Artificial Neural Networks (ANNs), avoiding further costly training.…
Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Networks
Qi Yan, Yajing Zheng, Shanshan Jia +6
Deep convolutional neural networks (CNNs) have demonstrated impressive performance on many visual tasks. Recently, they became useful models for the visual system in neuroscience.…
Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views
Jiyuan Zhang, Shiyan Chen, Yajing Zheng +2
The extraction of a clean background image by removing foreground occlusion holds immense practical significance, but it also presents several challenges. Presently, the majority o…
Autaptic Synaptic Circuit Enhances Spatio-temporal Predictive Learning of Spiking Neural Networks
Lihao Wang, Zhaofei Yu
Spiking Neural Networks (SNNs) emulate the integrated-fire-leak mechanism found in biological neurons, offering a compelling combination of biological realism and energy efficiency…
Deep Learning for Visual Neuroprosthesis
Peter Beech, Shanshan Jia, Zhaofei Yu +1
The visual pathway involves complex networks of cells and regions which contribute to the encoding and processing of visual information. While some aspects of visual perception are…
Neural System Identification with Spike-triggered Non-negative Matrix Factorization
Shanshan Jia, Zhaofei Yu, Arno Onken +3
Neuronal circuits formed in the brain are complex with intricate connection patterns. Such complexity is also observed in the retina as a relatively simple neuronal circuit. A reti…
Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios
Shiyan Chen, Jiyuan Zhang, Zhaofei Yu +1
Self-supervised denoising has attracted widespread attention due to its ability to train without clean images. However, noise in real-world scenarios is often spatially correlated,…
Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikes
Zecheng Hao, Jianhao Ding, Tong Bu +2
Spiking Neural Networks (SNNs) have attracted great attention due to their distinctive characteristics of low power consumption and temporal information processing. ANN-SNN convers…
Implementing feature binding through dendritic networks of a single neuron
Yuanhong Tang, Shanshan Jia, Tiejun Huang +2
A single neuron receives an extensive array of synaptic inputs through its dendrites, raising the fundamental question of how these inputs undergo integration and summation, culmin…
INeAT: Iterative Neural Adaptive Tomography
Bo Xiong, Changqing Su, Zihan Lin +2
Computed Tomography (CT) with its remarkable capability for three-dimensional imaging from multiple projections, enjoys a broad range of applications in clinical diagnosis, scienti…
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks
Wei Fang, Zhaofei Yu, Yanqi Chen +3
Spiking Neural Networks (SNNs) have attracted enormous research interest due to temporal information processing capability, low power consumption, and high biological plausibility.…
SOTA: Spike-Navigated Optimal TrAnsport Saliency Region Detection in Composite-bias Videos
Wenxuan Liu, Yao Deng, Kang Chen +3
Existing saliency detection methods struggle in real-world scenarios due to motion blur and occlusions. In contrast, spike cameras, with their high temporal resolution, significant…
SPKLIP: Aligning Spike Video Streams with Natural Language
Yongchang Gao, Meiling Jin, Zhaofei Yu +2
Spike cameras offer unique sensing capabilities but their sparse, asynchronous output challenges semantic understanding, especially for Spike Video-Language Alignment (Spike-VLA) w…
Faster and Stronger: When ANN-SNN Conversion Meets Parallel Spiking Calculation
Zecheng Hao, Qichao Ma, Kang Chen +3
Spiking Neural Network (SNN), as a brain-inspired and energy-efficient network, is currently facing the pivotal challenge of exploring a suitable and efficient learning framework.…
Winner-Take-All as Basic Probabilistic Inference Unit of Neuronal Circuits
Zhaofei Yu, Yonghong Tian, Tiejun Huang +1
Experimental observations of neuroscience suggest that the brain is working a probabilistic way when computing information with uncertainty. This processing could be modeled as Bay…
SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams
Hanwen Liang, Xian Zhong, Wenxuan Liu +4
Restoring clear frames from rainy videos presents a significant challenge due to the rapid motion of rain streaks. Traditional frame-based visual sensors, which capture scene conte…
Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval
Xintao Zong, Xian Zhong, Wenxuan Liu +3
Spiking neural networks (SNNs) have recently shown strong potential in unimodal visual and textual tasks, yet building a directly trained, low-energy, and high-performance SNN for…
Deep Pulse-Coupled Neural Networks
Zexiang Yi, Jing Lian, Yunliang Qi +4
Spiking Neural Networks (SNNs) capture the information processing mechanism of the brain by taking advantage of spiking neurons, such as the Leaky Integrate-and-Fire (LIF) model ne…
Spike Stream Denoising via Spike Camera Simulation
Liwen hu, Lei Ma, Zhaofei Yu +2
As a neuromorphic sensor with high temporal resolution, the spike camera shows enormous potential in high-speed visual tasks. However, the high-speed sampling of light propagation…
Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks
Tong Bu, Wei Fang, Jianhao Ding +3
Spiking Neural Networks (SNNs) have gained great attraction due to their distinctive properties of low power consumption and fast inference on neuromorphic hardware. As the most ef…
CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling
Zhaofei Yu, David Kappel, Robert Legenstein +3
Synaptic plasticity is implemented and controlled through over thousand different types of molecules in the postsynaptic density and presynaptic boutons that assume a staggering ar…
Latency Coding for Efficient and Low-Latency Deep Spiking Neural Networks
Yi Lu, Jianhao Ding, Zhaofei Yu
The paper introduces latency coding, an extension of time‑to‑first‑spike coding, and a training framework using backpropagation through time to build deep spiking neural networks t…
Optimized Potential Initialization for Low-latency Spiking Neural Networks
Tong Bu, Jianhao Ding, Zhaofei Yu +1
Spiking Neural Networks (SNNs) have been attached great importance due to the distinctive properties of low power consumption, biological plausibility, and adversarial robustness.…
Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation
Yajing Zheng, Shanshan Jia, Zhaofei Yu +3
Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like…
A Novel Noise Injection-based Training Scheme for Better Model Robustness
Zeliang Zhang, Jinyang Jiang, Minjie Chen +3
Noise injection-based method has been shown to be able to improve the robustness of artificial neural networks in previous work. In this work, we propose a novel noise injection-ba…
Driving in Spikes: An Entropy-Guided Object Detector for Spike Cameras
Ziyan Liu, Qi Su, Lulu Tang +2
Object detection in autonomous driving suffers from motion blur and saturation under fast motion and extreme lighting. Spike cameras, offer microsecond latency and ultra high dynam…
SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streams
Zhuoheng Gao, Jiyao Zhang, Zhiyong Xie +5
Most robotic grasping systems rely on converting sensor data into explicit 3D point clouds, which is a computational step not found in biological intelligence. This paper explores…
Spike Imaging Velocimetry: Dense Motion Estimation of Fluids Using Spike Cameras
Yunzhong Zhang, Bo Xiong, You Zhou +5
Particle Image Velocimetry (PIV) is a widely adopted non-invasive imaging technique that tracks the motion of tracer particles across image sequences to capture the velocity distri…
High-Speed Full-Color HDR Imaging via Unwrapping Modulo-Encoded Spike Streams
Chu Zhou, Siqi Yang, Kailong Zhang +4
Conventional RGB-based high dynamic range (HDR) imaging faces a fundamental trade-off between motion artifacts in multi-exposure captures and irreversible information loss in singl…
Training Deep Normalization-Free Spiking Neural Networks with Lateral Inhibition
Peiyu Liu, Jianhao Ding, Zhaofei Yu
Spiking Neural Networks (SNNs) have garnered significant attention as a central paradigm in neuromorphic computing, owing to their energy efficiency and biological plausibility. Ho…
Sampling-based Causal Inference in Cue Combination and its Neural Implementation
Zhaofei Yu, Feng Chen, Jianwu Dong +1
Causal inference in cue combination is to decide whether the cues have a single cause or multiple causes. Although the Bayesian causal inference model explains the problem of causa…
USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting
Kang Chen, Jiyuan Zhang, Zecheng Hao +3
Spike cameras, as an innovative neuromorphic camera that captures scenes with the 0-1 bit stream at 40 kHz, are increasingly employed for the 3D reconstruction task via Neural Radi…
Inter-event Interval Microscopy for Event Cameras
Changqing Su, Yanqin Chen, Zihan Lin +5
Event cameras, an innovative bio-inspired sensor, differ from traditional cameras by sensing changes in intensity rather than directly perceiving intensity and recording these vari…
Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecture
Qichao Ma, Rui-Jie Zhu, Peiye Liu +8
Large Language Models (LLMs) utilize extensive knowledge databases and show powerful text generation ability. However, their reliance on high-quality copyrighted datasets raises co…
A Chaotic Dynamics Framework Inspired by Dorsal Stream for Event Signal Processing
Yu Chen, Jing Lian, Zhaofei Yu +3
Event cameras are bio-inspired vision sensor that encode visual information with high dynamic range, high temporal resolution, and low latency.Current state-of-the-art event stream…
Towards High-performance Spiking Transformers from ANN to SNN Conversion
Zihan Huang, Xinyu Shi, Zecheng Hao +4
Spiking neural networks (SNNs) show great potential due to their energy efficiency, fast processing capabilities, and robustness. There are two main approaches to constructing SNNs…
Converting High-Performance and Low-Latency SNNs through Explicit Modelling of Residual Error in ANNs
Zhipeng Huang, Jianhao Ding, Zhiyu Pan +4
Spiking neural networks (SNNs) have garnered interest due to their energy efficiency and superior effectiveness on neuromorphic chips compared with traditional artificial neural ne…
Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks
Jianhao Ding, Zhaofei Yu, Yonghong Tian +1
Spiking Neural Networks (SNNs), as bio-inspired energy-efficient neural networks, have attracted great attentions from researchers and industry. The most efficient way to train dee…
: Online RL Fine-tuning for Flow-based Vision-Language-Action Models
Kang Chen, Zhihao Liu, Tonghe Zhang +11
Vision-Language-Action (VLA) models enable robots to understand and perform complex tasks from multimodal input. Although recent work explores using reinforcement learning (RL) to…
Reconstruction of Natural Visual Scenes from Neural Spikes with Deep Neural Networks
Yichen Zhang, Shanshan Jia, Yajing Zheng +5
Neural coding is one of the central questions in systems neuroscience for understanding how the brain processes stimulus from the environment, moreover, it is also a cornerstone fo…
Inference-Scale Complexity in ANN-SNN Conversion for High-Performance and Low-Power Applications
Tong Bu, Maohua Li, Zhaofei Yu
Spiking Neural Networks (SNNs) have emerged as a promising substitute for Artificial Neural Networks (ANNs) due to their advantages of fast inference and low power consumption. How…
SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams
Kang Chen, Shiyan Chen, Jiyuan Zhang +4
Reconstructing a sequence of sharp images from the blurry input is crucial for enhancing our insights into the captured scene and poses a significant challenge due to the limited t…
Revealing structure components of the retina by deep learning networks
Qi Yan, Zhaofei Yu, Feng Chen +1
Deep convolutional neural networks (CNNs) have demonstrated impressive performance on visual object classification tasks. In addition, it is a useful model for predication of neuro…
SpikingMOT: A Spike-Driven Multi-Object Tracker
Yiding Sun, Xiangyang Yang, Dongxu Zhang +7
Multi-object tracking (MOT) plays a fundamental role in visual perception, where accurate trajectory prediction is essential for reliable target association under complex motion pa…
PredNext: Explicit Cross-View Temporal Prediction for Unsupervised Learning in Spiking Neural Networks
Yiting Dong, Jianhao Ding, Zijie Xu +3
Spiking Neural Networks (SNNs), with their temporal processing capabilities and biologically plausible dynamics, offer a natural platform for unsupervised representation learning.…
Spike timing reshapes robustness against attacks in spiking neural networks
Jianhao Ding, Zhaofei Yu, Tiejun Huang +1
The success of deep learning in the past decade is partially shrouded in the shadow of adversarial attacks. In contrast, the brain is far more robust at complex cognitive tasks. Ut…
Pruning of Deep Spiking Neural Networks through Gradient Rewiring
Yanqi Chen, Zhaofei Yu, Wei Fang +2
Spiking Neural Networks (SNNs) have been attached great importance due to their biological plausibility and high energy-efficiency on neuromorphic chips. As these chips are usually…
STF: Shallow-Level Temporal Feedback to Enhance Spiking Transformers
Zeqi Zheng, Zizheng Zhu, Yingchao Yu +5
Transformer-based Spiking Neural Networks (SNNs) suffer from a great performance gap compared to floating-point \mbox{Artificial} Neural Networks (ANNs) due to the binary nature of…
SpikeGS: 3D Gaussian Splatting from Spike Streams with High-Speed Camera Motion
Jiyuan Zhang, Kang Chen, Shiyan Chen +3
Novel View Synthesis plays a crucial role by generating new 2D renderings from multi-view images of 3D scenes. However, capturing high-speed scenes with conventional cameras often…
Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Control
Zijie Xu, Tong Bu, Zecheng Hao +2
Spiking Neural Networks (SNNs) offer low-latency and energy-efficient decision making on neuromorphic hardware, making them attractive for Reinforcement Learning (RL) in resource-c…
1000x Faster Camera and Machine Vision with Ordinary Devices
Tiejun Huang, Yajing Zheng, Zhaofei Yu +13
In digital cameras, we find a major limitation: the image and video form inherited from a film camera obstructs it from capturing the rapidly changing photonic world. Here, we pres…
Enhancing Adversarial Robustness in SNNs with Sparse Gradients
Yujia Liu, Tong Bu, Jianhao Ding +3
Spiking Neural Networks (SNNs) have attracted great attention for their energy-efficient operations and biologically inspired structures, offering potential advantages over Artific…
Reducing ANN-SNN Conversion Error through Residual Membrane Potential
Zecheng Hao, Tong Bu, Jianhao Ding +2
Spiking Neural Networks (SNNs) have received extensive academic attention due to the unique properties of low power consumption and high-speed computing on neuromorphic chips. Amon…
General Self-Prediction Enhancement for Spiking Neurons
Zihan Huang, Zijie Xu, Yihan Huang +7
Spiking Neural Networks (SNNs) are highly energy-efficient due to event-driven, sparse computation, but their training is challenged by spike non-differentiability and trade-offs a…
LM-HT SNN: Enhancing the Performance of SNN to ANN Counterpart through Learnable Multi-hierarchical Threshold Model
Zecheng Hao, Xinyu Shi, Yujia Liu +2
Compared to traditional Artificial Neural Network (ANN), Spiking Neural Network (SNN) has garnered widespread academic interest for its intrinsic ability to transmit information in…
SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence
Wei Fang, Yanqi Chen, Jianhao Ding +7
Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency by introducing neural dynamics and spike properties. As…
Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies
Wei Fang, Zhaofei Yu, Zhaokun Zhou +5
Vanilla spiking neurons in Spiking Neural Networks (SNNs) use charge-fire-reset neuronal dynamics, which can only be simulated serially and can hardly learn long-time dependencies.…
Robust Stable Spiking Neural Networks
Jianhao Ding, Zhiyu Pan, Yujia Liu +2
Spiking neural networks (SNNs) are gaining popularity in deep learning due to their low energy budget on neuromorphic hardware. However, they still face challenges in lacking suffi…
Unsupervised Optical Flow Estimation with Dynamic Timing Representation for Spike Camera
Lujie Xia, Ziluo Ding, Rui Zhao +5
Efficiently selecting an appropriate spike stream data length to extract precise information is the key to the spike vision tasks. To address this issue, we propose a dynamic timin…
Uncertainty-Aware Token Importance Estimation in Spiking Transformers
Wenxuan Liu, Zecheng Hao, Tong Bu +2
Spiking transformers have shown strong potential for neuromorphic vision, yet their token processing across multiple spiking steps still introduces substantial redundancy and infer…