16 papers
SpikeOcc: Towards Resilient Multimodal 3D Occupancy Perception via Spiking Experts
Yiding Sun, Zijie Xu, Hongqiang Lin +7
Multimodal 3D occupancy prediction underpins safe autonomous driving through comprehensive scene understanding. However, existing methods have largely pursued higher accuracy throu…
Can We Perform Online RL for Image Editing without Editing Rewards?
Qichao Ma, Jikang Cheng, Ling Liang +3
Reinforcement learning (RL) enables direct preference optimization for image editing through editing-specific rewards, which remain less developed due to costly triplet supervision…
SpikingMOT: A Spike-Driven Multi-Object Tracker
Yiding Sun, Xiangyang Yang, Dongxu Zhang +7
Multi-object tracking (MOT) plays a fundamental role in visual perception, where accurate trajectory prediction is essential for reliable target association under complex motion pa…
Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval
Xintao Zong, Xian Zhong, Wenxuan Liu +3
Spiking neural networks (SNNs) have recently shown strong potential in unimodal visual and textual tasks, yet building a directly trained, low-energy, and high-performance SNN for…
General Self-Prediction Enhancement for Spiking Neurons
Zihan Huang, Zijie Xu, Yihan Huang +7
Spiking Neural Networks (SNNs) are highly energy-efficient due to event-driven, sparse computation, but their training is challenged by spike non-differentiability and trade-offs a…
Unleashing Temporal Capacity of Spiking Neural Networks through Spatiotemporal Separation
Yiting Dong, Zhaofei Yu, Jianhao Ding +2
Spiking Neural Networks (SNNs) are considered naturally suited for temporal processing, with membrane potential propagation widely regarded as the core temporal modeling mechanism.…
Driving in Spikes: An Entropy-Guided Object Detector for Spike Cameras
Ziyan Liu, Qi Su, Lulu Tang +2
Object detection in autonomous driving suffers from motion blur and saturation under fast motion and extreme lighting. Spike cameras, offer microsecond latency and ultra high dynam…
HAD: Hierarchical Asymmetric Distillation to Bridge Spatio-Temporal Gaps in Event-Based Object Tracking
Yao Deng, Xian Zhong, Wenxuan Liu +3
RGB cameras excel at capturing rich texture details with high spatial resolution, whereas event cameras offer exceptional temporal resolution and a high dynamic range (HDR). Levera…
: Online RL Fine-tuning for Flow-based Vision-Language-Action Models
Kang Chen, Zhihao Liu, Tonghe Zhang +11
Vision-Language-Action (VLA) models enable robots to understand and perform complex tasks from multimodal input. Although recent work explores using reinforcement learning (RL) to…
SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streams
Zhuoheng Gao, Jiyao Zhang, Zhiyong Xie +5
Most robotic grasping systems rely on converting sensor data into explicit 3D point clouds, which is a computational step not found in biological intelligence. This paper explores…
PredNext: Explicit Cross-View Temporal Prediction for Unsupervised Learning in Spiking Neural Networks
Yiting Dong, Jianhao Ding, Zijie Xu +3
Spiking Neural Networks (SNNs), with their temporal processing capabilities and biologically plausible dynamics, offer a natural platform for unsupervised representation learning.…
SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams
Zhuoheng Gao, Yihao Li, Jiyao Zhang +7
Conventional frame-based cameras often struggle with stereo depth estimation in rapidly changing scenes. In contrast, bio-inspired spike cameras emit asynchronous events at microse…
SPKLIP: Aligning Spike Video Streams with Natural Language
Yongchang Gao, Meiling Jin, Zhaofei Yu +2
Spike cameras offer unique sensing capabilities but their sparse, asynchronous output challenges semantic understanding, especially for Spike Video-Language Alignment (Spike-VLA) w…
SOTA: Spike-Navigated Optimal TrAnsport Saliency Region Detection in Composite-bias Videos
Wenxuan Liu, Yao Deng, Kang Chen +3
Existing saliency detection methods struggle in real-world scenarios due to motion blur and occlusions. In contrast, spike cameras, with their high temporal resolution, significant…
SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams
Hanwen Liang, Xian Zhong, Wenxuan Liu +4
Restoring clear frames from rainy videos presents a significant challenge due to the rapid motion of rain streaks. Traditional frame-based visual sensors, which capture scene conte…
Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks
Xian Zhong, Shengwang Hu, Wenxuan Liu +4
Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biological interpretability. Their rich spatio-temporal information proc…