10 papers
Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling
Qinwen Wang, Jieping Luo, Aoxiang Qin +5
Recurrent linear attention models (RLAs) such as Mamba offer efficient linear-time sequence modeling as an alternative to Transformers, yet their fixed-capacity recurrent states li…
SpikingMamba: Towards Energy-Efficient Large Language Models via Knowledge Distillation from Mamba
Yulong Huang, Jianxiong Tang, Chao Wang +5
Large Language Models (LLMs) have achieved remarkable performance across tasks but remain energy-intensive due to dense matrix operations. Spiking neural networks (SNNs) improve en…
Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks
Kejie Zhao, Wenjia Hua, Aiersi Tuerhong +3
Recently, spiking neural networks (SNNs), deployed on neuromorphic chips, provide highly efficient solutions on edge devices in different scenarios. However, their ability to adapt…
MAR: Efficient Large Language Models via Module-aware Architecture Refinement
Junhong Cai, Guiqin Wang, Kejie Zhao +6
Large Language Models (LLMs) excel across diverse domains but suffer from high energy costs due to quadratic attention and dense Feed-Forward Network (FFN) operations. To address t…
Hebbian Learning with Global Direction
Wenjia Hua, Kejie Zhao, Luziwei Leng +3
Backpropagation algorithm has driven the remarkable success of deep neural networks, but its lack of biological plausibility and high computational costs have motivated the ongoing…
Temporal-Guided Visual Foundation Models for Event-Based Vision
Ruihao Xia, Junhong Cai, Luziwei Leng +5
Event cameras offer unique advantages for vision tasks in challenging environments, yet processing asynchronous event streams remains an open challenge. While existing methods rely…