48 citations · 74 across the 13 of their papers we have counts for
13 papers
SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference
Yuqi Pan, Jinghao Zhuang, Yupeng Feng +16
Scaling context length is reshaping large-model development, yet full-attention Transformers suffer from prohibitive computation and inference bottlenecks at long sequences. A key…
A Survey on Latent Reasoning
Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng +30
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate s…
Scaling Linear Attention with Sparse State Expansion
Yuqi Pan, Yongqi An, Zheng Li +6
The Transformer architecture, despite its widespread success, struggles with long-context scenarios due to quadratic computation and linear memory growth. While various linear atte…
A Systematic Analysis of Hybrid Linear Attention
Dustin Wang, Rui-Jie Zhu, Steven Abreu +9
Transformers face quadratic complexity and memory issues with long sequences, prompting the adoption of linear attention mechanisms using fixed-size hidden states. However, linear…
Efficient 3D Recognition with Event-driven Spike Sparse Convolution
Xuerui Qiu, Man Yao, Jieyuan Zhang +5
Spiking Neural Networks (SNNs) provide an energy-efficient way to extract 3D spatio-temporal features. Point clouds are sparse 3D spatial data, which suggests that SNNs should be w…
Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training
Man Yao, Xuerui Qiu, Tianxiang Hu +7
The ambition of brain-inspired Spiking Neural Networks (SNNs) is to become a low-power alternative to traditional Artificial Neural Networks (ANNs). This work addresses two major c…
MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map
Yuhong Chou, Man Yao, Kexin Wang +7
Various linear complexity models, such as Linear Transformer (LinFormer), State Space Model (SSM), and Linear RNN (LinRNN), have been proposed to replace the conventional softmax a…
Scalable Autoregressive Image Generation with Mamba
Haopeng Li, Jinyue Yang, Kexin Wang +4
We introduce AiM, an autoregressive (AR) image generative model based on Mamba architecture. AiM employs Mamba, a novel state-space model characterized by its exceptional performan…
RSC-SNN: Exploring the Trade-off Between Adversarial Robustness and Accuracy in Spiking Neural Networks via Randomized Smoothing Coding
Keming Wu, Man Yao, Yuhong Chou +4
Spiking Neural Networks (SNNs) have received widespread attention due to their unique neuronal dynamics and low-power nature. Previous research empirically shows that SNNs with Poi…
Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection
Xinhao Luo, Man Yao, Yuhong Chou +2
Brain-inspired Spiking Neural Networks (SNNs) have bio-plausibility and low-power advantages over Artificial Neural Networks (ANNs). Applications of SNNs are currently limited to s…
High-Performance Temporal Reversible Spiking Neural Networks with Training Memory and Inference Cost
JiaKui Hu, Man Yao, Xuerui Qiu +6
Multi-timestep simulation of brain-inspired Spiking Neural Networks (SNNs) boost memory requirements during training and increase inference energy cost. Current training methods ca…
Gated Attention Coding for Training High-performance and Efficient Spiking Neural Networks
Xuerui Qiu, Rui-Jie Zhu, Yuhong Chou +3
Spiking neural networks (SNNs) are emerging as an energy-efficient alternative to traditional artificial neural networks (ANNs) due to their unique spike-based event-driven nature.…
Deep Directly-Trained Spiking Neural Networks for Object Detection
Qiaoyi Su, Yuhong Chou, Yifan Hu +4
Spiking neural networks (SNNs) are brain-inspired energy-efficient models that encode information in spatiotemporal dynamics. Recently, deep SNNs trained directly have shown great…