48 citations · 69 across the 18 of their papers we have counts for
5 papers · 1 filter
SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference
Yuqi Pan, Jinghao Zhuang, Yupeng Feng +16
Scaling context length is reshaping large-model development, yet full-attention Transformers suffer from prohibitive computation and inference bottlenecks at long sequences. A key…
Adaptive Hopfield Network: Rethinking Similarities in Associative Memory
Shurong Wang, Yuqi Pan, Zhuoyang Shen +3
Associative memory models are content-addressable memory systems fundamental to biological intelligence and are notable for their high interpretability. However, existing models ev…
SpikingBrain: Spiking Brain-inspired Large Models
Yuqi Pan, Yupeng Feng, Jinghao Zhuang +16
Mainstream Transformer-based large language models face major efficiency bottlenecks: training computation scales quadratically with sequence length, and inference memory grows lin…
Scaling Linear Attention with Sparse State Expansion
Yuqi Pan, Yongqi An, Zheng Li +6
The Transformer architecture, despite its widespread success, struggles with long-context scenarios due to quadratic computation and linear memory growth. While various linear atte…
MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map
Yuhong Chou, Man Yao, Kexin Wang +7
Various linear complexity models, such as Linear Transformer (LinFormer), State Space Model (SSM), and Linear RNN (LinRNN), have been proposed to replace the conventional softmax a…