1 citations · 1 across the 12 of their papers we have counts for
12 papers
LoGo: Token-Level Dynamic Local-Global Attention
Yuqi Pan, Zheng Li, Bohao Tang +2
As context lengths scale, attention increasingly becomes a primary computational bottleneck in large language models. Standard Transformers remain powerful but computationally inef…
Modular TTT: Rethinking Test-Time Training as Composable Modules
Bohao Tang, Zhen Qin, Yuqi Pan +3
Test-time training (TTT) views sequence modeling as an online learning problem in which fast weights are updated by an internal learning rule. Despite the growing number of TTT var…
Beyond Transformers: Linear Attention Policy for Open-Vocabulary Object Goal Navigation
Jiahong Zhang, Yifan Lin, Yandong Zhang +6
Open-Vocabulary Object Goal Navigation (OVON) requires agents to operate under partial observability, making effective internal state updates critical for navigation performance. T…
SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference
Yuqi Pan, Jinghao Zhuang, Yupeng Feng +16
Scaling context length is reshaping large-model development, yet full-attention Transformers suffer from prohibitive computation and inference bottlenecks at long sequences. A key…
Parallel Training in Spiking Neural Networks
Yanbin Huang, Man Yao, Yuqi Pan +5
The bio-inspired integrate-fire-reset mechanism of spiking neurons constitutes the foundation for efficient processing in Spiking Neural Networks (SNNs). Recent progress in large m…
Adaptive Hopfield Network: Rethinking Similarities in Associative Memory
Shurong Wang, Yuqi Pan, Zhuoyang Shen +3
Associative memory models are content-addressable memory systems fundamental to biological intelligence and are notable for their high interpretability. However, existing models ev…
SpikingBrain: Spiking Brain-inspired Large Models
Yuqi Pan, Yupeng Feng, Jinghao Zhuang +16
Mainstream Transformer-based large language models face major efficiency bottlenecks: training computation scales quadratically with sequence length, and inference memory grows lin…
A Survey on Latent Reasoning
Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng +30
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate s…
Scaling Linear Attention with Sparse State Expansion
Yuqi Pan, Yongqi An, Zheng Li +6
The Transformer architecture, despite its widespread success, struggles with long-context scenarios due to quadratic computation and linear memory growth. While various linear atte…
A Systematic Analysis of Hybrid Linear Attention
Dustin Wang, Rui-Jie Zhu, Steven Abreu +9
Transformers face quadratic complexity and memory issues with long sequences, prompting the adoption of linear attention mechanisms using fixed-size hidden states. However, linear…
SVL: Empowering Spiking Neural Networks for Efficient 3D Open-World Understanding
Xuerui Qiu, Peixi Wu, Yaozhi Wen +5
Spiking Neural Networks (SNNs) provide an energy-efficient way to extract 3D spatio-temporal features. However, existing SNNs still exhibit a significant performance gap compared t…
MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map
Yuhong Chou, Man Yao, Kexin Wang +7
Various linear complexity models, such as Linear Transformer (LinFormer), State Space Model (SSM), and Linear RNN (LinRNN), have been proposed to replace the conventional softmax a…