3 papers
cs.LG2026
SOLO: Pretraining Billion-Parameter Language Models with Shared-Output Local Learning
Bojian Yin, Shurong Wang, Yuqi Pan +1
Large language models are trained with backpropagation, whose global gradient coordinates all layers but forces each to hold its activations and wait for the gradient to pass back…
cs.LG2026
Efficient Sparse Selective-Update RNNs for Long-Range Sequence Modeling
Bojian Yin, Shurong Wang, Haoyu Tan +3
Real-world sequential signals, such as audio or video, contain critical information that is often embedded within long periods of silence or noise. While recurrent neural networks…
cs.LG2025
SpikingBrain: Spiking Brain-inspired Large Models
Yuqi Pan, Yupeng Feng, Jinghao Zhuang +16
Mainstream Transformer-based large language models face major efficiency bottlenecks: training computation scales quadratically with sequence length, and inference memory grows lin…