2 papers
cs.CL2026
HRM-Text: Efficient Pretraining Beyond Scaling
Guan Wang, Changling Liu, Chenyu Wang +6
The current pretraining paradigm for large language models relies on massive compute and internet-scale raw text, creating a significant barrier to foundational research. In contra…
cs.CL2026
Why Attend to Everything? Focus is the Key
Hengshuai Yao, Xing Chen, Ahmed Murtadha +8
Standard attention scales quadratically with sequence length. Efficient attention methods reduce this O(n^2) cost, but when retrofitted into pretrained models, they often degrade p…