3 papers
cs.LG2026
Linear RNN Scaling Laws: When Longer Sequences Beat More Sequences
Ziyan Chen, Zhongzhu Zhou, Peilin Liu +2
Empirical scaling laws for autoregressive language models relate prediction loss to model size, data size, and optimization compute, but their theoretical origin is still poorly un…
cs.LG2026
Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization
Peilin Liu, Ding-Xuan Zhou
Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning.…
stat.ML2026
Generalization Analysis of Transformers in Distribution Regression
Peilin Liu, Ding-Xuan Zhou
In recent years, models based on the Transformer architecture have seen widespread applications and have become one of the core tools in the field of deep learning. Numerous succes…