2 citations · 2 across the 17 of their papers we have counts for
4 papers · 1 filter
Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts
Di He, Pengxiang Li, Da Chang +3
Looped Transformers introduce recurrent depth as a new scaling axis for LLMs: by repeatedly applying shared Transformer blocks, they increase effective depth without increasing par…
DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation
Jordan Painter, Dipankar Srirag, Adarsh Kappiyath +3
Large language models increasingly understand dialectal English, yet still produce only standard, US-leaning English, leaving dialectal generation, the harder half of the problem,…
Progressive Residual Warmup for Language Model Pretraining
Tianhao Chen, Xin Xu, Lu Yin +4
Transformer architectures serve as the backbone for most modern Large Language Models, therefore their pretraining stability and convergence speed are of central concern. Motivated…
Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
Pengxiang Li, Dilxat Muhtar, Tianlong Chen +2
Diffusion Language Models (DLMs) are often advertised as enabling parallel token generation, yet practical fast DLMs frequently converge to left-to-right, autoregressive (AR)-like…