Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Loop the Loopies!
Zitian Gao, Yilong Chen, Yihao Xiao +4
We present Loopie, the most powerful open-source looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active…
cs.CL2026
Beyond N-gram: Data-Aware X-GRAM Extraction for Efficient Embedding Parameter Scaling
Yilong Chen, Yanxi Xie, Zitian Gao +10
Large token-indexed lookup tables provide a compute-decoupled scaling path, but their practical gains are often limited by poor parameter efficiency and rapid memory growth. We att…