6 papers · 1 filter
Loop the Loopies!
Zitian Gao, Yilong Chen, Yihao Xiao +4
We present the Loopie series, consisting of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6B-parameter model with 0.6B active parameter…
ClawGym: A Scalable Framework for Building Effective Claw Agents
Fei Bai, Huatong Song, Shuang Sun +11
Claw-style environments support multi-step workflows over local files, tools, and persistent workspace states. However, scalable development around these environments remains const…
Beyond N-gram: Data-Aware X-GRAM Extraction for Efficient Embedding Parameter Scaling
Yilong Chen, Yanxi Xie, Zitian Gao +10
Large token-indexed lookup tables provide a compute-decoupled scaling path, but their practical gains are often limited by poor parameter efficiency and rapid memory growth. We att…
What Makes Diffusion Language Models Super Data Learners?
Zitian Gao, Haoming Luo, Lynx Chen +4
Recent studies have shown that diffusion language models achieve remarkable data efficiency under limited-data constraints, yet the underlying mechanisms remain unclear. In this wo…
One-shot Entropy Minimization
Zitian Gao, Lynx Chen, Haoming Luo +2
We trained 13,440 large language models and found that entropy minimization requires only a single unlabeled data and 10 steps optimization to achieve performance improvements comp…
Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
Tian Xie, Zitian Gao, Qingnan Ren +7
Inspired by the success of DeepSeek-R1, we explore the potential of rule-based reinforcement learning (RL) in large reasoning models. To analyze reasoning dynamics, we use syntheti…