5 papers
Loop the Loopies!
Zitian Gao, Yilong Chen, Yihao Xiao +4
We present the Loopie series, consisting of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6B-parameter model with 0.6B active parameter…
Universal Reasoning Model
Zitian Gao, Lynx Chen, Yihao Xiao +5
Universal transformers (UTs) have been widely used for complex reasoning tasks such as ARC-AGI and Sudoku, yet the specific sources of their performance gains remain underexplored.…
What Makes Diffusion Language Models Super Data Learners?
Zitian Gao, Haoming Luo, Lynx Chen +4
Recent studies have shown that diffusion language models achieve remarkable data efficiency under limited-data constraints, yet the underlying mechanisms remain unclear. In this wo…
One-shot Entropy Minimization
Zitian Gao, Lynx Chen, Haoming Luo +2
We trained 13,440 large language models and found that entropy minimization requires only a single unlabeled data and 10 steps optimization to achieve performance improvements comp…
Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
Tian Xie, Zitian Gao, Qingnan Ren +7
Inspired by the success of DeepSeek-R1, we explore the potential of rule-based reinforcement learning (RL) in large reasoning models. To analyze reasoning dynamics, we use syntheti…