6 papers
Atomic Skills are the Prerequisite: When Reinforcement Learning Synthesizes Compositional Reasoning, and When It Only Amplifies
Sitao Cheng, Xunjian Yin, Ruiwen Zhou +5
Does Reinforcement Learning (RL) merely amplify existing skills, or synthesize novel skills? We investigate this question through the lens of Complementary Reasoning: the critical…
LEDOM: Reverse Language Model
Xunjian Yin, Sitao Cheng, Yuxi Xie +6
Autoregressive language models are trained exclusively left-to-right. We explore the complementary factorization, training right-to-left at scale, and ask what reasoning patterns e…
Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement
Xunjian Yin, Xinyi Wang, Liangming Pan +3
The rapid advancement of large language models (LLMs) has significantly enhanced the capabilities of AI-driven agents across various tasks. However, existing agentic systems, wheth…
Disentangling Memory and Reasoning Ability in Large Language Models
Mingyu Jin, Weidi Luo, Sitao Cheng +5
Large Language Models (LLMs) have demonstrated strong performance in handling complex tasks requiring both extensive knowledge and reasoning abilities. However, the existing LLM in…
Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
Xinyi Wang, Antonis Antoniades, Yanai Elazar +4
The impressive capabilities of large language models (LLMs) have sparked debate over whether these models genuinely generalize to unseen tasks or predominantly rely on memorizing v…
Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models
Sitao Cheng, Liangming Pan, Xunjian Yin +2
Large language models (LLMs) encode vast amounts of knowledge during pre-training (parametric knowledge, or PK) and can further be enhanced by incorporating contextual knowledge (C…