6 papers
A Tropical Geometry View of Forgetting: A Per-Unit Projector for Knowledge-Preserving Fine-Tuning
Yuyang Zhang, Xiaoyin Chen, Chunlin Ren +1
Fine-tuning a language model on new text degrades what it already does. Replay-free projectors such as Adam-NSCL and GPM forbid one shared subspace of a layer's inputs in every row…
Bayesian Symbolic Regression with Entropic Reinforcement Learning
Oussama Boussif, Mohammed Mahfoud, Younesse Kaddar +6
Symbolic regression is the problem of finding an algebraic expression describing a stochastic dependence of a target variable on a set of inputs. Unlike forms of regression that fi…
A Measure-Theoretic Analysis of Reasoning in Transformers: Structural Generalization and Approximation Limits
Yuyang Zhang, Yifu Zhang, Xuehai Zhou +1
Scaling laws for reasoning in large language models (LLMs) are well documented, but little theory explains their out-of-distribution (OOD) generalization. We formalize reasoning wi…
Learning to Self-Evolve
Xiaoyin Chen, Canwen Xu, Yite Wang +3
We introduce Learning to Self-Evolve (LSE), a reinforcement learning framework that trains large language models (LLMs) to improve their own contexts at test time. We situate LSE i…
When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Training
Sanxing Chen, Xiaoyin Chen, Yukun Huang +2
While Large Language Models (LLMs) hold promise to become autonomous agents, they often explore suboptimally in sequential decision-making. Recent work has sought to enhance this c…
How Much Backtracking is Enough? Exploring the Interplay of SFT and RL in Enhancing LLM Reasoning
Hongyi James Cai, Junlin Wang, Xiaoyin Chen +1
Recent advancements in large language models (LLMs) suggest that reinforcement learning (RL) effectively internalizes search strategies, yielding significant improvements on challe…