5 papers
Adaptive Layerwise Perturbation: Unifying Off-Policy Corrections for LLM RL
Chenlu Ye, Xuanchang Zhang, Yifan Hao +6
Off-policy problems such as policy staleness and training--inference mismatch have become a major bottleneck for training stability and further exploration in LLM RL. The distribut…
Towards Better Generalization via Distributional Input Projection Network
Yifan Hao, Yanxin Lu, Hanning Zhang +2
As overparameterized models become increasingly prevalent, training loss alone offers limited insight into generalization performance. While smoothness has been linked to improved…
Transformers as Multi-task Learners: Decoupling Features in Hidden Markov Models
Yifan Hao, Chenlu Ye, Chi Han +1
Transformer based models have shown remarkable capabilities in sequence learning across a wide range of tasks, often performing well on specific task by leveraging input-output exa…
Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methods
Yifan Hao, Xingyuan Pan, Hanning Zhang +3
Supervised fine-tuning (SFT) on domain-specific data is the dominant approach for adapting foundation models to specialized tasks. However, it has been observed that SFT models ten…
Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RL
Jiarui Yao, Yifan Hao, Hanning Zhang +4
Chain-of-thought (CoT) reasoning in large language models (LLMs) can be formalized as a latent variable problem, where the model needs to generate intermediate reasoning steps. Whi…