1 citations · 1 across the 8 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation
Huanxin Sheng, Zhiling Ye, Haonan Wang +3
Sparse on-policy distillation (OPD) allocates teacher supervision to a small subset of tokens in student-generated trajectories. However, useful teacher guidance can yield a noisy…
cs.LG2025
Learning to Align, Aligning to Learn: A Unified Approach for Self-Optimized Alignment
Haowen Wang, Yun Yue, Zhiling Ye +12
Alignment methodologies have emerged as a critical pathway for enhancing language model alignment capabilities. While SFT (supervised fine-tuning) accelerates convergence through d…
cs.LG2023
AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix
Yun Yue, Zhiling Ye, Jiadi Jiang +2
Adaptive optimizers, such as Adam, have achieved remarkable success in deep learning. A key component of these optimizers is the so-called preconditioning matrix, providing enhance…