Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
OC-OPD: Observation-coordinated Supervision for On-Policy Distillation
Ruoran Xu, Yilun Zhao, Ziyu Chen +1
Reverse and forward Kullback--Leibler (KL) divergences provide complementary supervision in on-policy distillation: reverse KL refines the student's active token rankings, and forw…
cs.LG2026
Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization
Ruoran Xu, Borong She, Xiaobo Jin +1
Deep learning optimization relies heavily on the assumption of smooth loss landscapes, a condition systematically violated by modern architectures due to non-smooth components such…