Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Near-Policy: Accelerating On-Policy Distillation via Asynchronous Generation and Selective Packing
Miao Rang, Zhenni Bi, Hang Zhou +6
Standard knowledge distillation for autoregressive models often suffers from distribution mismatch. While on-policy methods mitigate this by leveraging student-generated outputs, t…
cs.LG2026
Physics-Guided Multimodal Transformers are the Necessary Foundation for the Next Generation of Meteorological Science
Jing Han, Hanting Chen, Kai Han +4
This position paper argues that the next generation of artificial intelligence in meteorological and climate sciences must transition from fragmented hybrid heuristics toward a uni…
cs.LG2025
ROOT: Robust Orthogonalized Optimizer for Neural Network Training
Wei He, Kai Han, Hang Zhou +4
The optimization of large language models (LLMs) remains a critical challenge, particularly as model scaling exacerbates sensitivity to algorithmic imprecision and training instabi…