10 citations · 10 across the 3 of their papers we have counts for
3 papers · 1 filter
Scaling Properties of Same-Family On-Policy Distillation
Yuntai Bao, Qinfeng Li, Guoqing Jiang +5
*Reinforcement learning (RL)* can induce substantial reasoning capabilities in large language models (LLMs), but how much of this capability transfers across model scales, and how…
Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR)
Guo-qing Jiang, Jinlong Liu, Zixiang Ding +2
As models for nature language processing (NLP), computer vision (CV) and recommendation systems (RS) require surging computation, a large number of GPUs/TPUs are paralleled as a la…
Understanding Why Neural Networks Generalize Well Through GSNR of Parameters
Jinlong Liu, Guoqing Jiang, Yunzhi Bai +2
As deep neural networks (DNNs) achieve tremendous success across many application domains, researchers tried to explore in many aspects on why they generalize well. In this paper,…