14 citations · 14 across the 6 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
Yuanyi Wang, Su Lu, Yanggan Gu +6
On-policy distillation (OPD) trains a student on its own rollouts with token-level teacher supervision. Recent selective OPD methods exploit the non-uniformity of OPD signals by pr…
cs.LG2022
Selective Cross-Task Distillation
Su Lu, Han-Jia Ye, De-Chuan Zhan
The outpouring of various pre-trained models empowers knowledge distillation by providing abundant teacher resources, but there lacks a developed mechanism to utilize these teacher…
cs.LG2021
Towards Enabling Meta-Learning from Target Models
Su Lu, Han-Jia Ye, Le Gan +1
Meta-learning can extract an inductive bias from previous learning experience and assist the training of new tasks. It is often realized through optimizing a meta-model with the ev…