most citedImproving Generalization of Pre-trained Language Models via Stochastic Weight Averaging

1 citations · 2 across the 3 of their papers we have counts for

collaborators
Showing cs.CLShow all

1 paper · 1 filter

cs.CL2022★ 1 cited

Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging

Peng Lu, Ivan Kobyzev, Mehdi Rezagholizadeh +3

Knowledge Distillation (KD) is a commonly used technique for improving the generalization of compact Pre-trained Language Models (PLMs) on downstream tasks. However, such methods i…