2 citations · 3 across the 3 of their papers we have counts for
1 paper · 1 filter
Zixiang Ding, Guoqing Jiang, Shuai Zhang +2
In this paper, we propose Stochastic Knowledge Distillation (SKD) to obtain compact BERT-style language model dubbed SKDBERT. In each iteration, SKD samples a teacher model from a…