90 citations · 112 across the 12 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.CL2023★ 1 cited
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model
Shicheng Tan, Weng Lam Tam, Yuanchun Wang +9
Currently, the reduction in the parameter scale of large-scale pre-trained language models (PLMs) through knowledge distillation has greatly facilitated their widespread deployment…
cs.CL2023
Task-agnostic Distillation of Encoder-Decoder Language Models
Chen Zhang, Yang Yang, Jingang Wang +1
Finetuning pretrained language models (LMs) have enabled appealing performance on a diverse array of tasks. The intriguing task-agnostic property has driven a shifted focus from ta…
cs.CL2023★ 1 cited
Lifting the Curse of Capacity Gap in Distilling Language Models
Chen Zhang, Yang Yang, Jiahao Liu +4
Pretrained language models (LMs) have shown compelling performance on various downstream tasks, but unfortunately they require a tremendous amount of inference compute. Knowledge d…