most citedMeta-prediction Model for Distillation-Aware NAS on Unseen Datasets

2 citations · 3 across the 3 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2023

Co-training and Co-distillation for Quality Improvement and Compression of Language Models

Hayeon Lee, Rui Hou, Jongpil Kim +4

Knowledge Distillation (KD) compresses computationally expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models, allowing their use in resourc…

cs.CL2023★ 1 cited

A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models

Hayeon Lee, Rui Hou, Jongpil Kim +3

Distillation from Weak Teacher (DWT) is a method of transferring knowledge from a smaller, weaker teacher model to a larger student model to improve its performance. Previous studi…