3.1k citations · 3.2k across the 3 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.CL2023★ 13 cited
Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling
Sanchit Gandhi, Patrick von Platen, Alexander M. Rush
As the size of pre-trained speech recognition models increases, running these large models in low-latency or resource-constrained environments becomes challenging. In this work, we…
cs.LG2023★ 56 cited
Zephyr: Direct Distillation of LM Alignment
Lewis Tunstall, Edward Beeching, Nathan Lambert +11
We aim to produce a smaller language model that is aligned to user intent. Previous research has shown that applying distilled supervised fine-tuning (dSFT) on larger models signif…