Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data

Juanhui Li, Sreyashi Nag, Hui Liu +7

In real-world NLP applications, Large Language Models (LLMs) offer promising solutions due to their extensive training on vast datasets. However, the large size and high computatio…