BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillation

Minchong Li, Feng Zhou, Xiaohui Song

In recent years, large language models (LLMs) have shown exceptional capabilities across various natural language processing (NLP) tasks. However, such impressive performance often…