most citedSuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules

1 citations · 1 across the 3 of their papers we have counts for

collaborators
Showing cs.LGShow all

1 paper · 1 filter

cs.LG2026

The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works

Guanghui Wang, Kaiwen Lv Kacuila, Zhiyong Yang +5

Knowledge distillation (KD) transfers knowledge from a large teacher model to a smaller student. In language modeling, the student is trained either on tokens sampled from the teac…