2 papers
cs.CL2025
Circuit Distillation
Somin Wadhwa, Silvio Amir, Byron C. Wallace
Model distillation typically focuses on behavioral mimicry, where a student model is trained to replicate a teacher's output while treating its internal computations as a black box…
cs.CL2025
Who Taught You That? Tracing Teachers in Model Distillation
Somin Wadhwa, Chantal Shaib, Silvio Amir +1
Model distillation -- using outputs from a large teacher model to teach a small student model -- is a practical means of creating efficient models for a particular task. We ask: Ca…