3 papers
cs.CL2026
Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces
Debjyoti Saha Roy, Byron C. Wallace, Javed A. Aslam
Modern reasoning models offer surprisingly strong zero-shot performance on challenging multi-label tasks that require selecting a small set of relevant options from hundreds of tho…
cs.CL2025
Circuit Distillation
Somin Wadhwa, Silvio Amir, Byron C. Wallace
Model distillation typically focuses on behavioral mimicry, where a student model is trained to replicate a teacher's output while treating its internal computations as a black box…
cs.CL2025
Who Taught You That? Tracing Teachers in Model Distillation
Somin Wadhwa, Chantal Shaib, Silvio Amir +1
Model distillation -- using outputs from a large teacher model to teach a small student model -- is a practical means of creating efficient models for a particular task. We ask: Ca…