2 citations · 2 across the 6 of their papers we have counts for
6 papers · 1 filter
Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces
Debjyoti Saha Roy, Byron C. Wallace, Javed A. Aslam
Reasoning-trained language models can select, zero-shot, a small set of relevant labels from thousands to hundreds of thousands of candidates. We ask how they do it mechanistically…
Circuit Distillation
Somin Wadhwa, Silvio Amir, Byron C. Wallace
Model distillation typically focuses on behavioral mimicry, where a student model is trained to replicate a teacher's output while treating its internal computations as a black box…
Who Taught You That? Tracing Teachers in Model Distillation
Somin Wadhwa, Chantal Shaib, Silvio Amir +1
Model distillation -- using outputs from a large teacher model to teach a small student model -- is a practical means of creating efficient models for a particular task. We ask: Ca…
Investigating Mysteries of CoT-Augmented Distillation
Somin Wadhwa, Silvio Amir, Byron C. Wallace
Eliciting "chain of thought" (CoT) rationales -- sequences of token that convey a "reasoning" process -- has been shown to consistently improve LLM performance on tasks like questi…
Learning from Natural Language Explanations for Generalizable Entity Matching
Somin Wadhwa, Adit Krishnan, Runhui Wang +2
Entity matching is the task of linking records from different sources that refer to the same real-world entity. Past work has primarily treated entity linking as a standard supervi…
On-the-fly Definition Augmentation of LLMs for Biomedical NER
Monica Munnangi, Sergey Feldman, Byron C Wallace +3
Despite their general capabilities, LLMs still struggle on biomedical NER tasks, which are difficult due to the presence of specialized terminology and lack of training data. In th…