6 papers
Circuit Distillation
Somin Wadhwa, Silvio Amir, Byron C. Wallace
Model distillation typically focuses on behavioral mimicry, where a student model is trained to replicate a teacher's output while treating its internal computations as a black box…
Who Taught You That? Tracing Teachers in Model Distillation
Somin Wadhwa, Chantal Shaib, Silvio Amir +1
Model distillation -- using outputs from a large teacher model to teach a small student model -- is a practical means of creating efficient models for a particular task. We ask: Ca…
Investigating Mysteries of CoT-Augmented Distillation
Somin Wadhwa, Silvio Amir, Byron C. Wallace
Eliciting "chain of thought" (CoT) rationales -- sequences of token that convey a "reasoning" process -- has been shown to consistently improve LLM performance on tasks like questi…
Learning from Natural Language Explanations for Generalizable Entity Matching
Somin Wadhwa, Adit Krishnan, Runhui Wang +2
Entity matching is the task of linking records from different sources that refer to the same real-world entity. Past work has primarily treated entity linking as a standard supervi…
Revisiting Relation Extraction in the era of Large Language Models
Somin Wadhwa, Silvio Amir, Byron C. Wallace
Relation extraction (RE) is the core NLP task of inferring semantic relationships between entities from text. Standard supervised RE techniques entail training modules to tag token…
Distilling Event Sequence Knowledge From Large Language Models
Somin Wadhwa, Oktie Hassanzadeh, Debarun Bhattacharjya +2
Event sequence models have been found to be highly effective in the analysis and prediction of events. Building such models requires availability of abundant high-quality event seq…