3 papers
cs.CL2026
Attention to Mamba: A Recipe for Cross-Architecture Distillation
Abhinav Moudgil, Ningyuan Huang, Eeshan Gunesh Dhekane +3
State Space Models (SSMs) such as Mamba have become a popular alternative to Transformer models, due to their reduced memory consumption and higher throughput at generation compare…
cs.LG2024
Interpreting CLIP: Insights on the Robustness to ImageNet Distribution Shifts
Jonathan Crabbé, Pau RodrÃguez, Vaishaal Shankar +2
What distinguishes robust models from non-robust ones? While for ImageNet distribution shifts it has been shown that such differences in robustness can be traced back predominantly…
cs.CL2024
Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models
Xavier Suau, Pieter Delobelle, Katherine Metcalf +4
An important issue with Large Language Models (LLMs) is their undesired ability to generate toxic language. In this work, we show that the neurons responsible for toxicity can be d…