9 papers
Knowledge Offloading: Decomposing LLMs into Sparse Backbones and Memory Modules
Karim Galliamov, Rochelle Choenni, Ivan Titov
LLMs encode both general capabilities and domain-specific knowledge in a single set of parameters. We ask whether this capacity can be reorganized: keeping broadly useful computati…
Finding Culture-Sensitive Neurons in Vision-Language Models
Xiutian Zhao, Rochelle Choenni, Rohit Saxena +1
Despite their impressive performance, vision-language models (VLMs) still struggle on culturally situated inputs. To understand how VLMs process culturally grounded information, we…
Best-of-L: Cross-Lingual Reward Modeling for Mathematical Reasoning
Sara Rajaee, Rochelle Choenni, Ekaterina Shutova +1
While the reasoning abilities of large language models (LLMs) continue to advance, it remains unclear how such ability varies across languages in multilingual LLMs and whether diff…
Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning
Rochelle Choenni, Ekaterina Shutova
Improving the alignment of Large Language Models (LLMs) with respect to the cultural values that they encode has become an increasingly important topic. In this work, we study whet…
M-Wanda: Improving One-Shot Pruning for Multilingual LLMs
Rochelle Choenni, Ivan Titov
Multilingual LLM performance is often critically dependent on model size. With an eye on efficiency, this has led to a surge in interest in one-shot pruning methods that retain the…
Local Contrastive Editing of Gender Stereotypes
Marlene Lutz, Rochelle Choenni, Markus Strohmaier +1
Stereotypical bias encoded in language models (LMs) poses a threat to safe language technology, yet our understanding of how bias manifests in the parameters of LMs remains incompl…