565 citations · 2.5k across the 94 of their papers we have counts for
6 papers · 1 filter
LIFT: Scoped Constraints for Evaluating and Training Instruction Following
Eran Goldstein, Yuval Belfer, Yonatan Belinkov +1
Common instruction following (IF) benchmarks, such as IFEval, evaluate the ability of language models (LMs) to follow simple constraints that can easily be verified programmaticall…
Feeding BabyLMs Macaroni: Code-Switching Curricula Cause Cross-Lingual Convergence
Dries Rooryck, Alex Cai, Yonatan Belinkov +2
Children in multilingual communities often code-switch, using multiple languages in a single utterance. Can we induce cross-lingual alignment in language models by training on code…
Decomposing Query-Key Feature Interactions Using Contrastive Covariances
Andrew Lee, Yonatan Belinkov, Fernanda Viégas +1
Despite the central role of attention heads in Transformers, we lack tools to understand why a model attends to a particular token. To address this, we study the query-key (QK) spa…
Investigating the Development of Task-Oriented Communication in Vision-Language Models
Boaz Carmeli, Orr Paradise, Shafi Goldwasser +2
We investigate whether \emph{LLM-based agents} can develop task-oriented communication protocols that differ from standard natural language in collaborative reasoning tasks. Our fo…
CtD: Composition through Decomposition in Emergent Communication
Boaz Carmeli, Ron Meir, Yonatan Belinkov
Compositionality is a cognitive mechanism that allows humans to systematically combine known concepts in novel ways. This study demonstrates how artificial neural agents acquire an…
Will it Merge? On The Causes of Model Mergeability
Adir Rahamim, Asaf Yehudai, Boaz Carmeli +3
Model merging has emerged as a promising technique for combining multiple fine-tuned models into a single multitask model without retraining. However, the factors that determine wh…