From the 1 of 19 linked papers with an AI index.
1 citations · 1 across the 1 of their papers we have counts for
19 papers
Growing a Tail: Increasing Output Diversity in Large Language Models
Michal Shur-Ofry, Bar Horowitz-Amsalem, Adir Rahamim +1
The paper investigates how narrowly large language models generate answers compared to the broader range of human responses, and shows that increasing temperature, using diverse pr…
Decomposing Query-Key Feature Interactions Using Contrastive Covariances
Andrew Lee, Yonatan Belinkov, Fernanda Viégas +1
Despite the central role of attention heads in Transformers, we lack tools to understand why a model attends to a particular token. To address this, we study the query-key (QK) spa…
Investigating the Development of Task-Oriented Communication in Vision-Language Models
Boaz Carmeli, Orr Paradise, Shafi Goldwasser +2
We investigate whether \emph{LLM-based agents} can develop task-oriented communication protocols that differ from standard natural language in collaborative reasoning tasks. Our fo…
CtD: Composition through Decomposition in Emergent Communication
Boaz Carmeli, Ron Meir, Yonatan Belinkov
Compositionality is a cognitive mechanism that allows humans to systematically combine known concepts in novel ways. This study demonstrates how artificial neural agents acquire an…
Will it Merge? On The Causes of Model Mergeability
Adir Rahamim, Asaf Yehudai, Boaz Carmeli +3
Model merging has emerged as a promising technique for combining multiple fine-tuned models into a single multitask model without retraining. However, the factors that determine wh…
Structured RAG for Answering Aggregative Questions
Omri Koshorek, Niv Granot, Aviv Alloni +6
Retrieval-Augmented Generation (RAG) has become the dominant approach for answering questions over large corpora. However, current datasets and methods are highly focused on cases…