41 citations · 71 across the 14 of their papers we have counts for
3 papers · 1 filter
The representation landscape of few-shot learning and fine-tuning in large language models
Diego Doimo, Alessandro Serra, Alessio Ansuini +1
In-context learning (ICL) and supervised fine-tuning (SFT) are two common strategies for improving the performance of modern large language models (LLMs) on specific tasks. Despite…
Emergence of a High-Dimensional Abstraction Phase in Language Transformers
Emily Cheng, Diego Doimo, Corentin Kervadec +4
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate…
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
Francesco Ortu, Zhijing Jin, Diego Doimo +3
Interpretability research aims to bridge the gap between empirical success and our scientific understanding of the inner workings of large language models (LLMs). However, most exi…