8 citations · 8 across the 6 of their papers we have counts for
3 papers · 1 filter
Exact Learning of Arithmetic with Differentiable Agents
Hristo Papazov, Francesco D'Angelo, Nicolas Flammarion
We explore the possibility of exact algorithmic learning with gradient-based methods and introduce a differentiable framework capable of strong length generalization on arithmetic…
Selective Induction Heads: How Transformers Select Causal Structures In Context
Francesco D'Angelo, Francesco Croce, Nicolas Flammarion
Transformers have exhibited exceptional capabilities in sequence modeling tasks, leveraging self-attention and in-context learning. Critical to this success are induction heads, at…
The emergence of sparse attention: impact of data distribution and benefits of repetition
Nicolas Zucchet, Francesco d'Angelo, Andrew K. Lampinen +1
Emergence is a fascinating property of large language models and neural networks more broadly: as models scale and train for longer, they sometimes develop new abilities in sudden…