4 papers · 1 filter
Modeling Human-Like Color Naming Behavior in Context
Yuqing Zhang, Ecesu Ürker, Tessa Verhoef +2
Modeling the emergence of human-like lexicons in computational systems has advanced through the use of interacting neural agents, which simulate both learning and communicative pre…
Tracing Computation Density in LLMs
Corentin Kervadec, Iuliia Lysova, Iuri Macocco +2
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs, but it is not clear that they exploit their f…
Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs
Corentin Kervadec, Iuliia Lysova, Marco Baroni +1
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs. Several studies on LLM efficiency optimizatio…
The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models
Nora Graichen, Iria de-Dios-Flores, Gemma Boleda
We present a systematic review of 337 articles evaluating the syntactic abilities of Transformer-based language models (TLMs), reporting on over 3,000 datapoints spanning a wide ra…