22 citations · 107 across the 37 of their papers we have counts for
4 papers · 2 filters
The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
Shangwen Sun, Alfredo Canziani, Yann LeCun +1
We study two recurring phenomena in Transformer language models: massive activations, in which a small number of tokens exhibit extreme outliers in a few channels, and attention si…
AI+HW 2035: Shaping the Next Decade
Deming Chen, Jason Cong, Azalia Mirhoseini +27
Artificial intelligence (AI) and hardware (HW) are advancing at unprecedented rates, yet their trajectories have become inseparably intertwined. The global research community lacks…
AI Must Embrace Specialization via Superhuman Adaptable Intelligence
Judah Goldfeder, Philippe Wyder, Yann LeCun +1
Everyone from AI executives and researchers to doomsayers, politicians, and activists is talking about Artificial General Intelligence (AGI). Yet, they often don't seem to agree on…
Learning Latent Action World Models In The Wild
Quentin Garrido, Tushar Nagarajan, Basile Terver +3
Agents capable of reasoning and planning in the real world require the ability of predicting the consequences of their actions. While world models possess this capability, they mos…