977 citations · 1.1k across the 6 of their papers we have counts for
6 papers
The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
Shangwen Sun, Alfredo Canziani, Yann LeCun +1
We study two recurring phenomena in Transformer language models: massive activations, in which a small number of tokens exhibit extreme outliers in a few channels, and attention si…
TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning
Jiachen Zhu, Rafael M. Moraes, Serkan Karakulak +3
We present Transformation Invariance and Covariance Contrast (TiCo) for self-supervised visual representation learning. Similar to other recent self-supervised learning methods, ou…
Separating the World and Ego Models for Self-Driving
Vlad Sobal, Alfredo Canziani, Nicolas Carion +2
Training self-driving systems to be robust to the long-tail of driving scenarios is a critical problem. Model-based approaches leverage simulation to emulate a wide range of scenar…
Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic
Mikael Henaff, Alfredo Canziani, Yann LeCun
Learning a policy using only observational data is challenging because the distribution of states it induces at execution time may differ from the distribution observed during trai…
CortexNet: a Generic Network Family for Robust Visual Temporal Representations
Alfredo Canziani, Eugenio Culurciello
In the past five years we have observed the rise of incredibly well performing feed-forward neural networks trained supervisedly for vision related tasks. These models have achieve…
An Analysis of Deep Neural Network Models for Practical Applications
Alfredo Canziani, Adam Paszke, Eugenio Culurciello
Since the emergence of Deep Neural Networks (DNNs) as a prominent technique in the field of computer vision, the ImageNet classification challenge has played a major role in advanc…