2.1k citations · 5.2k across the 53 of their papers we have counts for
6 papers · 1 filter
Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes
Alexandros Delitzas, Maria Parelli, Nikolas Hars +4
Training models to apply common-sense linguistic knowledge and visual concepts from 2D images to 3D scene understanding is a promising direction that researchers have only recently…
Scaling MLPs: A Tale of Inductive Bias
Gregor Bachmann, Sotiris Anagnostidis, Thomas Hofmann
In this work we revisit the most fundamental building block in deep learning, the multi-layer perceptron (MLP), and study the limits of its performance on vision tasks. Empirical i…
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
Sotiris Anagnostidis, Dario Pavllo, Luca Biggio +3
Autoregressive Transformers adopted in Large Language Models (LLMs) are hard to scale to long sequences. Despite several works trying to reduce their computational cost, most of LL…
CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes
Maria Parelli, Alexandros Delitzas, Nikolas Hars +4
Training models to apply linguistic knowledge and visual concepts from 2D images to 3D world understanding is a promising direction that researchers have only recently started to e…
Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning
Sanghwan Kim, Lorenzo Noci, Antonio Orvieto +1
In contrast to the natural capabilities of humans to learn new tasks in a sequential fashion, neural networks are known to suffer from catastrophic forgetting, where the model's pe…
Random Teachers are Good Teachers
Felix Sarnthein, Gregor Bachmann, Sotiris Anagnostidis +1
In this work, we investigate the implicit regularization induced by teacher-student learning dynamics in self-distillation. To isolate its effect, we describe a simple experiment w…