36 citations · 78 across the 16 of their papers we have counts for
16 papers
A Language Model's Guide Through Latent Space
Dimitri von Rütte, Sotiris Anagnostidis, Gregor Bachmann +1
Concept guidance has emerged as a cheap and simple way to control the behavior of language models by probing their hidden representations for concept vectors and using them to pert…
How Good is a Single Basin?
Kai Lion, Lorenzo Noci, Thomas Hofmann +1
The multi-modal nature of neural loss landscapes is often considered to be the main driver behind the empirical success of deep ensembles. In this work, we probe this belief by con…
Disentangling Linear Mode-Connectivity
Gul Sena Altintas, Gregor Bachmann, Lorenzo Noci +1
Linear mode-connectivity (LMC) (or lack thereof) is one of the intriguing characteristics of neural network loss landscapes. While empirically well established, it unfortunately st…
Navigating Scaling Laws: Compute Optimality in Adaptive Model Training
Sotiris Anagnostidis, Gregor Bachmann, Imanol Schlag +1
In recent years, the state-of-the-art in deep learning has been dominated by very large models that have been pre-trained on vast amounts of data. The paradigm is very simple: inve…
Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes
Alexandros Delitzas, Maria Parelli, Nikolas Hars +4
Training models to apply common-sense linguistic knowledge and visual concepts from 2D images to 3D scene understanding is a promising direction that researchers have only recently…
Scaling MLPs: A Tale of Inductive Bias
Gregor Bachmann, Sotiris Anagnostidis, Thomas Hofmann
In this work we revisit the most fundamental building block in deep learning, the multi-layer perceptron (MLP), and study the limits of its performance on vision tasks. Empirical i…
CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes
Maria Parelli, Alexandros Delitzas, Nikolas Hars +4
Training models to apply linguistic knowledge and visual concepts from 2D images to 3D world understanding is a promising direction that researchers have only recently started to e…
Random Teachers are Good Teachers
Felix Sarnthein, Gregor Bachmann, Sotiris Anagnostidis +1
In this work, we investigate the implicit regularization induced by teacher-student learning dynamics in self-distillation. To isolate its effect, we describe a simple experiment w…
The Curious Case of Benign Memorization
Sotiris Anagnostidis, Gregor Bachmann, Lorenzo Noci +1
Despite the empirical advances of deep learning across a variety of learning tasks, our theoretical understanding of its success is still very restricted. One of the key challenges…
How Tempering Fixes Data Augmentation in Bayesian Neural Networks
Gregor Bachmann, Lorenzo Noci, Thomas Hofmann
While Bayesian neural networks (BNNs) provide a sound and principled alternative to standard neural networks, an artificial sharpening of the posterior usually needs to be applied…
Generalization Through The Lens Of Leave-One-Out Error
Gregor Bachmann, Thomas Hofmann, Aurélien Lucchi
Despite the tremendous empirical success of deep learning models to solve various learning tasks, our theoretical understanding of their generalization ability is very limited. Cla…
Analytic Insights into Structure and Rank of Neural Network Hessian Maps
Sidak Pal Singh, Gregor Bachmann, Thomas Hofmann
The Hessian of a neural network captures parameter interactions through second-order derivatives of the loss. It is a fundamental object of study, closely tied to various problems…
Precise characterization of the prior predictive distribution of deep ReLU networks
Lorenzo Noci, Gregor Bachmann, Kevin Roth +2
Recent works on Bayesian neural networks (BNNs) have highlighted the need to better understand the implications of using Gaussian priors in combination with the compositional struc…
Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effect
Lorenzo Noci, Kevin Roth, Gregor Bachmann +2
The "cold posterior effect" (CPE) in Bayesian deep learning describes the uncomforting observation that the predictive performance of Bayesian neural networks can be significantly…
Uniform Convergence, Adversarial Spheres and a Simple Remedy
Gregor Bachmann, Seyed-Mohsen Moosavi-Dezfooli, Thomas Hofmann
Previous work has cast doubt on the general framework of uniform convergence and its ability to explain generalization in neural networks. By considering a specific dataset, it was…
Constant Curvature Graph Convolutional Networks
Gregor Bachmann, Gary Bécigneul, Octavian-Eugen Ganea
Interest has been rising lately towards methods representing data in non-Euclidean spaces, e.g. hyperbolic or spherical, that provide specific inductive biases useful for certain r…