Publications (33)
A fast summation method for the DFT-D3 dispersion correction
Victoria Valeeva, Cheuk Hin Ho, Mario Geiger +4
The paper introduces FourierD3, a low‑rank decomposition technique that restores separability in the DFT‑D3 dispersion correction, enabling fast particle‑mesh evaluation in O(N log…
Symphony: Symmetry-Equivariant Point-Centered Spherical Harmonics for 3D Molecule Generation
Ameya Daigavane, Song Kim, Mario Geiger +1
We present Symphony, an -equivariant autoregressive generative model for 3D molecular geometries that iteratively builds a molecule from molecular fragments. Existing autoreg…
Ophiuchus: Scalable Modeling of Protein Structures through Hierarchical Coarse-graining SO(3)-Equivariant Autoencoders
Allan dos Santos Costa, Ilan Mitnikov, Mario Geiger +3
Three-dimensional native states of natural proteins display recurring and hierarchical patterns. Yet, traditional graph-based modeling of protein structures is often limited to ope…
Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks
Tess E. Smidt, Mario Geiger, Benjamin Kurt Miller
Curie's principle states that "when effects show certain asymmetry, this asymmetry must be found in the causes that gave rise to them". We demonstrate that symmetry equivariant neu…
Relative stability toward diffeomorphisms indicates performance in deep nets
Leonardo Petrini, Alessandro Favero, Mario Geiger +1
Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empi…
Cracking the Quantum Scaling Limit with Machine Learned Electron Densities
Joshua A. Rackers, Lucas Tecot, Mario Geiger +1
A long-standing goal of science is to accurately solve the Schrödinger equation for large molecular systems. The poor scaling of current quantum chemistry algorithms on classical…
Efficient Molecular Conformer Generation with SO(3)-Averaged Flow Matching and Reflow
Zhonglin Cao, Mario Geiger, Allan dos Santos Costa +6
Fast and accurate generation of molecular conformers is desired for downstream computational chemistry and drug discovery tasks. Currently, training and sampling state-of-the-art d…
Disentangling feature and lazy training in deep neural networks
Mario Geiger, Stefano Spigler, Arthur Jacot +1
Two distinct limits for deep learning have been derived as the network width , depending on how the weights of the last layer scale with . In the Neural Tan…
e3nn: Euclidean Neural Networks
Mario Geiger, Tess Smidt
We present e3nn, a generalized framework for creating E(3) equivariant trainable functions, also known as Euclidean neural networks. e3nn naturally operates on geometry and geometr…
Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)
Taco S. Cohen, Mario Geiger, Maurice Weiler
Group equivariant and steerable convolutional neural networks (regular and steerable G-CNNs) have recently emerged as a very effective model class for learning from signal data suc…
Scaling description of generalization with number of parameters in deep learning
Mario Geiger, Arthur Jacot, Stefano Spigler +6
Supervised deep learning involves the training of neural networks with a large number of parameters. For large enough , in the so-called over-parametrized regime, one can es…
Optimizing Data Distribution and Kernel Performance for Efficient Training of Chemistry Foundation Models: A Case Study with MACE
Jesun Firoz, Franco Pellegrini, Mario Geiger +17
Chemistry Foundation Models (CFMs) that leverage Graph Neural Networks (GNNs) operating on 3D molecular graph structures are becoming indispensable tools for computational chemists…
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning
Antonio Sclocchi, Mario Geiger, Matthieu Wyart
Understanding when the noise in stochastic gradient descent (SGD) affects generalization of deep neural networks remains a challenge, complicated by the fact that networks can oper…
A General Theory of Equivariant CNNs on Homogeneous Spaces
Taco Cohen, Mario Geiger, Maurice Weiler
We present a general theory of Group equivariant Convolutional Neural Networks (G-CNNs) on homogeneous spaces such as Euclidean space and the sphere. Feature maps in these networks…
A jamming transition from under- to over-parametrization affects loss landscape and generalization
Stefano Spigler, Mario Geiger, Stéphane d'Ascoli +3
We argue that in fully-connected networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved. Under some general condition…
EquiJump: Protein Dynamics Simulation via SO(3)-Equivariant Stochastic Interpolants
Allan dos Santos Costa, Ilan Mitnikov, Franco Pellegrini +7
Mapping the conformational dynamics of proteins is crucial for elucidating their functional mechanisms. While Molecular Dynamics (MD) simulation enables detailed time evolution of…
Phonon predictions with E(3)-equivariant graph neural networks
Shiang Fang, Mario Geiger, Joseph G. Checkelsky +1
We present an equivariant neural network for predicting vibrational and phonon modes of molecules and periodic crystals, respectively. These predictions are made by evaluating the…
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training
Mario Geiger, Leonardo Petrini, Matthieu Wyart
Deep learning algorithms are responsible for a technological revolution in a variety of tasks including image recognition or Go playing. Yet, why they work is not understood. Ultim…
A General Framework for Equivariant Neural Networks on Reductive Lie Groups
Ilyes Batatia, Mario Geiger, Jose Munoz +3
Reductive Lie Groups, such as the orthogonal groups, the Lorentz group, or the unitary groups, play essential roles across scientific fields as diverse as high energy physics, quan…
Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties
Benjamin Kurt Miller, Mario Geiger, Tess E. Smidt +1
Equivariant neural networks (ENNs) are graph neural networks embedded in and are well suited for predicting molecular properties. The ENN library e3nn has customizab…
Spherical CNNs
Taco S. Cohen, Mario Geiger, Jonas Koehler +1
Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have create…
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm
Stefano Spigler, Mario Geiger, Matthieu Wyart
How many training data are needed to learn a supervised task? It is often observed that the generalization error decreases as where is the number of training examples…
BioNeMo Framework: a modular, high-performance library for AI model development in drug discovery
Peter St. John, Dejun Lin, Polina Binder +89
Artificial Intelligence models encoding biology and chemistry are opening new routes to high-throughput and high-quality in-silico drug development. However, their training increas…
E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
Simon Batzner, Albert Musaelian, Lixin Sun +6
This work presents Neural Equivariant Interatomic Potentials (NequIP), an E(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations f…
Convolutional Networks for Spherical Signals
Taco Cohen, Mario Geiger, Jonas Köhler +1
The success of convolutional networks in learning problems involving planar signals such as images is due to their ability to exploit the translation symmetry of the data distribut…
3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
Maurice Weiler, Mario Geiger, Max Welling +2
We present a convolutional network that is equivariant to rigid body motions. The model uses scalar-, vector-, and tensor fields over 3D Euclidean space to represent data, and equi…
Proteina: Scaling Flow-based Protein Structure Generative Models
Tomas Geffner, Kieran Didi, Zuobai Zhang +8
Recently, diffusion- and flow-based generative models of protein structures have emerged as a powerful tool for de novo protein design. Here, we develop Proteina, a new large-scale…
The Strong Gravitational Lens Finding Challenge
R. Benton Metcalf, M. Meneghetti, Camille Avestruz +33
Large scale imaging surveys will increase the number of galaxy-scale strong lensing candidates by maybe three orders of magnitudes beyond the number known today. Finding these rare…
How memory architecture affects learning in a simple POMDP: the two-hypothesis testing problem
Mario Geiger, Christophe Eloy, Matthieu Wyart
Reinforcement learning is generally difficult for partially observable Markov decision processes (POMDPs), which occurs when the agent's observation is partial or noisy. To seek go…
Geometric compression of invariant manifolds in neural nets
Jonas Paccolat, Leonardo Petrini, Mario Geiger +2
We study how neural networks compress uninformative input space in models where data lie in dimensions, but whose label only vary within a linear manifold of dimension $d_\para…
SE(3)-equivariant prediction of molecular wavefunctions and electronic densities
Oliver T. Unke, Mihail Bogojeski, Michael Gastegger +3
Machine learning has enabled the prediction of quantum chemical properties with high accuracy and efficiency, allowing to bypass computationally costly ab initio calculations. Inst…
Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data
Ivan Diaz, Mario Geiger, Richard Iain McKinley
Convolutional neural networks (CNNs) allow for parameter sharing and translational equivariance by using convolutional kernels in their linear layers. By restricting these kernels…
The jamming transition as a paradigm to understand the loss landscape of deep neural networks
Mario Geiger, Stefano Spigler, Stéphane d'Ascoli +4
Deep learning has been immensely successful at a variety of tasks, ranging from classification to AI. Learning corresponds to fitting training data, which is implemented by descend…