11 citations · 24 across the 10 of their papers we have counts for
13 papers
SQuAT: Spherical Lattice Vector Quantization-Aware Training
Sebastian Loeschcke, Ivan Koryakovskiy, Tycho F. A. van der Ouderaa +4
Vector quantization (VQ), in particular lattice-based methods, offer strong rate-distortion trade-offs for low-bit large language models (LLMs), yet their behavior under quantizati…
Noether's razor: Learning Conserved Quantities
Tycho F. A. van der Ouderaa, Mark van der Wilk, Pim de Haan
Symmetries have proven useful in machine learning models, improving generalisation and overall performance. At the same time, recent advancements in learning dynamical systems rely…
Pyramid Vector Quantization for LLMs
Tycho F. A. van der Ouderaa, Maximilian L. Croci, Agrin Hilmkil +1
Recent works on compression of large language models (LLM) using quantization considered reparameterizing the architecture such that weights are distributed on the sphere. This dem…
Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks
Yoav Gelberg, Tycho F. A. van der Ouderaa, Mark van der Wilk +1
Weight space symmetries in neural network architectures, such as permutation symmetries in MLPs, give rise to Bayesian neural network (BNN) posteriors with many equivalent modes. T…
The LLM Surgeon
Tycho F. A. van der Ouderaa, Markus Nagel, Mart van Baalen +2
State-of-the-art language models are becoming increasingly large in an effort to achieve the highest performance on large corpora of available textual data. However, the sheer size…
Learning Layer-wise Equivariances Automatically using Gradients
Tycho F. A. van der Ouderaa, Alexander Immer, Mark van der Wilk
Convolutions encode equivariance symmetries into neural networks leading to better generalisation performance. However, symmetries provide fixed hard constraints on the functions a…
Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels
Alexander Immer, Tycho F. A. van der Ouderaa, Mark van der Wilk +2
Selecting hyperparameters in deep learning greatly impacts its effectiveness but requires manual effort and expertise. Recent works show that Bayesian model selection with Laplace…
Relaxing Equivariance Constraints with Non-stationary Continuous Filters
Tycho F. A. van der Ouderaa, David W. Romero, Mark van der Wilk
Equivariances provide useful inductive biases in neural network modeling, with the translation equivariance of convolutional neural networks being a canonical example. Equivariance…
Learning Invariant Weights in Neural Networks
Tycho F. A. van der Ouderaa, Mark van der Wilk
Assumptions about invariances or symmetries in data can significantly increase the predictive power of statistical models. Many commonly used models in machine learning are constra…
Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations
Alexander Immer, Tycho F. A. van der Ouderaa, Gunnar Rätsch +2
Data augmentation is commonly applied to improve performance of deep learning by enforcing the knowledge that certain transformations on the input preserve the output. Currently, t…
Deep Group-wise Variational Diffeomorphic Image Registration
Tycho F. A. van der Ouderaa, Ivana Išgum, Wouter B. Veldhuis +1
Deep neural networks are increasingly used for pair-wise image registration. We propose to extend current learning-based image registration to allow simultaneous registration of mu…
Chest CT Super-resolution and Domain-adaptation using Memory-efficient 3D Reversible GANs
Tycho F. A. van der Ouderaa, Daniel E. Worrall, Bram van Ginneken
Recently, paired (e.g. Pix2pix) and unpaired (e.g. CycleGAN) image-to-image translation methods have shown effective in medical imaging tasks. In practice, however, it can be diffi…
Reversible GANs for Memory-efficient Image-to-Image Translation
Tycho F. A. van der Ouderaa, Daniel E. Worrall
The Pix2pix and CycleGAN losses have vastly improved the qualitative and quantitative visual quality of results in image-to-image translation tasks. We extend this framework by exp…