81 citations · 429 across the 42 of their papers we have counts for
11 papers · 1 filter
The LLM Surgeon
Tycho F. A. van der Ouderaa, Markus Nagel, Mart van Baalen +2
State-of-the-art language models are becoming increasingly large in an effort to achieve the highest performance on large corpora of available textual data. However, the sheer size…
Scalarization for Multi-Task and Multi-Domain Learning at Scale
Amelie Royer, Tijmen Blankevoort, Babak Ehteshami Bejnordi
Training a single model on multiple input domains and/or output tasks allows for compressing information from multiple sources into a unified backbone hence improves model efficien…
VeRA: Vector-based Random Matrix Adaptation
Dawid J. Kopiczko, Tijmen Blankevoort, Yuki M. Asano
Low-rank adapation (LoRA) is a popular method that reduces the number of trainable parameters when finetuning large language models, but still faces acute storage challenges when s…
Efficient Neural PDE-Solvers using Quantization Aware Training
Winfried van den Dool, Tijmen Blankevoort, Max Welling +1
In the past years, the application of neural networks as an alternative to classical numerical methods to solve Partial Differential Equations has emerged as a potential paradigm s…
QBitOpt: Fast and Accurate Bitwidth Reallocation during Training
Jorn Peters, Marios Fournarakis, Markus Nagel +2
Quantizing neural networks is one of the most effective methods for achieving efficient inference on mobile and embedded devices. In particular, mixed precision quantized (MPQ) net…
Pruning vs Quantization: Which is Better?
Andrey Kuzmin, Markus Nagel, Mart van Baalen +2
Neural network pruning and quantization techniques are almost as old as neural networks themselves. However, to date only ad-hoc comparisons between the two have been published. In…