151 citations · 243 across the 40 of their papers we have counts for
6 papers · 1 filter
Variational Low-Rank Adaptation Using IVON
Bai Cong, Nico Daheim, Yuesong Shen +4
We show that variational learning can significantly improve the accuracy and calibration of Low-Rank Adaptation (LoRA) without a substantial increase in the cost. We replace AdamW…
Interactions Across Blocks in Post-Training Quantization of Large Language Models
Khasmamad Shabanovi, Lukas Wiest, Vladimir Golkov +2
Post-training quantization is widely employed to reduce the computational demands of neural networks. Typically, individual substructures, such as layers or blocks of layers, are q…
How to Choose a Reinforcement-Learning Algorithm
Fabian Bongratz, Vladimir Golkov, Lukas Mautner +5
The field of reinforcement learning offers a large variety of concepts and methods to tackle sequential decision-making problems. This variety has become so large that choosing an…
Enhancing Hypergradients Estimation: A Study of Preconditioning and Reparameterization
Zhenzhang Ye, Gabriel Peyré, Daniel Cremers +1
Bilevel optimization aims to optimize an outer objective function that depends on the solution to an inner optimization problem. It is routinely used in Machine Learning, notably f…
HoloNets: Spectral Convolutions do extend to Directed Graphs
Christian Koke, Daniel Cremers
Within the graph learning community, conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs: Only there could the existence of…
ResolvNet: A Graph Convolutional Network with multi-scale Consistency
Christian Koke, Abhishek Saroha, Yuesong Shen +2
It is by now a well known fact in the graph learning community that the presence of bottlenecks severely limits the ability of graph neural networks to propagate information over l…