2 citations · 5 across the 13 of their papers we have counts for
4 papers · 1 filter
SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale
Max Zimmer, Christophe Roux, Moritz Wagner +2
The resource requirements of neural networks can be significantly reduced through pruning - the removal of seemingly less important parameters. However, for LLMs, full retraining t…
Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe
Christophe Roux, Max Zimmer, Alexandre d'Aspremont +1
Pruning is a common technique to reduce the compute and storage requirements of Neural Networks. While conventional approaches typically retrain the model to recover pruning-induce…
A Free Lunch in LLM Compression: Revisiting Retraining after Pruning
Moritz Wagner, Christophe Roux, Max Zimmer +1
Post-training pruning can substantially reduce LLM inference costs, but it often degrades quality unless the remaining weights are adapted. Since global retraining is expensive at…
Implicit Riemannian Optimism with Applications to Min-Max Problems
Christophe Roux, David Martínez-Rubio, Sebastian Pokutta
We introduce a Riemannian optimistic online learning algorithm for Hadamard manifolds based on inexact implicit updates. Unlike prior work, our method can handle in-manifold constr…