4 citations · 13 across the 30 of their papers we have counts for
11 papers · 1 filter
SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale
Max Zimmer, Christophe Roux, Moritz Wagner +2
The resource requirements of neural networks can be significantly reduced through pruning - the removal of seemingly less important parameters. However, for LLMs, full retraining t…
Objective Coefficient Rounding and Almost Symmetries in Binary Programs
Dominik Kuzinowicz, Paweł Lichocki, Gioni Mexi +3
This article investigates the interplay of rounding objective coefficients in binary programs and almost symmetries. Empirically, reducing the number of significant bits through ro…
Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe
Christophe Roux, Max Zimmer, Alexandre d'Aspremont +1
Pruning is a common technique to reduce the compute and storage requirements of Neural Networks. While conventional approaches typically retrain the model to recover pruning-induce…
Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers
Nico Pelleriti, Christoph Spiegel, Shiwei Liu +3
Certifying nonnegativity of polynomials is a well-known NP-hard problem with direct applications spanning non-convex optimization, control, robotics, and beyond. A sufficient condi…
A Free Lunch in LLM Compression: Revisiting Retraining after Pruning
Moritz Wagner, Christophe Roux, Max Zimmer +1
Post-training pruning can substantially reduce LLM inference costs, but it often degrades quality unless the remaining weights are adapted. Since global retraining is expensive at…
Computational Algebra with Attention: Transformer Oracles for Border Basis Algorithms
Hiroshi Kera, Nico Pelleriti, Yuki Ishihara +2
Solving systems of polynomial equations, particularly those with finitely many solutions, is a crucial challenge across many scientific fields. Traditional methods like Gröbner and…
RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalization
Alonso Urbano, David W. Romero, Max Zimmer +1
Real world data often exhibits unknown, instance-specific symmetries that rarely exactly match a transformation group fixed a priori. Class-pose decompositions aim to create di…
DUNIA: Pixel-Sized Embeddings via Cross-Modal Alignment for Earth Observation Applications
Ibrahim Fayad, Max Zimmer, Martin Schwartz +6
Significant efforts have been directed towards adapting self-supervised multimodal learning for Earth observation applications. However, most current methods produce coarse patch-s…
Approximating Latent Manifolds in Neural Networks via Vanishing Ideals
Nico Pelleriti, Max Zimmer, Elias Wirth +1
Deep neural networks have reshaped modern machine learning by learning powerful latent representations that often align with the manifold hypothesis: high-dimensional data lie on l…
Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation
Jan Pauls, Max Zimmer, Berkant Turan +4
With the rise in global greenhouse gas emissions, accurate large-scale tree canopy height maps are essential for understanding forest structure, estimating above-ground biomass, an…
Neural Discovery in Mathematics: Do Machines Dream of Colored Planes?
Konrad Mundinger, Max Zimmer, Aldo Kiem +2
We demonstrate how neural networks can drive mathematical discovery through a case study of the Hadwiger-Nelson problem, a long-standing open problem at the intersection of discret…