181 citations · 620 across the 171 of their papers we have counts for
38 papers · 1 filter
LinearPFN: Amortized Variable Selection for Linear Models with Interactions
Louis Schiekiera, Max Zimmer, Christophe Roux +3
Spike-and-slab regression is a standard Bayesian formulation of variable selection: it returns a posterior distribution over which candidate effects are active rather than a single…
Neural Field Tokenizations with Hierarchy and Spatial Locality Priors
Alonso Urbano, David W. Romero, Max Zimmer +1
Neural fields parameterize data as functions from coordinates to values, providing a unified framework for representation learning across modalities. Existing approaches are domina…
The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning
Max Zimmer, Nico Pelleriti, Christophe Roux +1
AI tools and agents are reshaping how researchers work, from proving theorems to training neural networks. Yet for many, it remains unclear how these tools fit into everyday resear…
From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
Louis Schiekiera, Max Zimmer, Christophe Roux +2
We investigate the extent to which an LLM's hidden-state geometry can be recovered from its behavior in psycholinguistic experiments. Across eight instruction-tuned transformer mod…
SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale
Max Zimmer, Christophe Roux, Moritz Wagner +2
The resource requirements of neural networks can be significantly reduced through pruning - the removal of seemingly less important parameters. However, for LLMs, full retraining t…
Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe
Christophe Roux, Max Zimmer, Alexandre d'Aspremont +1
Pruning is a common technique to reduce the compute and storage requirements of Neural Networks. While conventional approaches typically retrain the model to recover pruning-induce…
Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers
Nico Pelleriti, Christoph Spiegel, Shiwei Liu +3
Certifying nonnegativity of polynomials is a well-known NP-hard problem with direct applications spanning non-convex optimization, control, robotics, and beyond. A sufficient condi…
A Free Lunch in LLM Compression: Revisiting Retraining after Pruning
Moritz Wagner, Christophe Roux, Max Zimmer +1
Post-training pruning can substantially reduce LLM inference costs, but it often degrades quality unless the remaining weights are adapted. Since global retraining is expensive at…
Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings
Berkant Turan, Suhrab Asadulla, David Steinmann +3
While Prover-Verifier Games (PVGs) offer a promising path toward verifiability in nonlinear classification models, they have not yet been applied to complex inputs such as high-dim…
Computational Algebra with Attention: Transformer Oracles for Border Basis Algorithms
Hiroshi Kera, Nico Pelleriti, Yuki Ishihara +2
Solving systems of polynomial equations, particularly those with finitely many solutions, is a crucial challenge across many scientific fields. Traditional methods like Gröbner and…
Training on Plausible Counterfactuals Removes Spurious Correlations
Shpresim Sadiku, Kartikeya Chitranshi, Hiroshi Kera +1
Plausible counterfactual explanations (p-CFEs) are perturbations that minimally modify inputs to change classifier decisions while remaining plausible under the data distribution.…
RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalization
Alonso Urbano, David W. Romero, Max Zimmer +1
Real world data often exhibits unknown, instance-specific symmetries that rarely exactly match a transformation group fixed a priori. Class-pose decompositions aim to create di…
Approximating Latent Manifolds in Neural Networks via Vanishing Ideals
Nico Pelleriti, Max Zimmer, Elias Wirth +1
Deep neural networks have reshaped modern machine learning by learning powerful latent representations that often align with the manifold hypothesis: high-dimensional data lie on l…
Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation
Jan Pauls, Max Zimmer, Berkant Turan +4
With the rise in global greenhouse gas emissions, accurate large-scale tree canopy height maps are essential for understanding forest structure, estimating above-ground biomass, an…
Neural Discovery in Mathematics: Do Machines Dream of Colored Planes?
Konrad Mundinger, Max Zimmer, Aldo Kiem +2
We demonstrate how neural networks can drive mathematical discovery through a case study of the Hadwiger-Nelson problem, a long-standing open problem at the intersection of discret…
S-CFE: Simple Counterfactual Explanations
Shpresim Sadiku, Moritz Wagner, Sai Ganesh Nagarajan +1
We study the problem of finding optimal sparse, manifold-aligned counterfactual explanations for classifiers. Canonically, this can be formulated as an optimization problem with mu…
Neural Parameter Regression for Explicit Representations of PDE Solution Operators
Konrad Mundinger, Max Zimmer, Sebastian Pokutta
We introduce Neural Parameter Regression (NPR), a novel framework specifically developed for learning solution operators in Partial Differential Equations (PDEs). Tailored for oper…
On the Byzantine-Resilience of Distillation-Based Federated Learning
Christophe Roux, Max Zimmer, Sebastian Pokutta
Federated Learning (FL) algorithms using Knowledge Distillation (KD) have received increasing attention due to their favorable properties with respect to privacy, non-i.i.d. data a…
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs
Max Zimmer, Megi Andoni, Christoph Spiegel +1
Neural Networks can be effectively compressed through pruning, significantly reducing storage and compute demands while maintaining predictive performance. Simple yet effective met…
Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging
Max Zimmer, Christoph Spiegel, Sebastian Pokutta
Neural networks can be significantly compressed by pruning, yielding sparse models with reduced storage and computational demands while preserving predictive performance. Model sou…
Approximate Vanishing Ideal Computations at Scale
Elias Wirth, Hiroshi Kera, Sebastian Pokutta
The vanishing ideal of a set of points is the set of polynomials that evaluate to over all points $\mathbf{x}…
Interpretability Guarantees with Merlin-Arthur Classifiers
Stephan Wäldchen, Kartikey Sharma, Berkant Turan +2
We propose an interactive multi-agent classifier that provides provable interpretability guarantees even for complex agents such as neural networks. These guarantees consist of low…
Compression-aware Training of Neural Networks using Frank-Wolfe
Max Zimmer, Christoph Spiegel, Sebastian Pokutta
Many existing Neural Network pruning approaches rely on either retraining or inducing a strong bias in order to converge to a sparse solution throughout training. A third paradigm,…
The Machine Learning for Combinatorial Optimization Competition (ML4CO): Results and Insights
Maxime Gasse, Quentin Cappart, Jonas Charfreitag +38
Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have focused on solving problem instances in isolatio…