16 papers · 1 filter
Neural Field Tokenizations with Hierarchy and Spatial Locality Priors
Alonso Urbano, David W. Romero, Max Zimmer +1
Neural fields parameterize data as functions from coordinates to values, providing a unified framework for representation learning across modalities. Existing approaches are domina…
The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning
Max Zimmer, Nico Pelleriti, Christophe Roux +1
AI tools and agents are reshaping how researchers work, from proving theorems to training neural networks. Yet for many, it remains unclear how these tools fit into everyday resear…
From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
Louis Schiekiera, Max Zimmer, Christophe Roux +2
We investigate the extent to which an LLM's hidden-state geometry can be recovered from its behavior in psycholinguistic experiments. Across eight instruction-tuned transformer mod…
SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale
Max Zimmer, Christophe Roux, Moritz Wagner +2
The resource requirements of neural networks can be significantly reduced through pruning - the removal of seemingly less important parameters. However, for LLMs, full retraining t…
Don't Be Greedy, Just Relax! Pruning LLMs via Frank-Wolfe
Christophe Roux, Max Zimmer, Alexandre d'Aspremont +1
Pruning is a common technique to reduce the compute and storage requirements of Neural Networks. While conventional approaches typically retrain the model to recover pruning-induce…
Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers
Nico Pelleriti, Christoph Spiegel, Shiwei Liu +3
Certifying nonnegativity of polynomials is a well-known NP-hard problem with direct applications spanning non-convex optimization, control, robotics, and beyond. A sufficient condi…