11 papers
pscaling small models: Principled warm starts and hyperparameter transfer
Yuxin Ma, Nan Chen, Mateo DÃaz +3
Modern large-scale neural networks are often trained and released in multiple sizes to accommodate diverse inference budgets. To improve efficiency, recent work has explored model…
Any-Dimensional Invariant Universality
Shengtai Yao, Eitan Levin, Mateo DÃaz
Several machine learning models are defined for inputs of any size, such as graphs with different numbers of nodes and point clouds containing varying numbers of points. The univer…
Nonsmooth Riemannian optimization with inexact manifold primitives via bundle methods
Mateo DÃaz, Benjamin Grimmer, Ian McPherson
Optimization on Hadamard manifolds -- the natural Riemannian setting for globally geodesically convex problems -- relies on exponential maps to retract tangent vectors and parallel…
PDLP: A Practical First-Order Method for Large-Scale Linear Programming
David Applegate, Mateo DÃaz, Oliver Hinder +4
We present PDLP, a practical first-order method for linear programming (LP) designed to solve large-scale LP problems. PDLP is based on the primal-dual hybrid gradient (PDHG) metho…
Active set identification and rapid convergence for degenerate primal-dual problems
Mateo DÃaz, Pedro Izquierdo Lehmann, Haihao Lu +1
Primal-dual methods for solving convex optimization problems with functional constraints often exhibit a distinct two-stage behavior. Initially, they converge towards a solution at…
On Transferring Transferability: Towards a Theory for Size Generalization
Eitan Levin, Yuxin Ma, Mateo DÃaz +1
Many modern learning tasks require models that can take inputs of varying sizes. Consequently, dimension-independent architectures have been proposed for domains where the inputs a…