18 citations · 26 across the 9 of their papers we have counts for
10 papers · 1 filter
Cooper: A Library for Constrained Optimization in Deep Learning
Jose Gallego-Posada, Juan Ramirez, Meraj Hashemizadeh +1
Cooper is an open-source package for solving constrained optimization problems involving deep learning models. Cooper implements several Lagrangian-based first-order update schemes…
Feasible Learning
Juan Ramirez, Ignacio Hounie, Juan Elenter +4
We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample. In…
On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization
Motahareh Sohrabi, Juan Ramirez, Tianyue H. Zhang +2
Constrained optimization offers a powerful framework to prescribe desired behaviors in neural network models. Typically, constrained problems are solved via their min-max Lagrangia…
Balancing Act: Constraining Disparate Impact in Sparse Models
Meraj Hashemizadeh, Juan Ramirez, Rohan Sukumaran +3
Model pruning is a popular approach to enable the deployment of large deep learning models on edge devices with restricted computational or storage capacities. Although sparse mode…
A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
Hao-Jun Michael Shi, Tsung-Hsien Lee, Shintaro Iwasaki +5
Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad family of methods for training neural networks. It constructs a block-diagonal preconditioner wh…
Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints
Jose Gallego-Posada, Juan Ramirez, Akram Erraqabi +2
The performance of trained neural networks is robust to harsh levels of pruning. Coupled with the ever-growing size of deep learning models, this observation has motivated extensiv…