72 citations · 542 across the 65 of their papers we have counts for
7 papers · 1 filter
i-SpaSP: Structured Neural Pruning via Sparse Signal Recovery
Cameron R. Wolfe, Anastasios Kyrillidis
We propose a novel, structured pruning algorithm for neural networks -- the iterative, Sparse Structured Pruning algorithm, dubbed as i-SpaSP. Inspired by ideas from sparse signal…
REX: Revisiting Budgeted Training with an Improved Schedule
John Chen, Cameron Wolfe, Anastasios Kyrillidis
Deep learning practitioners often operate on a computational and monetary budget. Thus, it is critical to design optimization algorithms that perform well under any budget. The lin…
Momentum-inspired Low-Rank Coordinate Descent for Diagonally Constrained SDPs
Junhyung Lyle Kim, JA Lara Benitez, Mohammad Taha Toghani +3
We present a novel, practical, and provable approach for solving diagonally constrained semi-definite programming (SDP) problems at scale using accelerated non-convex programming.…
Mitigating deep double descent by concatenating inputs
John Chen, Qihan Wang, Anastasios Kyrillidis
The double descent curve is one of the most intriguing properties of deep neural networks. It contrasts the classical bias-variance curve with the behavior of modern neural network…
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training
Chen Dun, Cameron R. Wolfe, Christopher M. Jermaine +1
We propose ResIST, a novel distributed training protocol for Residual Networks (ResNets). ResIST randomly decomposes a global ResNet into several shallow sub-ResNets that are train…
Fast quantum state reconstruction via accelerated non-convex programming
Junhyung Lyle Kim, George Kollias, Amir Kalev +2
We propose a new quantum state reconstruction method that combines ideas from compressed sensing, non-convex optimization, and acceleration methods. The algorithm, called Momentum-…