14 citations · 44 across the 14 of their papers we have counts for
10 papers · 1 filter
Better Schedules for Low Precision Training of Deep Neural Networks
Cameron R. Wolfe, Anastasios Kyrillidis
Low precision training can significantly reduce the computational overhead of training deep neural networks (DNNs). Though many such techniques exist, cyclic precision training (CP…
Cold Start Streaming Learning for Deep Networks
Cameron R. Wolfe, Anastasios Kyrillidis
The ability to dynamically adapt neural networks to newly-available data without performance deterioration would revolutionize deep learning applications. Streaming learning (i.e.,…
PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication
Cheng Wan, Youjie Li, Cameron R. Wolfe +3
Graph Convolutional Networks (GCNs) is the state-of-the-art method for learning graph-structured data, and training large-scale GCNs requires distributed training across multiple a…
i-SpaSP: Structured Neural Pruning via Sparse Signal Recovery
Cameron R. Wolfe, Anastasios Kyrillidis
We propose a novel, structured pruning algorithm for neural networks -- the iterative, Sparse Structured Pruning algorithm, dubbed as i-SpaSP. Inspired by ideas from sparse signal…
REX: Revisiting Budgeted Training with an Improved Schedule
John Chen, Cameron Wolfe, Anastasios Kyrillidis
Deep learning practitioners often operate on a computational and monetary budget. Thus, it is critical to design optimization algorithms that perform well under any budget. The lin…
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training
Chen Dun, Cameron R. Wolfe, Christopher M. Jermaine +1
We propose ResIST, a novel distributed training protocol for Residual Networks (ResNets). ResIST randomly decomposes a global ResNet into several shallow sub-ResNets that are train…
GIST: Distributed Training for Large-Scale Graph Convolutional Networks
Cameron R. Wolfe, Jingkang Yang, Arindam Chowdhury +4
The graph convolutional network (GCN) is a go-to solution for machine learning on graphs, but its training is notoriously difficult to scale both in terms of graph size and the num…
E-Stitchup: Data Augmentation for Pre-Trained Embeddings
Cameron R. Wolfe, Keld T. Lundgaard
In this work, we propose data augmentation methods for embeddings from pre-trained deep learning models that take a weighted combination of a pair of input embeddings, as inspired…
Demon: Improved Neural Network Training with Momentum Decay
John Chen, Cameron Wolfe, Zhao Li +1
Momentum is a widely used technique for gradient-based optimizers in deep learning. In this paper, we propose a decaying momentum (\textsc{Demon}) rule. We conduct the first large-…
Distributed Learning of Deep Neural Networks using Independent Subnet Training
Binhang Yuan, Cameron R. Wolfe, Chen Dun +3
Distributed machine learning (ML) can bring more computational resources to bear than single-machine learning, thus enabling reductions in training time. Distributed learning parti…