BrainSlug: Transparent Acceleration of Deep Learning Through Depth-First Parallelism
arXiv:1804.08378
Abstract
Neural network frameworks such as PyTorch and TensorFlow are the workhorses of numerous machine learning applications ranging from object recognition to machine translation. While these frameworks are versatile and straightforward to use, the training of and inference in deep neural networks is resource (energy, compute, and memory) intensive. In contrast to recent works focusing on algorithmic enhancements, we introduce BrainSlug, a framework that transparently accelerates neural network workloads by changing the default layer-by-layer processing to a depth-first approach, reducing the amount of data required by the computations and thus improving the performance of the available hardware caches. BrainSlug achieves performance improvements of up to 41.1% on CPUs and 35.7% on GPUs. These optimizations come at zero cost to the user as they do not require hardware changes and only need tiny adjustments to the software.
arXiv commentTechnical Report, 13 pages
References in corpus (9)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Caffe: Convolutional Architecture for Fast Feature Embedding
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- cuDNN: Efficient Primitives for Deep Learning
- Binarized Neural Networks
- Pruning Filters for Efficient ConvNets
- Don't Decay the Learning Rate, Increase the Batch Size
- Rethinking the Inception Architecture for Computer Vision