1 citations · 1 across the 2 of their papers we have counts for
3 papers
Representation range needs for 16-bit neural network training
Valentina Popescu, Abhinav Venigalla, Di Wu +1
Deep learning has grown rapidly thanks to its state-of-the-art performance across a wide range of real-world applications. While neural networks have been trained using IEEE-754 bi…
Adaptive Braking for Mitigating Gradient Delay
Abhinav Venigalla, Atli Kosson, Vitaliy Chiley +1
Neural network training is commonly accelerated by using multiple synchronized workers to compute gradient updates in parallel. Asynchronous methods remove synchronization overhead…
Pipelined Backpropagation at Scale: Training Large Models without Batches
Atli Kosson, Vitaliy Chiley, Abhinav Venigalla +2
New hardware can substantially increase the speed and efficiency of deep neural network training. To guide the development of future hardware architectures, it is pertinent to expl…