11 citations · 12 across the 2 of their papers we have counts for
3 papers
Adaptive Braking for Mitigating Gradient Delay
Abhinav Venigalla, Atli Kosson, Vitaliy Chiley +1
Neural network training is commonly accelerated by using multiple synchronized workers to compute gradient updates in parallel. Asynchronous methods remove synchronization overhead…
Pipelined Backpropagation at Scale: Training Large Models without Batches
Atli Kosson, Vitaliy Chiley, Abhinav Venigalla +2
New hardware can substantially increase the speed and efficiency of deep neural network training. To guide the development of future hardware architectures, it is pertinent to expl…
Higher Order Correlations within Cortical Layers Dominate Functional Connectivity in Microcolumns
Urs Köster, Jascha Sohl-Dickstein, Charles M. Gray +1
We report on simultaneous recordings from cells in all layers of visual cortex and models developed to capture the higher order structure of population spiking activity. Specifical…