Effective Approaches to Batch Parallelization for Dynamic Neural Network Architectures

Joseph Suarez, Clare Zhu

We present a simple dynamic batching approach applicable to a large class of dynamic architectures that consistently yields speedups of over 10x. We provide performance bounds when…