243 citations · 330 across the 6 of their papers we have counts for
4 papers · 1 filter
eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization
Aditya Agrawal, Matthew Hedlund, Blake Hechtman
eXmY is a novel data type for quantization of ML models. It supports both arbitrary bit widths and arbitrary integer and floating point formats. For example, it seamlessly supports…
Exploring the limits of Concurrency in ML Training on Google TPUs
Sameer Kumar, James Bradbury, Cliff Young +16
Recent results in language understanding using neural networks have required training hardware of unprecedentedscale, with thousands of chips cooperating on a single training run.…
Scale MLPerf-0.6 models on Google TPU-v3 Pods
Sameer Kumar, Victor Bitorff, Dehao Chen +9
The recent submission of Google TPU-v3 Pods to the industry wide MLPerf v0.6 training benchmark demonstrates the scalability of a suite of industry relevant ML models. MLPerf defin…
Mesh-TensorFlow: Deep Learning for Supercomputers
Noam Shazeer, Youlong Cheng, Niki Parmar +9
Batch-splitting (data-parallelism) is the dominant distributed Deep Neural Network (DNN) training strategy, due to its universal applicability and its amenability to Single-Program…