48 citations · 94 across the 6 of their papers we have counts for
7 papers
Scaling Up Models and Data with and
Adam Roberts, Hyung Won Chung, Anselm Levskaya +40
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can…
Pathways: Asynchronous Distributed Dataflow for ML
Paul Barham, Aakanksha Chowdhery, Jeff Dean +13
We present the design of a new large scale orchestration layer for accelerators. Our system, Pathways, is explicitly designed to enable exploration of new systems and ML research i…
Native Implementation of Mutable Value Semantics
Dimitri Racordon, Denys Shabalin, Daniel Zheng +2
Unrestricted mutation of shared state is a source of many well-known problems. The predominant safe solutions are pure functional programming, which bans mutation outright, and flo…
LazyTensor: combining eager execution with domain-specific compilers
Alex Suhan, Davide Libenzi, Ailing Zhang +4
Domain-specific optimizing compilers have demonstrated significant performance and portability benefits, but require programs to be represented in their specialized IRs. Existing f…
Tensors Fitting Perfectly
Adam Paszke, Brennan Saeta
Multidimensional arrays (NDArrays) are a central abstraction in modern scientific computing environments. Unfortunately, they can make reasoning about programs harder as the number…
Swift for TensorFlow: A portable, flexible platform for deep learning
Brennan Saeta, Denys Shabalin, Marc Rasi +10
Swift for TensorFlow is a deep learning platform that scales from mobile devices to clusters of hardware accelerators in data centers. It combines a language-integrated automatic d…