5.7k citations · 5.8k across the 2 of their papers we have counts for
3 papers
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts
Trevor Gale, Deepak Narayanan, Cliff Young +1
We present MegaBlocks, a system for efficient Mixture-of-Experts (MoE) training on GPUs. Our system is motivated by the limitations of current frameworks, which restrict the dynami…
MLPerf Training Benchmark
Peter Mattson, Christine Cheng, Cody Coleman +34
Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML. But M…
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen +28
Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, with the potential to overcome many of the weaknesses of conventional phrase-based tr…