20 citations · 86 across the 19 of their papers we have counts for
5 papers · 1 filter
Large-Scale Deep Learning Optimizations: A Comprehensive Survey
Xiaoxin He, Fuzhao Xue, Xiaozhe Ren +1
Deep learning have achieved promising results on a wide spectrum of AI applications. Larger datasets and models consistently yield better performance. However, we generally spend l…
Cross-token Modeling with Conditional Computation
Yuxuan Lou, Fuzhao Xue, Zangwei Zheng +1
Mixture-of-Experts (MoE), a conditional computation architecture, achieved promising performance by scaling local module (i.e. feed-forward network) of transformer. However, scalin…
Go Wider Instead of Deeper
Fuzhao Xue, Ziji Shi, Futao Wei +3
More transformer blocks with residual connections have recently achieved impressive results on various tasks. To achieve better performance with fewer trainable parameters, recent…
Concurrent Adversarial Learning for Large-Batch Training
Yong Liu, Xiangning Chen, Minhao Cheng +2
Large-batch training has become a commonly used technique when training neural networks with a large number of GPU/TPU processors. As batch size increases, stochastic optimizers te…
An Efficient Training Approach for Very Large Scale Face Recognition
Kai Wang, Shuo Wang, Panpan Zhang +7
Face recognition has achieved significant progress in deep learning era due to the ultra-large-scale and welllabeled datasets. However, training on the outsize datasets is time-con…