4 citations · 7 across the 2 of their papers we have counts for
4 papers
EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models
Jiangsu Du, Ziming Liu, Jiarui Fang +4
Large transformer models display promising performance on a wide range of natural language processing (NLP) tasks. Although the AI community has expanded the model scale to the tri…
TurboTransformers: An Efficient GPU Serving System For Transformer Models
Jiarui Fang, Yang Yu, Chengduo Zhao +1
The transformer is the most critical algorithm innovation of the Nature Language Processing (NLP) field in recent years. Unlike the Recurrent Neural Network (RNN) models, Transform…
swCaffe: a Parallel Framework for Accelerating Deep Learning Applications on Sunway TaihuLight
Jiarui Fang, Liandeng Li, Haohuan Fu +5
This paper reports our efforts on swCaffe, a highly efficient parallel framework for accelerating deep neural networks (DNNs) training on Sunway TaihuLight, the current fastest sup…
RedSync : Reducing Synchronization Traffic for Distributed Deep Learning
Jiarui Fang, Haohuan Fu, Guangwen Yang +1
Data parallelism has become a dominant method to scale Deep Neural Network (DNN) training across multiple nodes. Since synchronizing a large number of gradients of the local model…