15 citations · 29 across the 10 of their papers we have counts for
5 papers · 1 filter
Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR)
Guo-qing Jiang, Jinlong Liu, Zixiang Ding +2
As models for nature language processing (NLP), computer vision (CV) and recommendation systems (RS) require surging computation, a large number of GPUs/TPUs are paralleled as a la…
Ada-Grouper: Accelerating Pipeline Parallelism in Preempted Network by Adaptive Group-Scheduling for Micro-Batches
Siyu Wang, Zongyan Cao, Chang Si +3
Pipeline parallelism has been demonstrated to be a remarkable approach to improve throughput for training deep neural networks with billions of parameters over heterogeneous cluste…
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform
Shiwei Zhang, Lansong Diao, Siyu Wang +7
We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment. It transforms a tensor program written fo…
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment
Shiwei Zhang, Xiaodong Yi, Lansong Diao +3
This paper presents TAG, an automatic system to derive optimized DNN training graph and its deployment onto any device topology, for expedited training in device- and topology- het…
Source-Free Unsupervised Domain Adaptation: A Survey
Yuqi Fang, Pew-Thian Yap, Weili Lin +2
Unsupervised domain adaptation (UDA) via deep learning has attracted appealing attention for tackling domain-shift problems caused by distribution discrepancy across different doma…