34 citations · 102 across the 12 of their papers we have counts for
13 papers
Towards Robust Graph Incremental Learning on Evolving Graphs
Junwei Su, Difan Zou, Zijun Zhang +1
Incremental learning is a machine learning approach that involves training a model on a sequence of tasks, rather than all tasks at once. This ability to learn incrementally from a…
Adaptive Message Quantization and Parallelization for Distributed Full-graph GNN Training
Borui Wan, Juntao Zhao, Chuan Wu
Distributed full-graph training of Graph Neural Networks (GNNs) over large graphs is bandwidth-demanding and time-consuming. Frequent exchanges of node features, embeddings and emb…
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform
Shiwei Zhang, Lansong Diao, Siyu Wang +7
We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment. It transforms a tensor program written fo…
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment
Shiwei Zhang, Xiaodong Yi, Lansong Diao +3
This paper presents TAG, an automatic system to derive optimized DNN training graph and its deployment onto any device topology, for expedited training in device- and topology- het…
On the Limitation and Experience Replay for GNNs in Continual Learning
Junwei Su, Difan Zou, Chuan Wu
Continual learning seeks to empower models to progressively acquire information from a sequence of tasks. This approach is crucial for many real-world systems, which are dynamic an…
Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion
Xiaodong Yi, Shiwei Zhang, Lansong Diao +6
This paper proposes DisCo, an automatic deep learning compilation module for data-parallel distributed training. Unlike most deep learning compilers that focus on training or infer…
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
Hanpeng Hu, Chenyu Jiang, Yuchen Zhong +5
Distributed training using multiple devices (e.g., GPUs) has been widely adopted for learning DNN models over large datasets. However, the performance of large-scale distributed tr…
Optimizing Task Placement and Online Scheduling for Distributed GNN Training Acceleration
Ziyue Luo, Yixin Bao, Chuan Wu
Training Graph Neural Networks (GNN) on large graphs is resource-intensive and time-consuming, mainly due to the large graph data that cannot be fit into the memory of a single mac…
Efficient Pipeline Planning for Expedited Distributed DNN Training
Ziyue Luo, Xiaodong Yi, Guoping Long +4
To train modern large DNN models, pipeline parallelism has recently emerged, which distributes the model across GPUs and enables different devices to process different microbatches…
Large-scale Machine Learning Cluster Scheduling via Multi-agent Graph Reinforcement Learning
Xiaoyang Zhao, Chuan Wu
Efficient scheduling of distributed deep learning (DL) jobs in large GPU clusters is crucial for resource efficiency and job performance. While server sharing among jobs improves r…
BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing
Tianfeng Liu, Yangrui Chen, Dan Li +7
Graph neural networks (GNNs) have extended the success of deep neural networks (DNNs) to non-Euclidean graph data, achieving ground-breaking performance on various tasks such as no…
Adversarial Deep Learning for Online Resource Allocation
Bingqian Du, Zhiyi Huang, Chuan Wu
Online algorithm is an important branch in algorithm design. Designing online algorithms with a bounded competitive ratio (in terms of worst-case performance) can be hard and usual…
OneFlow: Redesign the Distributed Deep Learning Framework from Scratch
Jinhui Yuan, Xinqi Li, Cheng Cheng +9
Deep learning frameworks such as TensorFlow and PyTorch provide a productive interface for expressing and training a deep neural network (DNN) model on a single device or using dat…