40 citations · 84 across the 11 of their papers we have counts for
11 papers
Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models
Guangji Bai, Zheng Chai, Chen Ling +11
The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated models like OpenAI's ChatGPT, represents a significant advancement in artificial intelligence. Th…
Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction
Guangji Bai, Ziyang Yu, Zheng Chai +2
Despite the recent success of Graph Neural Networks (GNNs), it remains challenging to train GNNs on large-scale graphs due to neighbor explosions. As a remedy, distributed computin…
Distributed Graph Neural Network Training with Periodic Stale Representation Synchronization
Zheng Chai, Guangji Bai, Liang Zhao +1
Despite the recent success of Graph Neural Networks, it remains challenging to train a GNN on large graphs with millions of nodes and billions of edges, which are prevalent in many…
Asynchronous Federated Learning for Sensor Data with Concept Drift
Yujing Chen, Zheng Chai, Yue Cheng +1
Federated learning (FL) involves multiple distributed devices jointly training a shared model without any of the participants having to reveal their local data to a centralized ser…
Towards Quantized Model Parallelism for Graph-Augmented MLPs Based on Gradient-Free ADMM Framework
Junxiang Wang, Hongyi Li, Zheng Chai +3
While Graph Neural Networks (GNNs) are popular in the deep learning community, they suffer from several challenges including over-smoothing, over-squashing, and gradient vanishing.…
FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
Zheng Chai, Yujing Chen, Ali Anwar +3
Federated learning (FL) involves training a model over massive distributed devices, while keeping the training data localized. This form of collaborative learning exposes new trade…
Tunable Subnetwork Splitting for Model-parallelism of Neural Network Training
Junxiang Wang, Zheng Chai, Yue Cheng +1
Alternating minimization methods have recently been proposed as alternatives to the gradient descent for deep neural network optimization. Alternating minimization methods can typi…
Toward Model Parallelism for Deep Neural Network based on Gradient-free ADMM Framework
Junxiang Wang, Zheng Chai, Yue Cheng +1
Alternating Direction Method of Multipliers (ADMM) has recently been proposed as a potential alternative optimizer to the Stochastic Gradient Descent(SGD) for deep learning problem…
TiFL: A Tier-based Federated Learning System
Zheng Chai, Ahsan Ali, Syed Zawad +7
Federated Learning (FL) enables learning a shared model across many clients without violating the privacy requirements. One of the key attributes in FL is the heterogeneity that ex…
Federated Multi-task Hierarchical Attention Model for Sensor Analytics
Yujing Chen, Yue Ning, Zheng Chai +1
Sensors are an integral part of modern Internet of Things (IoT) applications. There is a critical need for the analysis of heterogeneous multivariate temporal data obtained from th…
Characterizing Co-located Datacenter Workloads: An Alibaba Case Study
Yue Cheng, Zheng Chai, Ali Anwar
Warehouse-scale cloud datacenters co-locate workloads with different and often complementary characteristics for improved resource utilization. To better understand the challenges…