2 citations · 5 across the 8 of their papers we have counts for
8 papers
Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks
Jinping Zou, Xiaoge Deng, Tao Sun
Sharpness-Aware Minimization (SAM) has proven highly effective in improving model generalization in machine learning tasks. However, SAM employs a fixed hyperparameter associated w…
Federated Prediction-Powered Inference from Decentralized Data
Ping Luo, Xiaoge Deng, Ziqing Wen +2
In various domains, the increasing application of machine learning allows researchers to access inexpensive predictive data, which can be utilized as auxiliary data for statistical…
Adaptive Momentum Benefits Score-based Generative Models
Ziqing Wen, Xiaoge Deng, Ping Luo +3
Score-based Generative Models (SGMs) have achieved remarkable success in generation tasks by establishing a diffusion process that gradually perturbs real data into Gaussian noise…
Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
Ping Luo, Xiaoge Deng, Ziqing Wen +2
Federated Learning (FL) is a distributed machine learning framework in communication network systems. However, the systems' Non-Independent and Identically Distributed (Non-IID) da…
Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
Xiaoge Deng, Li Shen, Shengwei Li +3
Stochastic gradient descent (SGD) performed in an asynchronous manner plays a crucial role in training large-scale machine learning models. However, the generalization performance…
Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
Shengwei Li, Zhiquan Lai, Dongsheng Li +5
Deep learning is experiencing a rise in large-scale models. Training large-scale models is costly, prompting researchers to train large-scale models on commodity servers that more…
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient
Xiaoge Deng, Dongsheng Li, Tao Sun +1
Gradient-based optimization methods implemented on distributed computing architectures are increasingly used to tackle large-scale machine learning applications. A key bottleneck i…
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning
Keshi Ge, Yongquan Fu, Zhiquan Lai +2
Distributed stochastic gradient descent (SGD) approach has been widely used in large-scale deep learning, and the gradient collective method is vital to ensure the training scalabi…