2 citations · 2 across the 4 of their papers we have counts for
4 papers · 1 filter
Biased Local SGD for Efficient Deep Learning on Heterogeneous Systems
Jihyun Lim, Junhyuk Jo, Chanhyeok Ko +3
Most parallel neural network training methods assume homogeneous computing resources. For example, synchronous data-parallel SGD suffers from significant synchronization overhead u…
Layer-wise Adaptive Gradient Norm Penalizing Method for Efficient and Accurate Deep Learning
Sunwoo Lee
Sharpness-aware minimization (SAM) is known to improve the generalization performance of neural networks. However, it is not widely used in real-world applications yet due to its e…
Asynchronous Sharpness-Aware Minimization For Fast and Accurate Deep Learning
Junhyuk Jo, Jihyun Lim, Sunwoo Lee +1
Sharpness-Aware Minimization (SAM) is an optimization method that improves the generalization performance of machine learning models. Despite its effectiveness, it has not been wid…
Enabling Weak Client Participation via On-device Knowledge Distillation in Heterogeneous Federated Learning
Jihyun Lim, Junhyuk Jo, Tuo Zhang +1
Online Knowledge Distillation (KD) is recently highlighted to train large models in Federated Learning (FL) environments. Many existing studies adopt the logit ensemble method to p…