9 citations · 16 across the 7 of their papers we have counts for
4 papers · 1 filter
FedFrozen: Two-Stage Federated Optimization via Attention Kernel Freezing
Junye Du, Zhenghao Li, Yushi Feng +1
Federated learning with heterogeneous clients remains a significant challenge for deep learning, primarily due to client drift arising from inconsistent local updates. Existing fed…
The activity-weight duality in feed forward neural networks: The geometric determinants of generalization
Yu Feng, Yuhai Tu
One of the fundamental problems in machine learning is generalization. In neural network models with a large number of weights (parameters), many solutions can be found to fit the…
Loss Landscape Dependent Self-Adjusting Learning Rates in Decentralized Stochastic Gradient Descent
Wei Zhang, Mingrui Liu, Yu Feng +3
Distributed Deep Learning (DDL) is essential for large-scale Deep Learning (DL) training. Synchronous Stochastic Gradient Descent (SSGD) 1 is the de facto DDL optimization method.…
Phases of learning dynamics in artificial neural networks: with or without mislabeled data
Yu Feng, Yuhai Tu
Despite tremendous success of deep neural network in machine learning, the underlying reason for its superior learning capability remains unclear. Here, we present a framework base…