9 citations · 16 across the 7 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.LG2021★ 1 cited
Loss Landscape Dependent Self-Adjusting Learning Rates in Decentralized Stochastic Gradient Descent
Wei Zhang, Mingrui Liu, Yu Feng +3
Distributed Deep Learning (DDL) is essential for large-scale Deep Learning (DL) training. Synchronous Stochastic Gradient Descent (SSGD) 1 is the de facto DDL optimization method.…
physics.bio-ph2021★ 9 cited
Storage capacity of networks with discrete synapses and sparsely encoded memories
Yu Feng, Nicolas Brunel
Attractor neural networks (ANNs) are one of the leading theoretical frameworks for the formation and retrieval of memories in networks of biological neurons. In this framework, a p…
cs.LG2021★ 1 cited
Phases of learning dynamics in artificial neural networks: with or without mislabeled data
Yu Feng, Yuhai Tu
Despite tremendous success of deep neural network in machine learning, the underlying reason for its superior learning capability remains unclear. Here, we present a framework base…