26 citations · 75 across the 8 of their papers we have counts for
13 papers
A Roadmap for Big Model
Sha Yuan, Hanyu Zhao, Shuai Zhao +97
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the c…
Mathematical Models of Overparameterized Neural Networks
Cong Fang, Hanze Dong, Tong Zhang
Deep learning has received considerable empirical successes in recent years. However, while many ad hoc tricks have been discovered by practitioners, until recently, there has been…
Improved Analysis of Clipping Algorithms for Non-convex Optimization
Bohang Zhang, Jikai Jin, Cong Fang +1
Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem. Recently, \citet{zhang2019gradient…
Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks
Cong Fang, Jason D. Lee, Pengkun Yang +1
This paper proposes a new mean-field framework for over-parameterized deep neural networks (DNNs), which can be used to analyze neural network training. In this framework, a DNN is…
Convex Formulation of Overparameterized Deep Neural Networks
Cong Fang, Yihong Gu, Weizhong Zhang +1
Analysis of over-parameterized neural networks has drawn significant attention in recentyears. It was shown that such systems behave like convex systems under various restrictedset…
Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature Representations
Cong Fang, Hanze Dong, Tong Zhang
Recently, over-parameterized neural networks have been extensively analyzed in the literature. However, the previous studies cannot satisfactorily explain why fully trained neural…