111 citations · 753 across the 34 of their papers we have counts for
8 papers · 1 filter
Achieving Adversarial Robustness Requires An Active Teacher
Chao Ma, Lexing Ying
A new understanding of adversarial examples and adversarial robustness is proposed by decoupling the data generator and the label generator (which we call the teacher). In our fram…
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
Pan Zhou, Jiashi Feng, Chao Ma +3
It is not clear yet why ADAM-alike adaptive gradient algorithms suffer from worse generalization performance than SGD despite their faster training speed. This work aims to provide…
Complexity Measures for Neural Networks with General Activation Functions Using Path-based Norms
Zhong Li, Chao Ma, Lei Wu
A simple approach is proposed to obtain complexity controls for neural networks with general activation functions. The approach is motivated by approximating the general activation…
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
Weinan E, Chao Ma, Stephan Wojtowytsch +1
The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-ba…
A Qualitative Study of the Dynamic Behavior for Adaptive Gradient Algorithms
Chao Ma, Lei Wu, Weinan E
The dynamic behavior of RMSprop and Adam algorithms is studied through a combination of careful numerical experiments and theoretical explanations. Three types of qualitative featu…
The Slow Deterioration of the Generalization Error of the Random Feature Model
Chao Ma, Lei Wu, Weinan E
The random feature model exhibits a kind of resonance behavior when the number of parameters is close to the training sample size. This behavior is characterized by the appearance…
The Quenching-Activation Behavior of the Gradient Descent Dynamics for Two-layer Neural Network Models
Chao Ma, Lei Wu, Weinan E
A numerical and phenomenological study of the gradient descent (GD) algorithm for training two-layer neural network models is carried out for different parameter regimes when the t…
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth
Yiping Lu, Chao Ma, Yulong Lu +2
Training deep neural networks with stochastic gradient descent (SGD) can often achieve zero training loss on real-world tasks although the optimization landscape is known to be hig…