133 citations · 160 across the 6 of their papers we have counts for
9 papers
Stability Analysis and Generalization Bounds of Adversarial Training
Jiancong Xiao, Yanbo Fan, Ruoyu Sun +2
In adversarial machine learning, deep neural networks can fit the adversarial examples on the training dataset but have poor generalization ability on the test set. This phenomenon…
When Expressivity Meets Trainability: Fewer than Neurons Can Work
Jiawei Zhang, Yushun Zhang, Mingyi Hong +2
Modern neural networks are often quite wide, causing large memory and computation costs. It is thus of great interest to train a narrower network. However, training narrow neural n…
Federated Semi-Supervised Learning with Class Distribution Mismatch
Zhiguo Wang, Xintong Wang, Ruoyu Sun +1
Many existing federated learning (FL) algorithms are designed for supervised learning tasks, assuming that the local data owned by the clients are well labeled. However, in many pr…
Achieving Small Test Error in Mildly Overparameterized Neural Networks
Shiyu Liang, Ruoyu Sun, R. Srikant
Recent theoretical works on over-parameterized neural nets have focused on two aspects: optimization and generalization. Many existing works that study optimization and generalizat…
Optimization for deep learning: theory and algorithms
Ruoyu Sun
When and why can a neural network be successfully trained? This article provides an overview of optimization algorithms and theory for training neural networks. First, we discuss t…
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
Xiangyi Chen, Sijia Liu, Ruoyu Sun +1
This paper studies a class of adaptive gradient based momentum algorithms that update the search directions and learning rates simultaneously using past gradients. This class, whic…