979 citations · 1.5k across the 63 of their papers we have counts for
11 papers · 1 filter
Fast Generalized Matrix Regression with Applications in Machine Learning
Haishan Ye, Shusen Wang, Zhihua Zhang +1
Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation s…
Distillation Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized Neural Network
Bin Dong, Jikai Hou, Yiping Lu +1
Distillation is a method to transfer knowledge from one model to another and often achieves higher accuracy with the same capacity. In this paper, we aim to provide a theoretical u…
Communication-Efficient Local Decentralized SGD Methods
Xiang Li, Wenhao Yang, Shusen Wang +1
Recently, the technique of local updates is a powerful tool in centralized settings to improve communication efficiency via periodical communication. For decentralized settings, it…
A Stochastic Proximal Point Algorithm for Saddle-Point Problems
Luo Luo, Cheng Chen, Yujun Li +2
We consider saddle point problems which objective functions are the average of strongly convex-concave individual components. Recently, researchers exploit variance reduction m…
A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization
Guangzeng Xie, Luo Luo, Zhihua Zhang
This paper studies the lower bound complexity for the optimization problem whose objective function is the average of individual smooth convex functions. We consider the algori…
Towards Better Generalization: BP-SVRG in Training Deep Neural Networks
Hao Jin, Dachao Lin, Zhihua Zhang
Stochastic variance-reduced gradient (SVRG) is a classical optimization method. Although it is theoretically proved to have better convergence performance than stochastic gradient…
On the Convergence of FedAvg on Non-IID Data
Xiang Li, Kaixuan Huang, Wenhao Yang +2
Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\tex…
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems
Tianle Cai, Ruiqi Gao, Jikai Hou +5
First-order methods such as stochastic gradient descent (SGD) are currently the standard algorithm for training deep neural networks. Second-order methods, despite their better con…
A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning
Xiang Li, Wenhao Yang, Zhihua Zhang
We propose and study a general framework for regularized Markov decision processes (MDPs) where the goal is to find an optimal policy that maximizes the expected discounted total r…
Lipschitz Generative Adversarial Nets
Zhiming Zhou, Jiadong Liang, Yuxuan Song +5
In this paper, we study the convergence of generative adversarial networks (GANs) from the perspective of the informativeness of the gradient of the optimal discriminative function…
Do Subsampled Newton Methods Work for High-Dimensional Data?
Xiang Li, Shusen Wang, Zhihua Zhang
Subsampled Newton methods approximate Hessian matrices through subsampling techniques, alleviating the cost of forming Hessian matrices but using sufficient curvature information.…