243 citations · 392 across the 23 of their papers we have counts for
10 papers · 1 filter
Partial Feedback Online Learning
Shihao Shao, Cong Fang, Zhouchen Lin +1
We study a new learning protocol, termed partial-feedback online learning, where each instance admits a set of acceptable labels, but the learner observes only one acceptable label…
CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity
Pengyun Yue, Hanzhen Zhao, Cong Fang +4
With distributed machine learning being a prominent technique for large-scale machine learning tasks, communication complexity has become a major bottleneck for speeding up trainin…
Policy Representation via Diffusion Probability Model for Reinforcement Learning
Long Yang, Zhixiong Huang, Fenghao Lei +6
Popular reinforcement learning (RL) algorithms tend to produce a unimodal policy distribution, which weakens the expressiveness of complicated policy and decays the ability of expl…
A Roadmap for Big Model
Sha Yuan, Hanyu Zhao, Shuai Zhao +97
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the c…
Mathematical Models of Overparameterized Neural Networks
Cong Fang, Hanze Dong, Tong Zhang
Deep learning has received considerable empirical successes in recent years. However, while many ad hoc tricks have been discovered by practitioners, until recently, there has been…
Improved Analysis of Clipping Algorithms for Non-convex Optimization
Bohang Zhang, Jikai Jin, Cong Fang +1
Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem. Recently, \citet{zhang2019gradient…
Convex Formulation of Overparameterized Deep Neural Networks
Cong Fang, Yihong Gu, Weizhong Zhang +1
Analysis of over-parameterized neural networks has drawn significant attention in recentyears. It was shown that such systems behave like convex systems under various restrictedset…
Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature Representations
Cong Fang, Hanze Dong, Tong Zhang
Recently, over-parameterized neural networks have been extensively analyzed in the literature. However, the previous studies cannot satisfactorily explain why fully trained neural…
Hessian-Aware Zeroth-Order Optimization for Black-Box Adversarial Attack
Haishan Ye, Zhichao Huang, Cong Fang +2
Zeroth-order optimization is an important research topic in machine learning. In recent years, it has become a key tool in black-box adversarial attack to neural network based imag…
Lifted Proximal Operator Machines
Jia Li, Cong Fang, Zhouchen Lin
We propose a new optimization method for training feed-forward neural networks. By rewriting the activation function as an equivalent proximal operator, we approximate a feed-forwa…