3.7k citations · 5k across the 45 of their papers we have counts for
11 papers · 1 filter
MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flows
Mingxuan Yi, Zhanxing Zhu, Song Liu
The conventional understanding of adversarial training in generative adversarial networks (GANs) is that the discriminator is trained to estimate a divergence, and the generator le…
Neural Approximate Sufficient Statistics for Implicit Models
Yanzhi Chen, Dinghuai Zhang, Michael Gutmann +2
We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of the likelihood function is intractab…
Spherical Motion Dynamics: Learning Dynamics of Neural Network with Normalization, Weight Decay, and SGD
Ruosi Wan, Zhanxing Zhu, Xiangyu Zhang +1
In this work, we comprehensively reveal the learning dynamics of neural network with normalization, weight decay (WD), and SGD (with momentum), named as Spherical Motion Dynamics (…
You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu +2
Deep learning achieves state-of-the-art results in many tasks in computer vision and natural language processing. However, recent works have shown that deep networks can be vulnera…
Neural Control Variates for Variance Reduction
Ruosi Wan, Mingjun Zhong, Haoyi Xiong +1
In statistics and machine learning, approximation of an intractable integration is often achieved by using the unbiased Monte Carlo estimator, but the variances of the estimation a…
The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects
Zhanxing Zhu, Jingfeng Wu, Bing Yu +2
Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general for…
Understanding and Enhancing the Transferability of Adversarial Examples
Lei Wu, Zhanxing Zhu, Cheng Tai +1
State-of-the-art deep neural networks are known to be vulnerable to adversarial examples, formed by applying small but malicious perturbations to the original inputs. Moreover, the…
Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning
Rui Luo, Jianhong Wang, Yaodong Yang +2
We propose a new sampling method, the thermostat-assisted continuously-tempered Hamiltonian Monte Carlo, for Bayesian learning on large datasets and multimodal distributions. It si…
Stochastic Parallel Block Coordinate Descent for Large-scale Saddle Point Problems
Zhanxing Zhu, Amos J. Storkey
We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method us…
Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling
Xiaocheng Shang, Zhanxing Zhu, Benedict Leimkuhler +1
Monte Carlo sampling for Bayesian posterior inference is a common approach used in machine learning. The Markov Chain Monte Carlo procedures that are used are often discrete-time a…
Adaptive Stochastic Primal-Dual Coordinate Descent for Separable Saddle Point Problems
Zhanxing Zhu, Amos J. Storkey
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we…