239 citations · 404 across the 7 of their papers we have counts for
14 papers
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers
Guodong Zhang, Aleksandar Botev, James Martens
Training very deep neural networks is still an extremely challenging task. The common solution is to use shortcut connections and normalization layers, which are both crucial ingre…
Learning to Give Checkable Answers with Prover-Verifier Games
Cem Anil, Guodong Zhang, Yuhuai Wu +1
Our ability to know when to trust the decisions made by machine learning systems has not kept up with the staggering improvements in their performance, limiting their applicability…
Differentiable Annealed Importance Sampling and the Perils of Gradient Noise
Guodong Zhang, Kyle Hsu, Jianing Li +2
Annealed importance sampling (AIS) and related algorithms are highly effective tools for marginal likelihood estimation, but are not fully differentiable due to the use of Metropol…
On the Suboptimality of Negative Momentum for Minimax Optimization
Guodong Zhang, Yuanhao Wang
Smooth game optimization has recently attracted great interest in machine learning as it generalizes the single-objective optimization paradigm. However, game dynamics is more comp…
Picking Winning Tickets Before Training by Preserving Gradient Flow
Chaoqi Wang, Guodong Zhang, Roger Grosse
Overparameterization has been shown to benefit both the optimization and generalization of neural networks, but large networks are resource hungry at both training and test time. N…
On Solving Minimax Optimization Locally: A Follow-the-Ridge Approach
Yuanhao Wang, Guodong Zhang, Jimmy Ba
Many tasks in modern machine learning can be formulated as finding equilibria in \emph{sequential} games. In particular, two-player zero-sum sequential games, also known as minimax…