2.6k citations · 3.1k across the 36 of their papers we have counts for
36 papers
Efficient Sampling on Riemannian Manifolds via Langevin MCMC
Xiang Cheng, Jingzhao Zhang, Suvrit Sra
We study the task of efficiently sampling from a Gibbs distribution over a Riemannian manifold via (geometric) Langevin MCMC; this algorithm involves…
Fast and Multiphase Rates for Nearest Neighbor Classifiers
Pengkun Yang, Jingzhao Zhang
We study the scaling of classification error rates with respect to the size of the training dataset. In contrast to classical results where rates are minimax optimal for a problem…
Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles
Lesi Chen, Yaohua Ma, Jingzhao Zhang
In this work, we consider bilevel optimization when the lower-level problem is strongly convex. Recent works show that with a Hessian-vector product (HVP) oracle, one can provably…
Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions
Xiang Cheng, Bohan Wang, Jingzhao Zhang +1
MCMC algorithms offer empirically efficient tools for sampling from a target distribution . However, on the theory side, MCMC algorithms suffer from slow…
Online Control with Adversarial Disturbance for Continuous-time Linear Systems
Jingwei Li, Jing Dong, Can Chang +2
We study online control for continuous-time linear systems with finite sampling rates, where the objective is to design an online procedure that learns under non-stochastic noise a…
Lower Generalization Bounds for GD and SGD in Smooth Stochastic Convex Optimization
Peiyuan Zhang, Jiaye Teng, Jingzhao Zhang
This work studies the generalization error of gradient methods. More specifically, we focus on how training steps and step-size might affect generalization in smooth stocha…
On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis
Lesi Chen, Jing Xu, Jingzhao Zhang
Bilevel optimization reveals the inner structure of otherwise oblique optimization problems, such as hyperparameter tuning, neural architecture search, and meta-learning. A common…
Online Policy Optimization for Robust MDP
Jing Dong, Jingwei Li, Baoxiang Wang +1
Reinforcement learning (RL) has exceeded human performance in many synthetic settings such as video games and Go. However, real-world deployment of end-to-end RL models is less com…
Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models
Kaiyue Wen, Jiaye Teng, Jingzhao Zhang
Studies on benign overfitting provide insights for the success of overparameterized deep learning models. In this work, we examine whether overfitting is truly benign in real-world…
Theory and Algorithms for Diffusion Processes on Riemannian Manifolds
Xiang Cheng, Jingzhao Zhang, Suvrit Sra
We study geometric stochastic differential equations (SDEs) and their approximations on Riemannian manifolds. In particular, we introduce a simple new construction of geometric SDE…
Understanding the unstable convergence of gradient descent
Kwangjun Ahn, Jingzhao Zhang, Suvrit Sra
Most existing analyses of (stochastic) gradient descent rely on the condition that for -smooth costs, the step size is less than . However, many works have observed that in…
Sion's Minimax Theorem in Geodesic Metric Spaces and a Riemannian Extragradient Algorithm
Peiyuan Zhang, Jingzhao Zhang, Suvrit Sra
Deciding whether saddle points exist or are approximable for nonconvex-nonconcave problems is usually intractable. This paper takes a step towards understanding a broad class of no…
EVBattery: A Large-Scale Electric Vehicle Dataset for Battery Health and Capacity Estimation
Haowei He, Jingzhao Zhang, Yanan Wang +9
Electric vehicles (EVs) play an important role in reducing carbon emissions. As EV adoption accelerates, safety issues caused by EV batteries have become an important research topi…
Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective
Jingzhao Zhang, Haochuan Li, Suvrit Sra +1
This work examines the deep disconnect between existing theoretical analyses of gradient-based algorithms and the practice of training deep neural networks. Specifically, we provid…
Provably Efficient Algorithms for Multi-Objective Competitive RL
Tiancheng Yu, Yi Tian, Jingzhao Zhang +1
We study multi-objective reinforcement learning (RL) where an agent's reward is represented as a vector. In settings where an agent competes against opponents, its performance is m…
Coping with Label Shift via Distributionally Robust Optimisation
Jingzhao Zhang, Aditya Menon, Andreas Veit +3
The label shift problem refers to the supervised learning setting where the train and test label distributions do not match. Existing work addressing label shift usually assumes ac…
Beyond Worst-Case Analysis in Stochastic Approximation: Moment Estimation Improves Instance Complexity
Jingzhao Zhang, Hongzhou Lin, Subhro Das +2
We study oracle complexity of gradient based methods for stochastic approximation problems. Though in many settings optimal algorithms and tight lower bounds are known for such pro…
Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions
Jingzhao Zhang, Hongzhou Lin, Stefanie Jegelka +2
We provide the first non-asymptotic analysis for finding stationary points of nonsmooth, nonconvex functions. In particular, we study the class of Hadamard semi-differentiable func…
Why are Adaptive Methods Good for Attention Models?
Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit +4
While stochastic gradient descent (SGD) is still the \emph{de facto} algorithm in deep learning, adaptive methods like Clipped SGD/Adam have been observed to outperform SGD across…
Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization
Jingzhao Zhang, Suvrit Sra, Ali Jadbabaie
We study gradient-based optimization methods obtained by direct Runge-Kutta discretization of the ordinary differential equation (ODE) describing the movement of a heavy-ball under…
Why gradient clipping accelerates training: A theoretical justification for adaptivity
Jingzhao Zhang, Tianxing He, Suvrit Sra +1
We provide a theoretical explanation for the effectiveness of gradient clipping in training deep neural networks. The key ingredient is a new smoothness condition derived from prac…
Intrinsic long range antiferromagnetic coupling in dilutely V doped CuInTe
Weiyi Gong, Ching-Him Leung, Chuen-Keung Sin +4
Despite the various magnetic orders mediated by superexchange mechanism, the existence of a long range antiferromagnetic (AFM) coupling is unknown. Based on DFT calculations, we di…
A Probe Towards Understanding GAN and VAE Models
Lu Mi, Macheng Shen, Jingzhao Zhang
This project report compares some known GAN and VAE models proposed prior to 2017. There has been significant progress after we finished this report. We upload this report as an in…
R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate
Jingzhao Zhang, Hongyi Zhang, Suvrit Sra
We study smooth stochastic optimization problems on Riemannian manifolds. Via adapting the recently proposed SPIDER algorithm \citep{fang2018spider} (a variance reduced stochastic…