103 citations · 311 across the 16 of their papers we have counts for
16 papers
Depth Dependence of P Learning Rates in ReLU MLPs
Samy Jelassi, Boris Hanin, Ziwei Ji +3
In this short note we consider random fully connected ReLU networks of width and depth equipped with a mean-field weight initialization. Our purpose is to study the depende…
Reproducibility in Optimization: Theoretical Framework and Limits
Kwangjun Ahn, Prateek Jain, Ziwei Ji +3
We initiate a formal study of reproducibility in optimization. We define a quantitative measure of reproducibility of optimization procedures in the face of noisy or error-prone op…
Agnostic Learnability of Halfspaces via Logistic Loss
Ziwei Ji, Kwangjun Ahn, Pranjal Awasthi +2
We investigate approximation guarantees provided by logistic regression for the fundamental problem of agnostic learning of homogeneous halfspaces. Previously, for a certain broad…
Actor-critic is implicitly biased towards high entropy optimal policies
Yuzheng Hu, Ziwei Ji, Matus Telgarsky
We show that the simplest actor-critic method -- a linear softmax policy updated with TD through interaction with a linear MDP, but featuring no explicit regularization or explorat…
Fast Margin Maximization via Dual Acceleration
Ziwei Ji, Nathan Srebro, Matus Telgarsky
We present and analyze a momentum-based gradient method for training linear classifiers with an exponentially-tailed loss (e.g., the exponential or logistic loss), which maximizes…
Early-stopped neural networks are consistent
Ziwei Ji, Justin D. Li, Matus Telgarsky
This work studies the behavior of shallow ReLU networks trained with the logistic loss via gradient descent on binary classification data where the underlying data distribution is…
Gradient descent follows the regularization path for general losses
Ziwei Ji, Miroslav Dudík, Robert E. Schapire +1
Recent work across many machine learning disciplines has highlighted that standard descent methods, even without explicit regularization, do not merely minimize the training error,…
Directional convergence and alignment in deep learning
Ziwei Ji, Matus Telgarsky
In this paper, we show that although the minimizers of cross-entropy and related classification losses are off at infinity, network weights learned by gradient flow converge in dir…
Neural tangent kernels, transportation mappings, and universal approximation
Ziwei Ji, Matus Telgarsky, Ruicheng Xian
This paper establishes rates of universal approximation for the shallow neural tangent kernel (NTK): network weights are only allowed microscopic changes from random initialization…
Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks
Ziwei Ji, Matus Telgarsky
Recent theoretical work has guaranteed that overparameterized networks trained by gradient descent achieve arbitrarily low training error, and sometimes even low test error. The re…
Approximation power of random neural networks
Bolton Bailey, Ziwei Ji, Matus Telgarsky +1
This paper investigates the approximation power of three types of random neural networks: (a) infinite width networks, with weights following an arbitrary distribution; (b) finite…
Characterizing the implicit bias via a primal-dual analysis
Ziwei Ji, Matus Telgarsky
This paper shows that the implicit bias of gradient descent on linearly separable data is exactly characterized by the optimal solution of a dual optimization problem given by a sm…
Gradient descent aligns the layers of deep linear networks
Ziwei Ji, Matus Telgarsky
This paper establishes risk convergence and asymptotic weight matrix alignment --- a form of implicit regularization --- of gradient flow and gradient descent when applied to deep…
Risk and parameter convergence of logistic regression
Ziwei Ji, Matus Telgarsky
Gradient descent, when applied to the task of logistic regression, outputs iterates which are biased to follow a unique ray defined by the data. The direction of this ray is the ma…
Wikidata Vandalism Detection - The Loganberry Vandalism Detector at WSDM Cup 2017
Qi Zhu, Hongwei Ng, Liyuan Liu +4
Wikidata is the new, large-scale knowledge base of the Wikimedia Foundation. As it can be edited by anyone, entries frequently get vandalized, leading to the possibility that it mi…
Social welfare and profit maximization from revealed preferences
Ziwei Ji, Ruta Mehta, Matus Telgarsky
Consider the seller's problem of finding optimal prices for her (divisible) goods when faced with a set of consumers, given that she can only observe their purchased bundle…