1 citations · 1 across the 3 of their papers we have counts for
5 papers
The Effect of Mini-Batch Noise on the Implicit Bias of Adam
Matias D. Cattaneo, Boris Shigida
With limited high-quality data and growing compute, multi-epoch training is gaining back its importance across sub-areas of deep learning. Adam(W), versions of which are go-to opti…
Modified Loss of Momentum Gradient Descent: Fine-Grained Analysis
Matias D. Cattaneo, Boris Shigida
We analyze gradient descent with Polyak (1964) heavy-ball momentum (HB) whose fixed momentum hyperparameter provides exponential decay of memory. Building on Kovachki…
How Memory in Optimization Algorithms Implicitly Modifies the Loss
Matias D. Cattaneo, Boris Shigida
In modern optimization methods used in deep learning, each update depends on the history of previous iterations, often referred to as memory, and this dependence decays fast as the…
Uniform Estimation and Inference for Nonparametric Partitioning-Based M-Estimators
Matias D. Cattaneo, Yingjie Feng, Boris Shigida
This paper presents uniform estimation and inference theory for a large class of nonparametric partitioning-based M-estimators. The main theoretical results include: (i) uniform co…
Nonlinear Binscatter Methods
Matias D. Cattaneo, Richard K. Crump, Max H. Farrell +1
Binscatters are a powerful tool for empirical work in the social, behavioral, and biomedical sciences. Available tools rely on least squares estimation of the conditional mean. We…