11 citations · 43 across the 9 of their papers we have counts for
9 papers
Cosine-Gated Adam-Decay: Drop-In Staleness-Aware Outer Optimization for Decoupled DiLoCo
Vatsal Shah, Jiahao Sun
Asynchronous DiLoCo systems may receive pseudo-gradients computed several outer rounds earlier, yet the standard Nesterov outer optimizer does not explicitly condition its update o…
Recoverability Landscape of Tree Structured Markov Random Fields under Symmetric Noise
Ashish Katiyar, Soumya Basu, Vatsal Shah +1
We study the problem of learning tree-structured Markov random fields (MRF) on discrete random variables with common support when the observations are corrupted by a -ary symmet…
On Generalization of Adaptive Methods for Over-parameterized Linear Regression
Vatsal Shah, Soumya Basu, Anastasios Kyrillidis +1
Over-parameterization and adaptive methods have played a crucial role in the success of deep learning in the last decade. The widespread use of over-parameterization has forced us…
Robust Estimation of Tree Structured Ising Models
Ashish Katiyar, Vatsal Shah, Constantine Caramanis
We consider the task of learning Ising models when the signs of different random variables are flipped independently with possibly unequal, unknown probabilities. In this paper, we…
Choosing the Sample with Lowest Loss makes SGD Robust
Vatsal Shah, Xiaoxia Wu, Sujay Sanghavi
The presence of outliers can potentially significantly skew the parameters of machine learning models trained via stochastic gradient descent (SGD). In this paper we propose a simp…
Negative sampling in semi-supervised learning
John Chen, Vatsal Shah, Anastasios Kyrillidis
We introduce Negative Sampling in Semi-Supervised Learning (NS3L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS3L is motivated by the success of ne…
Minimum weight norm models do not always generalize well for over-parameterized problems
Vatsal Shah, Anastasios Kyrillidis, Sujay Sanghavi
This work is substituted by the paper in arXiv:2011.14066. Stochastic gradient descent is the de facto algorithm for training deep neural networks (DNNs). Despite its popularity, i…
Matrix Completion via Factorizing Polynomials
Vatsal Shah, Nikhil Rao, Weicong Ding
Predicting unobserved entries of a partially observed matrix has found wide applicability in several areas, such as recommender systems, computational biology, and computer vision.…
Trading-off variance and complexity in stochastic gradient descent
Vatsal Shah, Megasthenis Asteris, Anastasios Kyrillidis +1
Stochastic gradient descent is the method of choice for large-scale machine learning problems, by virtue of its light complexity per iteration. However, it lags behind its non-stoc…