48 citations · 72 across the 13 of their papers we have counts for
Showing 2023Show all
3 papers · 1 filter
cs.LG2023
Corgi^2: A Hybrid Offline-Online Approach To Storage-Aware Data Shuffling For SGD
Etay Livne, Gal Kaplun, Eran Malach +1
When using Stochastic Gradient Descent (SGD) for training machine learning models, it is often crucial to provide the model with examples sampled at random from the dataset. Howeve…
cs.LG2023★ 1 cited
Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
Nikhil Vyas, Depen Morwani, Rosie Zhao +3
The success of SGD in deep learning has been ascribed by prior works to the implicit bias induced by finite batch sizes ("SGD noise"). While prior works focused on offline learning…
cs.LG2023
Less is More: Selective Layer Finetuning with SubTuning
Gal Kaplun, Andrey Gurevich, Tal Swisa +3
Finetuning a pretrained model has become a standard approach for training neural networks on novel tasks, resulting in fast convergence and improved performance. In this work, we s…