3 papers
cs.LG2025
Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
Martin Marek, Sanae Lotfi, Aditya Somasundaram +2
Conventional wisdom dictates that small batch sizes make language model pretraining and fine-tuning unstable, motivating gradient accumulation, which trades off the number of optim…
cs.AI2024
Learning Using a Single Forward Pass
Aditya Somasundaram, Pushkal Mishra, Ayon Borthakur
We propose a learning algorithm to overcome the limitations of traditional backpropagation in resource-constrained environments: Solo Pass Embedded Learning Algorithm (SPELA). SPEL…
astro-ph.EP2019
Trans-Neptunian objects found in the first four years of the Dark Energy Survey
Pedro H. Bernardinelli, Gary M. Bernstein, Masao Sako +61
We present a catalog of 316 trans-Neptunian bodies detected by the Dark Energy Survey (DES). These objects include 245 discoveries by DES (139 not previously published) detected in…