2 papers
cs.LG2025
Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
Martin Marek, Sanae Lotfi, Aditya Somasundaram +2
Conventional wisdom dictates that small batch sizes make language model pretraining and fine-tuning unstable, motivating gradient accumulation, which trades off the number of optim…
cs.CL2025
Transformers Boost the Performance of Decision Trees on Tabular Data across Sample Sizes
Mayuka Jayawardhana, Renbo, Samuel Dooley +6
Large language models (LLMs) perform remarkably well on tabular datasets in zero- and few-shot settings, since they can extract meaning from natural language column headers that de…