6 papers
Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning
Tolga Dimlioglu, Anna Choromanska
We study centralized distributed data parallel training of deep neural networks (DNNs), aiming to improve the trade-off between communication efficiency and model performance of th…
Outer-Momentum Restarting in High-Dimensional Two-Phase Optimization
Kristi Topollai, Allan Ma, Tolga Dimlioglu +2
Communication-efficient distributed optimizers such as DiLoCo reduce synchronization costs by letting workers perform many local updates before aggregating their progress with an o…
Worker Disagreement Reveals Sharp Directions in Local SGD
Tolga Dimlioglu, Kristi Topollai, Anna Choromanska
Deep neural network training often exhibits highly anisotropic loss geometry, where a few sharp dominant Hessian directions coexist with a large flatter bulk. Gradients tend to ali…
Scaling-Aware Data Selection for End-to-End Autonomous Driving Systems
Tolga Dimlioglu, Nadine Chang, Maying Shen +2
Large-scale deep learning models for physical AI applications depend on diverse training data collection efforts. These models and correspondingly, the training data, must address…
Streamlining Industrial Contract Management with Retrieval-Augmented LLMs
Kristi Topollai, Tolga Dimlioglu, Anna Choromanska +2
Contract management involves reviewing and negotiating provisions, individual clauses that define rights, obligations, and terms of agreement. During this process, revisions to pro…
Data Scaling Laws for End-to-End Autonomous Driving
Alexander Naumann, Xunjiang Gu, Tolga Dimlioglu +7
Autonomous vehicle (AV) stacks have traditionally relied on decomposed approaches, with separate modules handling perception, prediction, and planning. However, this design introdu…