1 citations · 1 across the 8 of their papers we have counts for
10 papers
Cost-free Spectral Estimation for Adaptive Newton--Schulz in Matrix Optimizers
Kristi Topollai, Anna Choromanska
Matrix optimizers such as Muon transform each momentum matrix through an approximate orthogonalization, typically implemented by a small number of Newton-Schulz matrix multiplicati…
Balancing Early Performance Sacrifices with Long-Term Gains: Scaling Learning-Rate Warmup Duration Across Training Horizons
Kristi Topollai, Anna Choromanska
Learning-rate warmup is a standard technique in language-model training, yet its duration remains largely heuristic. Common approaches use either a fixed number of updates or a fix…
Outer-Momentum Restarting in High-Dimensional Two-Phase Optimization
Kristi Topollai, Allan Ma, Tolga Dimlioglu +2
Communication-efficient distributed optimizers such as DiLoCo reduce synchronization costs by letting workers perform many local updates before aggregating their progress with an o…
Worker Disagreement Reveals Sharp Directions in Local SGD
Tolga Dimlioglu, Kristi Topollai, Anna Choromanska
Deep neural network training often exhibits highly anisotropic loss geometry, where a few sharp dominant Hessian directions coexist with a large flatter bulk. Gradients tend to ali…
Understanding Quantization of Optimizer States in LLM Pre-training: Dynamics of State Staleness and Effectiveness of State Resets
Kristi Topollai, Anna Choromanska
Quantizing optimizer states is becoming an important ingredient of memory-efficient large-scale pre-training, but the resulting optimizer dynamics remain only partially understood.…
Streamlining Industrial Contract Management with Retrieval-Augmented LLMs
Kristi Topollai, Tolga Dimlioglu, Anna Choromanska +2
Contract management involves reviewing and negotiating provisions, individual clauses that define rights, obligations, and terms of agreement. During this process, revisions to pro…
Task-Level Contrastiveness for Cross-Domain Few-Shot Learning
Kristi Topollai, Anna Choromanska
Few-shot classification and meta-learning methods typically struggle to generalize across diverse domains, as most approaches focus on a single dataset, failing to transfer knowled…
OncoReason: Structuring Clinical Reasoning in LLMs for Robust and Interpretable Survival Prediction
Raghu Vamshi Hemadri, Geetha Krishna Guruju, Kristi Topollai +1
Predicting cancer treatment outcomes requires models that are both accurate and interpretable, particularly in the presence of heterogeneous clinical data. While large language mod…
Adaptive Memory Momentum via a Model-Based Framework for Deep Learning Optimization
Kristi Topollai, Anna Choromanska
The vast majority of modern deep learning models are trained with momentum-based first-order optimizers. The momentum term governs the optimizer's memory by determining how much ea…
Self-Supervised Representation Learning with Joint Embedding Predictive Architecture for Automotive LiDAR Object Detection
Haoran Zhu, Zhenyuan Dong, Kristi Topollai +2
Recently, self-supervised representation learning relying on vast amounts of unlabeled data has been explored as a pre-training method for autonomous driving. However, directly app…