414 citations · 723 across the 50 of their papers we have counts for
10 papers · 1 filter
KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning
Eric Zimmermann, Harley Wiltzer, Justin Szeto +2
Breakthroughs in self-supervised Joint-Embedding Predictive Architectures (JEPAs) have established that regularizing Euclidean representations toward isotropic Gaussian priors yiel…
Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
Rachit Bansal, Aston Zhang, Rishabh Tiwari +8
Progress on training and architecture strategies has enabled LLMs with millions of tokens in context length. However, empirical evidence suggests that such long-context LLMs can co…
Adam or Gauss-Newton? A Comparative Study In Terms of Basis Alignment and SGD Noise
Bingbin Liu, Rachit Bansal, Depen Morwani +3
Preconditioned methods are central to deep learning optimization. Predominant approaches include computationally light diagonal preconditioners such as Adam, which rely on gradient…
Boomerang Distillation Enables Zero-Shot Model Size Interpolation
Sara Kangaslahti, Nihal V. Nayak, Jonathan Geuter +3
Large language models (LLMs) are typically deployed under diverse memory and compute constraints. Existing approaches build model families by training each size independently, whic…
Bispectral OT: Dataset Comparison using Symmetry-Aware Optimal Transport
Annabel Ma, Kaiying Hou, David Alvarez-Melis +1
Optimal transport (OT) is a widely used technique in machine learning, graphics, and vision that aligns two distributions or datasets using their relative geometry. In symmetry-ric…
Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles
Antara Raaghavi Bhattacharya, Isabel Papadimitriou, Kathryn Davidson +1
Across languages, numeral systems vary widely in how they construct and combine numbers. While humans consistently learn to navigate this diversity, large language models (LLMs) st…