104 citations · 270 across the 46 of their papers we have counts for
7 papers · 1 filter
Provably Learning from Modern Language Models via Low Logit Rank
Noah Golowich, Allen Liu, Abhishek Shetty
While modern language models and their inner workings are incredibly complex, recent work (Golowich, Liu & Shetty; 2025) has proposed a simple and potentially tractable abstraction…
Improved Pseudorandom Codes from Permuted Puzzles
Miranda Christ, Noah Golowich, Sam Gunn +2
Watermarks are an essential tool for identifying AI-generated content. Recently, Christ and Gunn (CRYPTO '24) introduced pseudorandom error-correcting codes (PRCs), which are equiv…
Sequences of Logits Reveal the Low Rank Structure of Language Models
Noah Golowich, Allen Liu, Abhishek Shetty
A major problem in the study of large language models is to understand their inherent low-dimensional structure. We introduce an approach to study the low-dimensional structure of…
The Coverage Principle: How Pre-Training Enables Post-Training
Fan Chen, Audrey Huang, Noah Golowich +5
Language models demonstrate remarkable abilities when pre-trained on large text corpora and fine-tuned for specific tasks, but how and why pre-training shapes the success of the fi…
The Hidden Game Problem
Gon Buzaglo, Noah Golowich, Elad Hazan
This paper investigates a class of games with large strategy spaces, motivated by challenges in AI alignment and language games. We introduce the hidden game problem, where for eac…
High-Dimensional Calibration from Swap Regret
Maxwell Fishelson, Noah Golowich, Mehryar Mohri +1
We study online calibration of multi-dimensional forecasts over an arbitrary convex set relative to an arbitrary norm . We connect this to externa…
The Role of Sparsity for Length Generalization in Transformers
Noah Golowich, Samy Jelassi, David Brandfonbrener +2
Training large language models to predict beyond their training context lengths has drawn much attention in recent years, yet the principles driving such behavior of length general…