5 citations · 5 across the 5 of their papers we have counts for
5 papers
Express Language Modeling
Albert Gong, Annabelle Michael Carrell, Raaz Dwivedi +1
We introduce a new tool, Express, for converting a non-causal attention approximation into a causal approximation with matching approximation guarantees. When combined with the sta…
Haiku to Opus in Just 10 bits: LLMs Unlock Large Compression Gains
Roy Rinberg, Annabelle Michael Carrell, Simon Henniger +2
We study the compression of LLM-generated text across lossless and lossy regimes, characterizing a compression-compute frontier where more compression is possible at the cost of mo…
Low-Rank Thinning
Annabelle Michael Carrell, Albert Gong, Abhishek Shetty +2
The goal in thinning is to summarize a dataset using a small set of representative points. Remarkably, sub-Gaussian thinning algorithms like Kernel Halving and Compress can match t…
Attribute Diversity Determines the Systematicity Gap in VQA
Ian Berlot-Attwell, Kumar Krishna Agrawal, A. Michael Carrell +2
Although modern neural networks often generalize to new combinations of familiar concepts, the conditions that enable such compositionality have long been an open question. In this…
The Calibration Generalization Gap
A. Michael Carrell, Neil Mallinar, James Lucas +1
Calibration is a fundamental property of a good predictive model: it requires that the model predicts correctly in proportion to its confidence. Modern neural networks, however, pr…