From the 1 of 5 linked papers with an AI index.
5 papers
Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization
Ellen Su, Andres Potapczynski, Shikai Qiu +2
Modern systems are increasingly expected to transfer across tasks not specified during training. What data facilitates generalization in these new, unanticipated settings? One hypo…
Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data
Shikai Qiu, Marc Finzi, Yujia Zheng +2
The paper proposes requential coding, a method where a teacher model selects training samples from the student’s own distribution so that only disagreements need to be encoded, yie…
Compute-Optimal LLMs Provably Generalize Better With Scale
Marc Finzi, Sanyam Kapoor, Diego Granziol +4
Why do larger language models generalize better? To investigate this question, we develop generalization bounds on the pretraining objective of large language models (LLMs) in the…
When Should We Orchestrate Multiple Agents?
Umang Bhatt, Sanyam Kapoor, Mihir Upadhyay +6
Strategies for orchestrating the interactions between multiple agents, both human and artificial, can wildly overestimate performance and underestimate the cost of orchestration. W…
Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices
Andres Potapczynski, Shikai Qiu, Marc Finzi +6
Dense linear layers are the dominant computational bottleneck in large neural networks, presenting a critical need for more efficient alternatives. Previous efforts focused on a sm…