5 citations · 8 across the 14 of their papers we have counts for
4 papers · 1 filter
KV-streams for Efficient Compaction in Agentic Reinforcement Learning
Emiliano Penaloza, Dane Malenfant, Dheeraj Vattikonda +15
Scaling the horizon of agentic LLMs is bottlenecked by the need to fit ever longer context traces in GPU memory. Context compaction has been the most popular mechanism to alleviate…
Scale Dependent Data Duplication
Joshua Kazdan, Noam Levi, Rylan Schaeffer +6
Data duplication during pretraining can degrade generalization and lead to memorization, motivating aggressive deduplication pipelines. However, at web scale, it is unclear what co…
Quantifying the Effect of Test Set Contamination on Generative Evaluations
Rylan Schaeffer, Joshua Kazdan, Baber Abbasi +8
As frontier AI systems are pretrained on web-scale data, test set contamination has become a critical concern for accurately assessing their capabilities. While research has thorou…
BigDocs: An Open Dataset for Training Multimodal Models on Document and Code Tasks
Juan Rodriguez, Xiangru Jian, Siba Smarak Panigrahi +40
Multimodal AI has the potential to significantly enhance document-understanding tasks, such as processing receipts, understanding workflows, extracting data from documents, and sum…