436 citations · 2.1k across the 92 of their papers we have counts for
5 papers · 1 filter
Federation over Text: Insight Sharing for Multi-Agent Reasoning
Dixi Yao, Tahseen Rabbani, Manzil Zaheer +1
Modern agents specialize in varying domains while there is no clear approach combining different domain skills. We propose a federated learning-like framework, Federation over Text…
Differentially Private Model Merging
Qichuan Yin, Manzil Zaheer, Tian Li
In machine learning, privacy requirements at inference or deployment time often evolve due to changing policies, regulations, or user preferences. In this work, we aim to construct…
Scaling Test-Time Compute for Agentic Coding
Joongwon Kim, Wannan Yang, Kelvin Niu +13
Test-time scaling has become a powerful way to improve large language models. However, existing methods are best suited to short, bounded outputs that can be directly compared, ran…
Interleaved Head Attention
Sai Surya Duvvuri, Chanakya Ekbote, Rachit Bansal +6
Multi-Head Attention (MHA) is the core computational primitive underlying modern Large Language Models (LLMs). However, MHA suffers from a fundamental linear scaling limitation: $H…
Asynchronous Heavy-Tailed Optimization
Junfei Sun, Dixi Yao, Xuchen Gong +3
Heavy-tailed stochastic gradient noise, commonly observed in transformer models, can destabilize the optimization process. Recent works mainly focus on developing and understanding…