334 citations · 1.4k across the 54 of their papers we have counts for
63 papers
Joules-to-GDP: How Much Economic Value Do AI Systems Produce per Unit of Energy?
Amirreza Zeinali, Jon Saad-Falcon, Avanika Narayan +5
An AI system can produce more economically valuable work while delivering less value for the energy it consumes. We therefore ask: How much economic value does an AI system create…
Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation
Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi +6
Diffusion language models (DLMs) offer parallel sampling for faster generation than autoregressive models, an increasingly important advantage as inference becomes a dominant compu…
ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement Learning
Seil Kang, Hangoo Kang, Tarun Suresh +4
Language models are moving beyond generating answers to pursuing long-horizon goals in interactive environments. Post-training these agents requires long, heterogeneous trajectorie…
Sharpening Tax in Post-Training
Changdae Oh, Qi Zeng, Qi Qi +7
An emerging hypothesis about reinforcement learning (RL) post-training of large language models (LLMs) is that it merely sharpens existing behaviors of a base model, improving sing…
AI as a Compiler: Compiling Triton kernels without the Triton compiler
François Costa, Charly Castes, Thomas Bourgeat +1
Compiler backends are expensive to build and maintain as programming models, workloads, and accelerators evolve. We investigate whether large language models can replace the conven…
Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training
Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang +4
Block diffusion language models (BDLMs) combine autoregressive dependencies across blocks with parallel denoising within blocks, but long-context training is constrained by distrib…
LLM-as-a-Verifier: A General-Purpose Verification Framework
Jacky Kwok, Shulu Li, Pranav Atreya +6
Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the abi…
Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
Shayan Talaei, Abhinav Chinta, Devvrit Khatri +3
Language models deployed in high-stakes roles can potentially favor certain entities, brands, or viewpoints, steering user decisions at scale. Such preferential biases can be intro…
KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
Yunxiang Zhang, Ping Yu, Jianyu Wang +5
Recent large language models (LLMs) can generate custom CUDA kernels that appear to outperform PyTorch on benchmarks such as KernelBench. Building upon this foundational framework,…
Decentralized Multi-Agent Systems with Shared Context
Yuzhen Mao, Azalia Mirhoseini, Jerry Gu +3
Multi-agent systems (MAS) can scale large language model agents on long-horizon tasks by running them in parallel, yet existing designs waste much of this parallelism in bubbles: a…
PolyAlign: Conditional Human-Distribution Alignment
L. D. M. S. Sai Teja, Ufaq Khan, Sathira Silva +2
Post-training methods such as supervised fine-tuning (SFT) and preference optimization typically align language models toward a single global assistant behavior. While effective fo…
Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling
Caleb Winston, Ron Yifeng Wang, Azalia Mirhoseini +1
Computer-use agents (CUAs) automate tasks specified with natural language such as "order the cheapest item from Taco Bell" by generating sequences of calls to tools such as click,…
OpenJarvis: Personal AI, On Personal Devices
Jon Saad-Falcon, Avanika Narayan, Robby Manihani +10
Personal AI stacks, like OpenClaw and Hermes Agent, are becoming central to daily work, yet they route nearly every query (often over sensitive local data) to cloud-hosted frontier…
Federation of Experts: Communication Efficient Distributed Inference for Large Language Models
Muhammad Shahir Abdurrahman, Chun Deng, Azalia Mirhoseini +1
Mixture of experts has emerged as the primary mechanism for making Large Language Models (LLMs) computationally efficient. However, in distributed settings, communicating token emb…
TRACE: Capability-Targeted Agentic Training
Hangoo Kang, Tarun Suresh, Jon Saad-Falcon +1
Models often fail to complete agentic tasks because they lack core capabilities required by the target environment. However, mainstream approaches for addressing these failures typ…
AI+HW 2035: Shaping the Next Decade
Deming Chen, Jason Cong, Azalia Mirhoseini +27
Artificial intelligence (AI) and hardware (HW) are advancing at unprecedented rates, yet their trajectories have become inseparably intertwined. The global research community lacks…
Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment
Jacky Kwok, Xilun Zhang, Mengdi Xu +4
The long-standing vision of general-purpose robots hinges on their ability to understand and act upon natural language instructions. Vision-Language-Action (VLA) models have made r…
Intelligence per Watt: Measuring Intelligence Efficiency of Local AI
Jon Saad-Falcon, Avanika Narayan, Hakki Orhun Akengin +12
Large language model (LLM) queries are predominantly processed by frontier models in centralized cloud infrastructure. Demand growth strains this paradigm faster than providers can…
On the Role of Temperature Sampling in Test-Time Scaling
Yuheng Wu, Azalia Mirhoseini, Thierry Tambe
Large language models (LLMs) can improve reasoning at inference time through test-time scaling (TTS), where multiple reasoning traces are generated and the best one is selected. Pr…
ForTIFAI: Fending Off Recursive Training Induced Failure for AI Model Collapse
Soheil Zibakhsh Shabgahi, Pedram Aghazadeh, Azalia Mirhoseini +1
The increasing reliance on generative AI models is rapidly increasing the volume of synthetic data, with some projections suggesting that most available new data for training could…
Astra: A Multi-Agent System for GPU Kernel Performance Optimization
Anjiang Wei, Tianran Sun, Yogesh Seenichamy +5
GPU kernel optimization has long been a central challenge at the intersection of high-performance computing and machine learning. Efficient kernels are crucial for accelerating lar…
RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models
Jacky Kwok, Christopher Agia, Rohan Sinha +5
Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in visuomotor control, yet ensuring their robustness in unstructured real-world environments remains a…
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
Gheorghe Comanici, Eric Bieber, Mike Schaekermann +3431
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our…
Cartridges: Lightweight and general-purpose long context representations via self-study
Sabri Eyuboglu, Ryan Ehrlich, Simran Arora +8
Large language models are often used to answer queries grounded in large text corpora (e.g. codebases, legal documents, or chat histories) by placing the entire corpus in the conte…