5 papers
Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch
Ziyang Zhang, Xinheng Ding, Jiayi Yuan +4
Deterministic inference is increasingly critical for large language model (LLM) applications such as LLM-as-a-judge evaluation, multi-agent systems, and Reinforcement Learning (RL)…
BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
Jiayi Yuan, Cameron Shinn, Kai Xu +19
The growing demand for long-context inference capabilities in Large Language Models (LLMs) has intensified the computational and memory bottlenecks inherent to the self-attention m…
Generating Piano Music with Transformers: A Comparative Study of Scale, Data, and Metrics
Jonathan Lehmkuhl, Ãbel Ilyés-Kun, Nico Bremes +3
Although a variety of transformers have been proposed for symbolic music generation in recent years, there is still little comprehensive study on how specific design choices affect…
RouterArena: An Open Platform for Comprehensive Comparison of LLM Routers
Yifan Lu, Rixin Liu, Jiayi Yuan +4
Today's LLM ecosystem comprises a wide spectrum of models that differ in size, capability, and cost. No single model is optimal for all scenarios; hence, LLM routers have become es…
Robot Learning with Super-Linear Scaling
Marcel Torne, Arhan Jain, Jiayi Yuan +5
Scaling robot learning requires data collection pipelines that scale favorably with human effort. In this work, we propose Crowdsourcing and Amortizing Human Effort for Real-to-Sim…