4 papers · 1 filter
Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration
Jucheng Shen, Gaurav Sarkar, Yeonju Ro +4
We present CadLLM, a training-free method to accelerate the inference throughput of diffusion-based LLMs (dLLMs). We first investigate the dynamic nature of token unmasking confide…
Sherlock: Reliable and Efficient Agentic Workflow Execution
Yeonju Ro, Haoran Qiu, Íñigo Goiri +6
With the increasing adoption of large language models (LLM), agentic workflows, which compose multiple LLM calls with tools, retrieval, and reasoning steps, are increasingly replac…
On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention
Yeonju Ro, Zhenyu Zhang, Souvik Kundu +2
Large language models (LLMs) excel at capturing global token dependencies via self-attention but face prohibitive compute and memory costs on lengthy inputs. While sub-quadratic me…
HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training
Geon-Woo Kim, Junbo Li, Shashidhar Gandham +5
Training large language models (LLMs) increasingly relies on geographically distributed accelerators, causing prohibitive communication costs across regions and uneven utilization…