7 papers
Agentic AI Workload Characteristics
Yichao Yuan, Ankita Nayak, Souvik Kundu +1
Agentic AI shifts LLM serving from isolated prompt-generation requests to stateful, multi-turn executions that repeatedly invoke the model, call tools, and grow context over time.…
One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness
Erfan Baghaei Potraghloo, Seyedarmin Azizi, Souvik Kundu +1
Instruction-tuned large language models produce helpful, structured responses, but how robust is this helpfulness under trivial constraints? We show that simple lexical constraints…
SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models
Jiayi Tian, Seyedarmin Azizi, Yequan Zhao +7
Large reasoning models (LRMs) often incur significant key-value (KV) cache overhead, due to their linear growth with the verbose chain-of-thought (CoT) reasoning. This incurs both…
RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning
Jiayi Tian, Yupeng Su, Ryan Solgi +2
Large reasoning models (LRMs) enhance problem-solving capabilities by generating explicit multi-step chains of thought (CoT) reasoning; however, they incur substantial inference la…
Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration
Jucheng Shen, Gaurav Sarkar, Yeonju Ro +4
We present CadLLM, a training-free method to accelerate the inference throughput of diffusion-based LLMs (dLLMs). We first investigate the dynamic nature of token unmasking confide…
Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning
Seyedarmin Azizi, Erfan Baghaei Potraghloo, Minoo Ahmadi +2
Many recent reasoning gains in large language models can be explained as distribution sharpening: biasing generation toward high-likelihood trajectories already supported by the pr…