6 papers
ActTune: Action-Aware Precision and GPU Operating-Point Adaptation for Energy-Efficient Vision-Language-Action Inference
Zou Qingyun, Bin Gao, Wenju Zhao +3
Vision-language-action (VLA) policies repeatedly invoke inference to control robots, making graphics processing unit (GPU) energy a recurring cost of task execution. Reducing energ…
ECO: Energy-Oriented Configuration Optimization for Attention FFN Disaggregated LLM Serving
Zou Qingyun, Bin Gao, Zhuobin Huang +3
Energy-efficient LLM serving requires minimizing serving GPU energy while meeting latency and throughput service-level objectives (SLOs). Attention--FFN disaggregation (AFD) enable…
SPLASH: Co-Designing Sparse Attention with High-Bandwidth Flash for Efficient Long-Context Inference
Aditya Anirudh Jonnalagadda, Agasthi Haputhanthri, Pranav Dangi +5
The key-value (KV) cache has become the dominant consumer of memory in large language model (LLM) serving systems as context lengths, concurrency, and request lifetimes grow. High-…
SAEM: Stage-Aware Expert Management for Memory-Efficient MoE Inference in Chain-of-Thought Reasoning
Yujie Zhang, Bin Gao, Tulika Mitra
Chain-of-thought (CoT) prompting improves LLM reasoning by decomposing complex problems into intermediate steps, but its sequential nature increases decoding latency and memory usa…
Accelerating Multi-Scale Deformable Attention Using Near-Memory-Processing Architecture
Huize Li, Qinggang Wang, Bing Gao +3
Multi Scale Deformable Attention (MSDAttn) has become a fundamental component in various vision tasks due to its effective multi scale grid sampling (MSGS). However, its reliance o…
AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference
Zhuomin He, Yizhen Yao, Pengfei Zuo +4
Long-context large language models (LLMs) inference is increasingly critical, motivating a number of studies devoted to alleviating the substantial storage and computational costs…