2 citations · 6 across the 13 of their papers we have counts for
5 papers · 1 filter
Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever
Yixin Chen, Ying Xiong, Shangyu Wu +4
Tool-augmented LLMs invoke external functions to extend their capabilities, but errors in the invocation decision, such as calling a tool when none is needed or omitting a needed c…
AD-EE: Early Exiting for Fast and Reliable Vision-Language Models in Autonomous Driving
Lianming Huang, Haibo Hu, Yufei Cui +4
With the rapid advancement of autonomous driving, deploying Vision-Language Models (VLMs) to enhance perception and decision-making has become increasingly common. However, the rea…
FlexInfer: Breaking Memory Constraint via Flexible and Efficient Offloading for On-Device LLM Inference
Hongchao Du, Shangyu Wu, Arina Kharlamova +2
Large Language Models (LLMs) face challenges for on-device inference due to high memory demands. Traditional methods to reduce memory usage often compromise performance and lack ad…
EvoP: Robust LLM Inference via Evolutionary Pruning
Shangyu Wu, Hongchao Du, Ying Xiong +4
Large Language Models (LLMs) have achieved remarkable success in natural language processing tasks, but their massive size and computational demands hinder their deployment in reso…
AATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization
Junhui He, Junna Xing, Nan Wang +6
Long context large language models (LLMs) pose significant challenges for efficient serving due to the large memory footprint and high access overhead of KV cache. Retrieval-based…