13 citations · 13 across the 6 of their papers we have counts for
3 papers · 1 filter
ReCal: Reward Calibration for RL-based LLM Routing
Qihang Yu, Hanwen Tong, Zhengqi Zhang +5
Large language model (LLM) routing has emerged as an effective paradigm for leveraging the complementary strengths of multiple LLMs through dynamic model and reasoning-strategy sel…
MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
Zhaode Wang, Jingbang Yang, Xinyu Qian +4
Large language models (LLMs) have demonstrated exceptional performance across a variety of tasks. However, their substantial scale leads to significant computational resource consu…
MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
Kunxi Li, Zhonghua Jiang, Zhouzhou Shen +5
This paper introduces MadaKV, a modality-adaptive key-value (KV) cache eviction strategy designed to enhance the efficiency of multimodal large language models (MLLMs) in long-cont…