13 citations · 23 across the 6 of their papers we have counts for
7 papers
AccKV: Towards Efficient Audio-Video LLMs Inference via Adaptive-Focusing and Cross-Calibration KV Cache Optimization
Zhonghua Jiang, Kui Chen, Kunxi Li +5
Recent advancements in Audio-Video Large Language Models (AV-LLMs) have enhanced their capabilities in tasks like audio-visual question answering and multimodal dialog systems. Vid…
FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
Kunxi Li, Yufan Xiong, Zhonghua Jiang +4
Traditional KV cache eviction strategies, which discard less critical KV-pairs based on attention scores, often degrade generation quality, causing context loss or hallucinations.…
PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Models
Zhonghua Jiang, Kunxi Li, Yiyun Zhou +4
Vision-Language Large Models (VLLMs) face significant efficiency challenges when processing high-resolution inputs. The quadratic complexity in attention and autoregressive generat…
MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection
Zhengxiang Huang, Chaoyue Niu, Zhaode Wang +8
As the demand for on-device Large Language Model (LLM) inference grows, energy efficiency has become a major concern, especially for battery-limited mobile devices. Our analysis sh…
MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
Zhaode Wang, Jingbang Yang, Xinyu Qian +4
Large language models (LLMs) have demonstrated exceptional performance across a variety of tasks. However, their substantial scale leads to significant computational resource consu…
MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
Kunxi Li, Zhonghua Jiang, Zhouzhou Shen +5
This paper introduces MadaKV, a modality-adaptive key-value (KV) cache eviction strategy designed to enhance the efficiency of multimodal large language models (MLLMs) in long-cont…