9 papers
RecGPT-Mobile: On-Device Large Language Models for User Intent Understanding in Taobao Feed Recommendation
Bin Zhang, Weipeng Huang, Dimin Wang +9
Predicting a user's next search query from recent interaction behaviors is a critical problem in modern e-commerce systems, particularly in scenarios where user intent evolves rapi…
RetentiveKV: State-Space Memory for Uncertainty-Aware Multimodal KV Cache Eviction
Sihao Liu, YuFan Xiong, Zhonghua Jiang +2
Multimodal Large Language Models face severe challenges in computational efficiency and memory consumption due to the substantial expansion of the visual KV cache when processing l…
Semantic Trimming and Auxiliary Multi-step Prediction for Generative Recommendation
Tianyu Zhan, Kairui Fu, Chengfei Lv +2
Generative Recommendation (GR) has recently transitioned from atomic item-indexing to Semantic ID (SID)-based frameworks to capture intrinsic item relationships and enhance general…
AccKV: Towards Efficient Audio-Video LLMs Inference via Adaptive-Focusing and Cross-Calibration KV Cache Optimization
Zhonghua Jiang, Kui Chen, Kunxi Li +5
Recent advancements in Audio-Video Large Language Models (AV-LLMs) have enhanced their capabilities in tasks like audio-visual question answering and multimodal dialog systems. Vid…
FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
Kunxi Li, Yufan Xiong, Zhonghua Jiang +4
Traditional KV cache eviction strategies, which discard less critical KV-pairs based on attention scores, often degrade generation quality, causing context loss or hallucinations.…
PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Models
Zhonghua Jiang, Kunxi Li, Yiyun Zhou +4
Vision-Language Large Models (VLLMs) face significant efficiency challenges when processing high-resolution inputs. The quadratic complexity in attention and autoregressive generat…