4 papers
You Need an Encoder for Native Position-Independent Caching
Shiju Zhao, Junhao Hu, Jiaqi Zheng +1
The Key-Value (KV) cache of Large Language Models (LLMs) is prefix-based, making it highly inefficient for processing contexts retrieved in arbitrary order. Position-Independent Ca…
Bi-Level Decision-Focused Causal Learning for Large-Scale Marketing Optimization: Bridging Observational and Experimental Data
Shuli Zhang, Hao Zhou, Jiaqi Zheng +4
Online Internet platforms require sophisticated marketing strategies to optimize user retention and platform revenue -- a classical resource allocation problem. Traditional solutio…
MPIC: Position-Independent Multimodal Context Caching System for Efficient MLLM Serving
Shiju Zhao, Junhao Hu, Rongxiao Huang +2
The context caching technique is employed to accelerate the Multimodal Large Language Model (MLLM) inference by prevailing serving platforms currently. However, this approach merel…
CSR:Achieving 1 Bit Key-Value Cache via Sparse Representation
Hongxuan Zhang, Yao Zhao, Jiaqi Zheng +3
The emergence of long-context text applications utilizing large language models (LLMs) has presented significant scalability challenges, particularly in memory footprint. The linea…