7 papers
MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models
Jinsong Shu, Chenyang Wu, Zhongle Xie +2
Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a m…
TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs
Sibo Xiao, Jinyuan Fu, Zhongle Xie +1
Accelerating the inference of large language models (LLMs) has been a critical challenge in generative AI. Speculative decoding (SD) substantially improves LLM inference efficiency…
SVFusion: A CPU-GPU Co-Processing Architecture for Large-Scale Real-Time Vector Search
Yuchen Peng, Dingyu Yang, Zhongle Xie +4
Approximate Nearest Neighbor Search (ANNS) underpins modern applications such as information retrieval and recommendation. With the rapid growth of vector data, efficient indexing…
SafeLoad: Efficient Admission Control Framework for Identifying Memory-Overloading Queries in Cloud Data Warehouses
Yifan Wu, Yuhan Li, Zhenhua Wang +8
Memory overload is a common form of resource exhaustion in cloud data warehouses. When database queries fail due to memory overload, it not only wastes critical resources such as C…
DeXOR: Enabling XOR in Decimal Space for Streaming Lossless Compression of Floating-point Data
Chuanyi Lv, Huan Li, Dingyu Yang +3
With streaming floating-point numbers being increasingly prevalent, effective and efficient compression of such data is critical. Compression schemes must be able to exploit the si…
MorphingDB: A Task-Centric AI-Native DBMS for Model Management and Inference
Wu Sai, Xia Ruichen, Yang Dingyu +9
The increasing demand for deep neural inference within database environments has driven the emergence of AI-native DBMSs. However, existing solutions either rely on model-centric d…