7 papers
QCATS: Query Context-Aware Transformer Slicing for Efficient Predictive Query Processing
Yueying Li, Zhongle Xie, Ke Chen +1
In-database predictive query processing increasingly applies Transformer-based models within relational pipelines. However, existing in-database inference typically exposes only tu…
MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models
Jinsong Shu, Chenyang Wu, Zhongle Xie +2
Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a m…
SVFusion: A CPU-GPU Co-Processing Architecture for Large-Scale Real-Time Vector Search
Yuchen Peng, Dingyu Yang, Zhongle Xie +4
Approximate Nearest Neighbor Search (ANNS) underpins modern applications such as information retrieval and recommendation. With the rapid growth of vector data, efficient indexing…
SafeLoad: Efficient Admission Control Framework for Identifying Memory-Overloading Queries in Cloud Data Warehouses
Yifan Wu, Yuhan Li, Zhenhua Wang +8
Memory overload is a common form of resource exhaustion in cloud data warehouses. When database queries fail due to memory overload, it not only wastes critical resources such as C…
MorphingDB: A Task-Centric AI-Native DBMS for Model Management and Inference
Wu Sai, Xia Ruichen, Yang Dingyu +9
The increasing demand for deep neural inference within database environments has driven the emergence of AI-native DBMSs. However, existing solutions either rely on model-centric d…
TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs
Sibo Xiao, Jinyuan Fu, Zhongle Xie +1
Accelerating the inference of large language models (LLMs) has been a critical challenge in generative AI. Speculative decoding (SD) substantially improves LLM inference efficiency…
FloE: On-the-Fly MoE Inference on Memory-constrained GPU
Yuxin Zhou, Zheng Li, Jun Zhang +5
With the widespread adoption of Mixture-of-Experts (MoE) models, there is a growing demand for efficient inference on memory-constrained devices. While offloading expert parameters…