1 citations · 1 across the 9 of their papers we have counts for
10 papers
BELT : BEhavior-Aligned Low-Rank KV Cache Compression with Learned Top-k Rank Allocation
Yaoyang Liu, Shuwei Shen, Jincheng Tang +5
Long-context inference makes the key-value (KV) cache a major memory bottleneck for large language models. We propose BELT, a low-rank KV cache compression framework. Its key-side…
FlowTrain: Flow-Based Decoupled Training for Industrial-Grade Vision-Language Models
Zhida Jiang, Zhaolong Xing, Yang Pei +14
Industrial-grade distributed training of vision-language models (VLMs) remains far less efficient than that of unimodal LLMs. Existing solutions either follow a monolithic design t…
BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training
Jiaxing Wang, Deping Xiang, Jin Xu +9
As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive lear…
CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
Xiaoyi Huang, Kejia Zhang, Zhiming Luo
Large Vision-Language Models have shown strong multimodal reasoning capabilities, yet they remain susceptible to object hallucinations when language priors dominate insufficient or…
FastOCR: Dynamic Visual Fixation via KV Cache Pruning for Efficient Document Parsing
Zihan Tang, Leqi Shen, Hui Chen +7
Vision-Language Models (VLMs) have shown strong promise on Optical Character Recognition (OCR), yet the sheer number of visual tokens required to encode dense documents incurs proh…
ProServe: Unified Multi-Priority Request Scheduling for LLM Serving
Weizhe Huang, Tao Peng, Tongxuan Liu +4
The widespread deployment of large language models (LLMs) for interactive applications necessitates serving systems that can handle thousands of concurrent requests with diverse Se…
xGR: Efficient Generative Recommendation Serving at Scale
Qingxiao Sun, Tongxuan Liu, Shen Zhang +13
Recommendation system delivers substantial economic benefits by providing personalized predictions. Generative recommendation (GR) integrates LLMs to enhance the understanding of l…
OOCO: Latency-disaggregated Architecture for Online-Offline Co-locate LLM Serving
Siyu Wu, Zihan Tang, Yuting Zeng +5
Large Language Models (LLMs) are increasingly deployed in both latency-sensitive online services and cost-sensitive offline workloads. Co-locating these workloads on shared serving…
HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
Xianzhe Dong, Tongxuan Liu, Yuting Zeng +7
Multimodal Large Language Models (MLLMs) have been rapidly advancing, enabling cross-modal understanding and generation, and propelling artificial intelligence towards artificial g…
Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture
Yu Wu, Tongxuan Liu, Yuting Zeng +6
Existing large language model (LLM) serving systems typically employ Prefill-Decode disaggregated architecture to prevent computational interference between the prefill and decode…