5 papers
LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration
Xuyue Huang, Zhe Chen, Wang Shen +1
Diffusion Transformers (DiTs) have driven substantial progress in image and video generation but suffer from prohibitive computational costs. Feature caching accelerates inference…
GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimization
Ojas Nimase, Zhe Chen, Gengpei Qi +2
Large language models (LLMs) increasingly rank products, documents, and recommendations for user queries, which makes manipulating these rankings a growing concern for fairness and…
Rec-Distill: An Industrial Distillation Pipeline for Large-Scale Recommendation Models
Haoran Ding, Wenlin Zhao, Yuchen Jiang +16
Large recommendation models have demonstrated substantial potential gains under scaling laws, yet these gains are difficult to realize in industrial recommendation systems because…
RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations
Zunhai Su, Zhe Chen, Wang Shen +4
Key-Value (KV) cache facilitates efficient large language models (LLMs) inference by avoiding recomputation of past KVs. As the batch size and context length increase, the oversize…
AKVQ-VL: Attention-Aware KV Cache Adaptive 2-Bit Quantization for Vision-Language Models
Zunhai Su, Wang Shen, Linge Li +4
Vision-language models (VLMs) show remarkable performance in multimodal tasks. However, excessively long multimodal inputs lead to oversized Key-Value (KV) caches, resulting in sig…