5 papers
MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization
Yangchengyu Zhou, Zhiping Wu, Runchang Jiang +5
The Key-Value (KV) cache constitutes a dominant source of memory consumption during large language model (LLM) inference, particularly in long-context and complex reasoning scenari…
TReVS: Integrating Textual Relevance and Visual Saliency for Efficient Vision-Language Model Token Pruning
Jing Wang, Zhiping Wu, Dongdong Ren +3
Vision-Language Models (VLMs) excel at visual understanding and reasoning but often incur substantial inference costs due to the large number of visual tokens. Recent visual token…
RGSQ: Riemannian Geometry-Sensitive Quantization for Large Vision-Language Models
Zhiping Wu, Dongdong Ren, Yangchengyu Zhou +4
Large vision-language models (VLMs) can be efficiently deployed under stringent memory and latency constraints through post training quantization (PTQ). However, most PTQ methods a…
Robust Dataset Distillation by Matching Adversarial Trajectories
Wei Lai, Tianyu Ding, ren dongdong +4
Dataset distillation synthesizes compact datasets that enable models to achieve performance comparable to training on the original large-scale datasets. However, existing distillat…
ONNXPruner: ONNX-Based General Model Pruning Adapter
Dongdong Ren, Wenbin Li, Tianyu Ding +5
Recent advancements in model pruning have focused on developing new algorithms and improving upon benchmarks. However, the practical application of these algorithms across various…