12 papers
Prototype-Based Test-Time Adaptation of Vision-Language Models
Zhaohong Huang, Yuxin Zhang, Wenjing Liu +2
Test-time adaptation (TTA) has emerged as a promising paradigm for vision-language models (VLMs) to bridge the distribution gap between pre-training and test data. Recent works hav…
ID-Selection: Importance-Diversity Based Visual Token Selection for Efficient LVLM Inference
Zhaohong Huang, Wenjing Liu, Yuxin Zhang +2
Recent advances have explored visual token pruning to accelerate the inference of large vision-language models (LVLMs). However, existing methods often struggle to balance token im…
Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts
Wenhao Li, Daohai Yu, Gen Luo +7
Training Large Language Models (LLMs) on long contexts is severely constrained by prohibitive GPU memory overhead, not training time. The primary culprits are the activations, whos…
CCF: A Context Compression Framework for Efficient Long-Sequence Language Modeling
Wenhao Li, Bangcheng Sun, Weihao Ye +4
Scaling language models to longer contexts is essential for capturing rich dependencies across extended discourse. However, naïve context extension imposes significant computation…
Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers
Yunshan Zhong, Yuyao Zhou, Yuxin Zhang +5
Data-free quantization (DFQ) enables model quantization without accessing real data, addressing concerns regarding data security and privacy. With the growing adoption of Vision Tr…
Spotlight Attention: Towards Efficient LLM Generation via Non-linear Hashing-based KV Cache Retrieval
Wenhao Li, Yuxin Zhang, Gen Luo +4
Reducing the key-value (KV) cache burden in Large Language Models (LLMs) significantly accelerates inference. Dynamically selecting critical KV caches during decoding helps maintai…