13 papers
Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation
Che Liu, Lichao Ma, Xiangyu Tony Zhang +4
Omni-modal language models are intended to jointly understand audio, visual inputs, and language, but benchmark gains can be inflated when visual evidence alone is enough to answer…
Prototype-Based Test-Time Adaptation of Vision-Language Models
Zhaohong Huang, Yuxin Zhang, Wenjing Liu +2
Test-time adaptation (TTA) has emerged as a promising paradigm for vision-language models (VLMs) to bridge the distribution gap between pre-training and test data. Recent works hav…
ID-Selection: Importance-Diversity Based Visual Token Selection for Efficient LVLM Inference
Zhaohong Huang, Wenjing Liu, Yuxin Zhang +2
Recent advances have explored visual token pruning to accelerate the inference of large vision-language models (LVLMs). However, existing methods often struggle to balance token im…
A Synthetic Eye Movement Dataset for Script Reading Detection: Real Trajectory Replay on a 3D Simulator
Kidus Zewde, Yuchen Zhou, Dennis Ng +6
Large vision-language models have achieved remarkable capabilities by training on massive internet-scale data, yet a fundamental asymmetry persists: while LLMs can leverage self-su…
Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts
Wenhao Li, Daohai Yu, Gen Luo +7
Training Large Language Models (LLMs) on long contexts is severely constrained by prohibitive GPU memory overhead, not training time. The primary culprits are the activations, whos…
Spotlight Attention: Towards Efficient LLM Generation via Non-linear Hashing-based KV Cache Retrieval
Wenhao Li, Yuxin Zhang, Gen Luo +4
Reducing the key-value (KV) cache burden in Large Language Models (LLMs) significantly accelerates inference. Dynamically selecting critical KV caches during decoding helps maintai…