11 papers
OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models
Yue Ding, Yiyan Ji, Jungang Li +12
Omni-modal Large Language Models (Omni-LLMs) have demonstrated strong capabilities in audio-video understanding tasks. However, their reliance on long multimodal token sequences le…
Variation-aware Vision Token Dropping for Faster Large Vision-Language Models
Junjie Chen, Xuyang Liu, Zichen Wen +3
Large vision-language models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding tasks. However, the increasing demand for high-resolution image and long-…
Structure-based RNA Design by Step-wise Optimization of Latent Diffusion Model
Qi Si, Xuyang Liu, Penglei Wang +3
RNA inverse folding, designing sequences to form specific 3D structures, is critical for therapeutics, gene regulation, and synthetic biology. Current methods, focused on sequence…
Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference
Ting Liu, Xuyang Liu, Liangtao Shi +6
Parameter-efficient fine-tuning (PEFT) has emerged as a popular solution for adapting pre-trained Vision Transformer (ViT) models to downstream applications by updating only a smal…
Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration
Yuhang Han, Xuyang Liu, Zihan Zhang +6
The quadratic complexity of Multimodal Large Language Models (MLLMs) with respect to context length poses significant computational and memory challenges, hindering their real-worl…
GamerAstra: Supporting 2D Non-Twitch Video Games for Blind and Low-Vision Players through a Multi-Agent Framework
Tianrun Qiu, Changxin Chen, Sizhe Cheng +4
Blind and low-vision (BLV) players face critical challenges in engaging with video games due to the inaccessibility of visual elements, difficulties navigating interfaces, and limi…