4 papers · 1 filter
VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling
Yuqi Zhang, Cheng Chen, Yuyu Guo +6
Vision Language Models (VLMs) face significant challenges with ultra-long, interleaved image-text sequences due to the quadratic complexity of self-attention. Current solutions eit…
From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion
Cheng Chen, Yuyu Guo, Pengpeng Zeng +4
Vision-Language Models (VLMs) create a severe visual feature bottleneck by using a crude, asymmetric connection that links only the output of the vision encoder to the input of the…
Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
Cheng Chen, Pengpeng Zeng, Yuyu Guo +3
Low-Rank Adaptation (LoRA) has emerged as a promising paradigm for Continual Learning. It independently updates its low-rank factors ( and ), creating a composite update to t…
CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model
Cheng Chen, Junchen Zhu, Xu Luo +3
Instruction tuning represents a prevalent strategy employed by Multimodal Large Language Models (MLLMs) to align with human instructions and adapt to new tasks. Nevertheless, MLLMs…