10 papers
LLaVAFlow: Preserving Latent Alignment Flow for Parameter-Efficient Multimodal Fine-Tuning
Muyao Yuan, Muyan Jiao, Jiangyong Ying +5
While Multimodal Large Language Models (MLLMs) exhibit strong generalization, visual instruction tuning for downstream tasks inevitably causes catastrophic forgetting, impairing ov…
Beyond Pixels: Benchmarking and Reward-Based Assessing Framework for Visual Spatial Aesthetics
Yuan Gao, Jin Song, Yiyun Fei +2
In recent years, Image Quality Assessment (IQA) for AI-generated images (AIGI) has advanced rapidly; however, existing methods primarily target portraits and artistic images, lacki…
VideoPerceiver: Enhancing Fine-Grained Temporal Perception in Video Multimodal Large Language Models
Fufangchen Zhao, Liao Zhang, Daiqi Shi +5
We propose VideoPerceiver, a novel video multimodal large language model (VMLLM) that enhances fine-grained perception in video understanding, addressing VMLLMs' limited ability to…
InfoCLIP: Bridging Vision-Language Pretraining and Open-Vocabulary Semantic Segmentation via Information-Theoretic Alignment Transfer
Muyao Yuan, Yuanhong Zhang, Weizhan Zhang +4
Recently, the strong generalization ability of CLIP has facilitated open-vocabulary semantic segmentation, which labels pixels using arbitrary text. However, existing methods that…
On the Faithfulness of Visual Thinking: Measurement and Enhancement
Zujing Liu, Junwen Pan, Qi She +2
Recent large vision-language models (LVLMs) can generate vision-text multimodal chain-of-thought (MCoT) traces after reinforcement fine-tuning (RFT). However, we observe that the v…
Comprehensive language-image pre-training for 3D medical image understanding
Tassilo Wald, Ibrahim Ethem Hamamci, Yuan Gao +13
In the 3D medical image domain, vision-language pre-training is used to create vision-language encoders (VLEs) that can support radiologists by retrieving patients with similar abn…
EvoMakeup: High-Fidelity and Controllable Makeup Editing with MakeupQuad
Huadong Wu, Yi Fu, Yunhao Li +2
Facial makeup editing aims to realistically transfer makeup from a reference to a target face. Existing methods often produce low-quality results with coarse makeup details and str…
MERIT: Multilingual Semantic Retrieval with Interleaved Multi-Condition Query
Wei Chow, Yuan Gao, Linfeng Li +15
Semantic retrieval is crucial for modern applications yet remains underexplored in current research. Existing datasets are limited to single languages, single images, or singular r…
Valley2: Exploring Multimodal Models with Scalable Vision-Language Design
Ziheng Wu, Zhenghao Chen, Ruipu Luo +6
Recently, vision-language models have made remarkable progress, demonstrating outstanding capabilities in various tasks such as image captioning and video understanding. We introdu…
Enhancing Vision-Language Pre-training with Rich Supervisions
Yuan Gao, Kunyu Shi, Pengkai Zhu +7
We propose Strongly Supervised pre-training with ScreenShots (S4) - a novel pre-training paradigm for Vision-Language Models using data from large-scale web screenshot rendering. U…