9 papers
DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection
Yuyang Hong, Jinhui Guo, Jiaqi Gu +6
Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optim…
DiffPDE: Masked Diffusion Language Models as PDE Solver
Wenxuan Guo, Yuyang Hong, Lubin Fan +4
Existing approaches for synthesizing Partial Differential Equation (PDE) solvers predominantly rely on autoregressive models, yet their global left-to-right decoding incurs substan…
Audio-Visual Segmentation via Depth-Guided Collaborative Modeling
Zhaojin Fu, Yuyang Hong, Qi Yang +4
Audio-Visual Segmentation (AVS) is a fundamental task in multimodal perception that performs pixel-level segmentation of sounding objects in videos by leveraging both visual and au…
SeaVIS: Sound-Enhanced Association for Online Audio-Visual Instance Segmentation
Yingjian Zhu, Ying Wang, Yuyang Hong +5
Recently, an audio-visual instance segmentation (AVIS) task has been introduced, aiming to identify, segment and track individual sounding instances in videos. However, prevailing…
CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering
Yuyang Hong, Jiaqi Gu, Yujin Lou +7
Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric kno…
Enhanced Graph Transformer with Serialized Graph Tokens
Ruixiang Wang, Yuyang Hong, Shiming Xiang +1
Transformers have demonstrated success in graph learning, particularly for node-level tasks. However, existing methods encounter an information bottleneck when generating graph-lev…
IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Prompting
Tao Zhang, Yuyang Hong, Yang Xia +5
Recent advances in multimodal large language models (MLLMs) have led to impressive progress across various benchmarks. However, their capability in understanding infrared images re…
Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering
Yuyang Hong, Jiaqi Gu, Qi Yang +6
Knowledge-based visual question answering (KB-VQA) requires visual language models (VLMs) to integrate visual understanding with external knowledge retrieval. Although retrieval-au…
Taming Modality Entanglement in Continual Audio-Visual Segmentation
Yuyang Hong, Qi Yang, Tao Zhang +5
Recently, significant progress has been made in multi-modal continual learning, aiming to learn new tasks sequentially in multi-modal settings while preserving performance on previ…