4 citations · 4 across the 15 of their papers we have counts for
19 papers
How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining
Lin Chen, Bolin Ni, Qi Yang +6
Most modern multimodal large language models (MLLMs) build on a pretrained visual encoder that provides a strong visual prior. Encoder-free MLLMs instead learn visual representatio…
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
Yingjian Zhu, Zhenyi Wang, Jiaxin Guo +6
Rubric-based tasks are increasingly addressed through reinforcement learning (RL), with rubric scores used as training rewards. However, these rewards typically supervise final ans…
DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection
Yuyang Hong, Jinhui Guo, Jiaqi Gu +6
Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optim…
DiffPDE: Masked Diffusion Language Models as PDE Solver
Wenxuan Guo, Yuyang Hong, Lubin Fan +4
Existing approaches for synthesizing Partial Differential Equation (PDE) solvers predominantly rely on autoregressive models, yet their global left-to-right decoding incurs substan…
Audio-Visual Segmentation via Depth-Guided Collaborative Modeling
Zhaojin Fu, Yuyang Hong, Qi Yang +4
Audio-Visual Segmentation (AVS) is a fundamental task in multimodal perception that performs pixel-level segmentation of sounding objects in videos by leveraging both visual and au…
WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering
Yingjian Zhu, Xinming Wang, Kun Ding +3
Multi-modal Retrieval-Augmented Generation (RAG) has emerged as a highly effective paradigm for Knowledge-Based Visual Question Answering (KB-VQA). Despite recent advancements, pre…
SeaVIS: Sound-Enhanced Association for Online Audio-Visual Instance Segmentation
Yingjian Zhu, Ying Wang, Yuyang Hong +5
Recently, an audio-visual instance segmentation (AVIS) task has been introduced, aiming to identify, segment and track individual sounding instances in videos. However, prevailing…
Beyond Sequential Distance: Inter-Modal Distance Invariant Position Encoding
Lin Chen, Bolin Ni, Qi Yang +5
Despite the remarkable capabilities of Multimodal Large Language Models (MLLMs), they still suffer from visual fading in long-context scenarios. Specifically, the attention to visu…
CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering
Yuyang Hong, Jiaqi Gu, Yujin Lou +7
Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric kno…
InfEngine: A Self-Verifying and Self-Optimizing Intelligent Engine for Infrared Radiation Computing
Kun Ding, Jian Xu, Ying Wang +2
Infrared radiation computing underpins advances in climate science, remote sensing and spectroscopy but remains constrained by manual workflows. We introduce InfEngine, an autonomo…
Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactions
Lin Chen, Xiaoke Zhao, Kun Ding +9
Multimodal Large Language Models (MLLMs) demonstrate impressive cross-modal capabilities, yet their substantial size poses significant deployment challenges. Knowledge distillation…
DSFC-Net: A Dual-Encoder Spatial and Frequency Co-Awareness Network for Rural Road Extraction
Zhengbo Zhang, Yihe Tian, Wanke Xia +6
Accurate extraction of rural roads from high-resolution remote sensing imagery is essential for infrastructure planning and sustainable development. However, this task presents uni…
SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM
Lin Chen, Yingjian Zhu, Qi Yang +3
Open-vocabulary semantic segmentation (OVSS) aims to segment and recognize objects universally. Trained on extensive high-quality segmentation data, the segment anything model (SAM…
Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering
Yuyang Hong, Jiaqi Gu, Qi Yang +6
Knowledge-based visual question answering (KB-VQA) requires visual language models (VLMs) to integrate visual understanding with external knowledge retrieval. Although retrieval-au…
Taming Modality Entanglement in Continual Audio-Visual Segmentation
Yuyang Hong, Qi Yang, Tao Zhang +5
Recently, significant progress has been made in multi-modal continual learning, aiming to learn new tasks sequentially in multi-modal settings while preserving performance on previ…
Re-ranking Reasoning Context with Tree Search Makes Large Vision-Language Models Stronger
Qi Yang, Chenghao Zhang, Lubin Fan +3
Recent advancements in Large Vision Language Models (LVLMs) have significantly improved performance in Visual Question Answering (VQA) tasks through multimodal Retrieval-Augmented…
Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentation
Lin Chen, Qi Yang, Kun Ding +5
Open-vocabulary semantic segmentation (OVSS) is an open-world task that aims to assign each pixel within an image to a specific class defined by arbitrary text descriptions. While…
Rethinking Comprehensive Benchmark for Chart Understanding: A Perspective from Scientific Literature
Lingdong Shen, Qigqi, Kun Ding +2
Scientific Literature charts often contain complex visual elements, including multi-plot figures, flowcharts, structural diagrams and etc. Evaluating multimodal models using these…
Continuous Speculative Decoding for Autoregressive Image Generation
Zili Wang, Zheng Zhang, Kun Ding +3
Continuous visual autoregressive (AR) models have demonstrated promising performance in image generation, but their inherently sequential nature results in slow inference speed. Sp…