16 citations · 54 across the 56 of their papers we have counts for
11 papers · 1 filter
Self-Evolving Code-with-Image Reasoning
Tianze Yang, Liang Wu, Ruitong Sun +6
Multimodal models increasingly reach for tools when solving visual tasks (crop, zoom, rotate, brighten), a paradigm known as thinking-with-images. The central challenge is one of p…
ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning
Yu-Cheng Shi, Zhen-Hao Xie, Jun-Tao Tang +1
Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, but real-world deployment requires them to continually acquire new vision-language c…
Self-Improving Small Object Grounding in LVLMs
Tianze Yang, Yucheng Shi, Ruitong Sun +2
Can internal attention patterns in Large Vision Language Models (LVLMs) identify reliable small-object boxes without fine-tuning? In this work, we provide an affirmative answer. At…
Reinforcing Multimodal Reasoning Against Visual Degradation
Rui Liu, Dian Yu, Haolin Liu +6
Reinforcement Learning has significantly advanced the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet the resulting policies remain brittle against real-wor…
RadFabric: Agentic AI System with Reasoning Capability for Radiology
Wenting Chen, Yi Dong, Zhaojun Ding +14
Chest X ray (CXR) imaging remains a critical diagnostic tool for thoracic conditions, but current automated systems face limitations in pathology coverage, diagnostic accuracy, and…
Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models
Yulei Qin, Gang Li, Zongyi Li +6
Existing large language models (LLMs) face challenges of following complex instructions, especially when multiple constraints are present and organized in paralleling, chaining, an…
Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Tianze Yang, Yucheng Shi, Mengnan Du +4
Vector-Quantized Generative Models (VQGMs) have emerged as powerful tools for image generation. However, the key component of VQGMs -- the codebook of discrete tokens -- is still n…
Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Data
Yucheng Shi, Quanzheng Li, Jin Sun +2
Large Multimodal Models (LMMs), or Vision-Language Models (VLMs), have shown impressive capabilities in a wide range of visual tasks. However, they often struggle with fine-grained…
Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models
Yuncheng Yang, Yulei Qin, Tong Wu +9
The cultivation of expertise for large language models (LLMs) to solve tasks of specific areas often requires special-purpose tuning with calibrated behaviors on the expected stabl…
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models
Yulei Qin, Yuncheng Yang, Pengcheng Guo +7
Instruction tuning plays a critical role in aligning large language models (LLMs) with human preference. Despite the vast amount of open instruction datasets, naively training a LL…
Automated Natural Language Explanation of Deep Visual Neurons with Large Models
Chenxu Zhao, Wei Qian, Yucheng Shi +2
Deep neural networks have exhibited remarkable performance across a wide range of real-world tasks. However, comprehending the underlying reasons for their effectiveness remains a…