26 citations · 56 across the 22 of their papers we have counts for
26 papers
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
Weihao Bo, Shan Zhang, Yanpeng Sun +7
Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a free workspace for scientific wri…
V-FIND: Revealing the Intrinsic Forgery Knowledge Encoded in Video Forgery Detectors
Shichao Kan, Chengpeng Hong, Jingtong Dou +8
As generated videos become increasingly realistic, reliable video forgery detection is increasingly important. Existing studies typically optimize and use video forgery detectors a…
OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysis
Jing Hao, Siyuan Dai, Yongxin Zhang +13
Dental image analysis plays a pivotal role in supporting accurate diagnosis and treatment planning in oral healthcare. Although recent advances have produced dental AI models for s…
SSP-SAM: SAM with Semantic-Spatial Prompt for Referring Expression Segmentation
Wei Tang, Xuejing Liu, Yanpeng Sun +1
The Segment Anything Model (SAM) excels at general image segmentation but has limited ability to understand natural language, which restricts its direct application in Referring Ex…
Artemis: Structured Visual Reasoning for Perception Policy Learning
Wei Tang, Yanpeng Sun, Shan Zhang +6
Recent reinforcement-learning frameworks for visual perception policy usually incorporate intermediate reasoning chains expressed in natural language. Empirical observations indica…
OralGPT-Omni: A Versatile Dental Multimodal Large Language Model
Jing Hao, Yuci Liang, Lizhuo Lin +12
Multimodal Large Language Models (MLLMs) have exhibited immense potential across numerous medical specialties; yet, dentistry remains underexplored, in part due to limited domain-s…
FVAR: Visual Autoregressive Modeling via Next Focus Prediction
Xiaofan Li, Chenming Wu, Yanpeng Sun +6
Visual autoregressive models achieve remarkable generation quality through next-scale predictions across multi-scale token pyramids. However, the conventional method uses uniform s…
Video4Edit: Viewing Image Editing as a Degenerate Temporal Process
Xiaofan Li, Yanpeng Sun, Chenming Wu +5
We observe that recent advances in multimodal foundation models have propelled instruction-driven image generation and editing into a genuinely cross-modal, cooperative regime. Nev…
Agentic Learner with Grow-and-Refine Multimodal Semantic Memory
Weihao Bo, Shan Zhang, Yanpeng Sun +9
MLLMs exhibit strong reasoning on isolated queries, yet they operate de novo -- solving each problem independently and often repeating the same mistakes. Existing memory-augmented…
FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Prompts
Weihao Bo, Yanpeng Sun, Yu Wang +2
In this paper, we introduce FedMGP, a new paradigm for personalized federated prompt learning in vision-language models. FedMGP equips each client with multiple groups of paired te…
Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis
Jing Hao, Yuxuan Fan, Yanpeng Sun +7
Recent advances in large vision-language models (LVLMs) have demonstrated strong performance on general-purpose medical tasks. However, their effectiveness in specialized domains s…
DeRIS: Decoupling Perception and Cognition for Enhanced Referring Image Segmentation through Loopback Synergy
Ming Dai, Wenxuan Cheng, Jiang-jiang Liu +4
Referring Image Segmentation (RIS) is a challenging task that aims to segment objects in an image based on natural language expressions. While prior studies have predominantly conc…
Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs
Yanpeng Sun, Shan Zhang, Wei Tang +5
Diagrams represent a form of visual language that encodes abstract concepts and relationships through structured symbols and their spatial arrangements. Unlike natural images, they…
Visual Position Prompt for MLLM based Visual Grounding
Wei Tang, Yanpeng Sun, Qinying Gu +1
Although Multimodal Large Language Models (MLLMs) excel at various image-related tasks, they encounter challenges in precisely aligning coordinates with spatial information within…
Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs
Shan Zhang, Aotian Chen, Yanpeng Sun +6
Current multimodal large language models (MLLMs) often underperform on mathematical problem-solving tasks that require fine-grained visual understanding. The limitation is largely…
Enhancing Descriptive Captions with Visual Attributes for Multimodal Perception
Yanpeng Sun, Jing Hao, Ke Zhu +6
Training Large Multimodality Models (LMMs) relies on descriptive image caption that connects image and language. Existing methods for generating such captions often rely on distill…
Continual SFT Matches Multimodal RLHF with Negative Supervision
Ke Zhu, Yu Wang, Yanpeng Sun +4
Multimodal RLHF usually happens after supervised finetuning (SFT) stage to continually improve vision-language models' (VLMs) comprehension. Conventional wisdom holds its superiori…
Improving Multi-modal Large Language Model through Boosting Vision Capabilities
Yanpeng Sun, Huaxin Zhang, Qiang Chen +5
We focus on improving the visual understanding capability for boosting the vision-language models. We propose \textbf{Arcana}, a multiModal language model, which introduces two cru…
FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMs
Jing Hao, Yuxiang Zhao, Song Chen +6
Multimodal Large Language Models (MLLMs) have shown promise in a broad range of vision-language tasks with their strong reasoning and generalization capabilities. However, they hea…
CSGO: Content-Style Composition in Text-to-Image Generation
Peng Xing, Haofan Wang, Yanpeng Sun +5
The diffusion model has shown exceptional capabilities in controlled image generation, which has further fueled interest in image style transfer. Existing works mainly focus on tra…
VRP-SAM: SAM with Visual Reference Prompt
Yanpeng Sun, Jiahui Chen, Shan Zhang +7
In this paper, we propose a novel Visual Reference Prompt (VRP) encoder that empowers the Segment Anything Model (SAM) to utilize annotated reference images as prompts for segmenta…
Exploring Effective Factors for Improving Visual In-Context Learning
Yanpeng Sun, Qiang Chen, Xiaofan Li +3
The In-Context Learning (ICL) is to understand a new task via a few demonstrations (aka. prompt) and predict new inputs without tuning the models. While it has been widely studied…
Self-Supervised Guided Segmentation Framework for Unsupervised Anomaly Detection
Peng Xing, Yanpeng Sun, Zechao Li
Unsupervised anomaly detection is a challenging task in industrial applications since it is impracticable to collect sufficient anomalous samples. In this paper, a novel Self-Super…
Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning
Yanpeng Sun, Qiang Chen, Xiangyu He +7
Freezing the pre-trained backbone has become a standard paradigm to avoid overfitting in few-shot segmentation. In this paper, we rethink the paradigm and explore a new regime: {\e…