5 papers
Evian: Towards Explainable Visual Instruction-tuning Data Auditing
Zimu Jia, Mingjie Xu, Andrew Estornell +1
The efficacy of Large Vision-Language Models (LVLMs) is critically dependent on the quality of their training data, requiring a precise balance between visual fidelity and instruct…
VP-Bench: A Comprehensive Benchmark for Visual Prompting in Multimodal Large Language Models
Mingjie Xu, Jinpeng Chen, Yuzhi Zhao +12
Multimodal large language models (MLLMs) have enabled a wide range of advanced vision-language applications, including fine-grained object recognition and contextual understanding.…
Better Reasoning with Less Data: Enhancing VLMs Through Unified Modality Scoring
Mingjie Xu, Andrew Estornell, Hongzheng Yang +4
The application of visual instruction tuning and other post-training techniques has significantly enhanced the capabilities of Large Language Models (LLMs) in visual understanding,…
ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderation
Mengyang Wu, Yuzhi Zhao, Jialun Cao +7
Controversial contents largely inundate the Internet, infringing various cultural norms and child protection standards. Traditional Image Content Moderation (ICM) models fall short…
LLaVA-SpaceSGG: Visual Instruct Tuning for Open-vocabulary Scene Graph Generation with Enhanced Spatial Relations
Mingjie Xu, Mengyang Wu, Yuzhi Zhao +2
Scene Graph Generation (SGG) converts visual scenes into structured graph representations, providing deeper scene understanding for complex vision tasks. However, existing SGG mode…