17 citations · 23 across the 16 of their papers we have counts for
16 papers
MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment
Changhao Xiang, Shangyu Xing, Zhen Wu +2
Existing Multimodal Large Language Models (MLLMs) predominantly rely on image-text pairs for modality alignment pretraining, mapping global image representations to long textual de…
OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories
Changhao Xiang, Shilin Zhang, Zheng Ma +8
Visual tool use has emerged as a fundamental capability for multimodal agents to actively acquire evidence beyond a fixed image encoding. The prevailing recipe learns this capabili…
LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
Yifan Dai, Zhenhua Wu, Bohan Zeng +18
Joint audio-visual reasoning is essential for omnimodal understanding, yet current multimodal large language models (MLLMs) still struggle when reasoning requires fine-grained evid…
Causal Evidence for Attention Head Imbalance in Modality Conflict Hallucination
Jinrui Jiang, Zhangtai Wu, Zhen Wu +1
Modality-conflict hallucination occurs when multimodal large language models (MLLMs) prioritize erroneous textual premises over contradictory visual evidence. To understand why vis…
PersuasiveToM: A Benchmark for Evaluating Machine Theory of Mind in Persuasive Dialogues
Fangxu Yu, Lai Jiang, Shenyi Huang +2
The ability to understand and predict the mental states of oneself and others, known as the Theory of Mind (ToM), is crucial for effective social scenarios. Although recent studies…
Rethinking Relation Extraction: Beyond Shortcuts to Generalization with a Debiased Benchmark
Liang He, Yougang Chu, Zhen Wu +3
Benchmarks are crucial for evaluating machine learning algorithm performance, facilitating comparison and identifying superior solutions. However, biases within datasets can lead m…
GePBench: Evaluating Fundamental Geometric Perception for Multimodal Large Language Models
Shangyu Xing, Changhao Xiang, Yuteng Han +6
Geometric shapes play important roles in both physical world and human cognition. While multimodal large language models (MLLMs) have made significant advancements in visual unders…
Extroversion or Introversion? Controlling The Personality of Your Large Language Models
Yanquan Chen, Zhen Wu, Junjie Guo +2
Large language models (LLMs) exhibit robust capabilities in text generation and comprehension, mimicking human behavior and exhibiting synthetic personalities. However, some LLMs h…
AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capability
Fei Zhao, Taotian Pang, Chunhui Li +4
Multimodal Large Language Models (MLLMs) are widely regarded as crucial in the exploration of Artificial General Intelligence (AGI). The core of MLLMs lies in their capability to a…
Emotion-Anchored Contrastive Learning Framework for Emotion Recognition in Conversation
Fangxu Yu, Junjie Guo, Zhen Wu +1
Emotion Recognition in Conversation (ERC) involves detecting the underlying emotion behind each utterance within a conversation. Effectively generating representations for utteranc…
EFUF: Efficient Fine-grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models
Shangyu Xing, Fei Zhao, Zhen Wu +5
Multimodal large language models (MLLMs) have attracted increasing attention in the past few years, but they may still generate descriptions that include objects not present in the…
MORE: A Multimodal Object-Entity Relation Extraction Dataset with a Benchmark Evaluation
Liang He, Hongke Wang, Yongchang Cao +3
Extracting relational facts from multimodal data is a crucial task in the field of multimedia and knowledge graphs that feeds into widespread real-world applications. The emphasis…
FOAL: Fine-grained Contrastive Learning for Cross-domain Aspect Sentiment Triplet Extraction
Ting Xu, Zhen Wu, Huiyun Yang +1
Aspect Sentiment Triplet Extraction (ASTE) has achieved promising results while relying on sufficient annotation data in a specific domain. However, it is infeasible to annotate da…
M2DF: Multi-grained Multi-curriculum Denoising Framework for Multimodal Aspect-based Sentiment Analysis
Fei Zhao, Chunhui Li, Zhen Wu +3
Multimodal Aspect-based Sentiment Analysis (MABSA) is a fine-grained Sentiment Analysis task, which has attracted growing research interests recently. Existing work mainly utilizes…
DRIN: Dynamic Relation Interactive Network for Multimodal Entity Linking
Shangyu Xing, Fei Zhao, Zhen Wu +3
Multimodal Entity Linking (MEL) is a task that aims to link ambiguous mentions within multimodal contexts to referential entities in a multimodal knowledge base. Recent methods for…
Dynamic Demonstrations Controller for In-Context Learning
Fei Zhao, Taotian Pang, Zhen Wu +3
In-context learning (ICL) is a new paradigm for natural language processing (NLP), where a large language model (LLM) observes a small number of demonstrations and a test instance…