6 papers · 1 filter
Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitration
Yu Zhang, Mufan Xu, Xuefeng Bai +4
Modality following is the ability to selectively leverage multimodal contexts based on user instructions. It is fundamental to the safety and reliability of multimodal large langua…
Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition
Jinlong Ma, Yu Zhang, Xuefeng Bai +5
Grounded Multimodal Named Entity Recognition (GMNER) aims to extract text-based entities, assign them semantic categories, and ground them to corresponding visual regions. In this…
Evaluating and Steering Modality Preferences in Multimodal Large Language Model
Yu Zhang, Jinlong Ma, Yongshuai Hou +5
Multi-modal large language models (MLLMs) have achieved remarkable success on complex multi-modal tasks. However, it remains insufficiently explored whether they exhibit $\textbf{m…
BANER: Boundary-Aware LLMs for Few-Shot Named Entity Recognition
Quanjiang Guo, Yihong Dong, Ling Tian +3
Despite the recent success of two-stage prototypical networks in few-shot named entity recognition (NER), challenges such as over/under-detected false spans in the span detection s…
Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering
Yu Zhang, Kehai Chen, Xuefeng Bai +3
Knowledge graph question answering (KGQA) involves answering natural language questions by leveraging structured information stored in a knowledge graph. Typically, KGQA initially…
TPN: Transferable Proto-Learning Network towards Few-shot Document-Level Relation Extraction
Yu Zhang, Zhao Kang
Few-shot document-level relation extraction suffers from poor performance due to the challenging cross-domain transferability of NOTA (none-of-the-above) relation representation. I…