11 papers
Symbolic and Abstractive Reasoning with Complex Visual Queries
Yichi Zhang, Jingdian Lu, Zhuo Chen +4
Understanding and reasoning over abstract visual content remains a challenge for current multi-modal large language models (MLLMs). In this paper, we explore a novel abstract data…
ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models
Wenbin Guo, Xin Wang, Jiaoyan Chen +3
Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond tra…
Structured and Abstractive Reasoning on Multi-modal Relational Knowledge Images
Yichi Zhang, Zhuo Chen, Lingbing Guo +2
Understanding and reasoning with abstractive information from the visual modality presents significant challenges for current multi-modal large language models (MLLMs). Among the v…
Every Little Helps: Building Knowledge Graph Foundation Model with Fine-grained Transferable Multi-modal Tokens
Yichi Zhang, Zhuo Chen, Lingbing Guo +2
Multi-modal knowledge graph reasoning (MMKGR) aims to predict the missing links by exploiting both graph structure information and multi-modal entity contents. Most existing works…
Abstractive Visual Understanding of Multi-modal Structured Knowledge: A New Perspective for MLLM Evaluation
Yichi Zhang, Zhuo Chen, Lingbing Guo +4
Multi-modal large language models (MLLMs) incorporate heterogeneous modalities into LLMs, enabling a comprehensive understanding of diverse scenarios and objects. Despite the proli…
Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning
Yichi Zhang, Zhuo Chen, Lingbing Guo +5
Learning high-quality multi-modal entity representations is an important goal of multi-modal knowledge graph (MMKG) representation learning, which can enhance reasoning tasks withi…