40 citations · 200 across the 45 of their papers we have counts for
45 papers
Symbolic and Abstractive Reasoning with Complex Visual Queries
Yichi Zhang, Jingdian Lu, Zhuo Chen +4
Understanding and reasoning over abstract visual content remains a challenge for current multi-modal large language models (MLLMs). In this paper, we explore a novel abstract data…
Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning
Jingzhi Chen, Landi He, Zhuo Chen +2
The processing of gigapixel whole slide images within vision language models faces a major difficulty due to an excessive number of visual tokens. Existing solutions typically rely…
Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models
Yang Zhou, Ranajoy Sadhukhan, Zhaofeng Sun +7
Despite being powerful, reinforcement learning with verifiable rewards (RLVR) induces extremely long COT, making it computationally expensive. Since RLVR per-step cost is dominated…
Every Little Helps: Building Knowledge Graph Foundation Model with Fine-grained Transferable Multi-modal Tokens
Yichi Zhang, Zhuo Chen, Lingbing Guo +2
Multi-modal knowledge graph reasoning (MMKGR) aims to predict the missing links by exploiting both graph structure information and multi-modal entity contents. Most existing works…
Abstractive Visual Understanding of Multi-modal Structured Knowledge: A New Perspective for MLLM Evaluation
Yichi Zhang, Zhuo Chen, Lingbing Guo +4
Multi-modal large language models (MLLMs) incorporate heterogeneous modalities into LLMs, enabling a comprehensive understanding of diverse scenarios and objects. Despite the proli…
Multi-modal Knowledge Graph Generation with Semantics-enriched Prompts
Yajing Xu, Zhiqiang Liu, Jiaoyan Chen +7
Multi-modal Knowledge Graphs (MMKGs) have been widely applied across various domains for knowledge representation. However, the existing MMKGs are significantly fewer than required…
K-ON: Stacking Knowledge On the Head Layer of Large Language Model
Lingbing Guo, Yichi Zhang, Zhongpu Bo +5
Recent advancements in large language models (LLMs) have significantly improved various natural language processing (NLP) tasks. Typically, LLMs are trained to predict the next tok…
Have We Designed Generalizable Structural Knowledge Promptings? Systematic Evaluation and Rethinking
Yichi Zhang, Zhuo Chen, Lingbing Guo +8
Large language models (LLMs) have demonstrated exceptional performance in text generation within current NLP research. However, the lack of factual accuracy is still a dark cloud h…
UniHR: Hierarchical Representation Learning for Unified Knowledge Graph Link Prediction
Zhiqiang Liu, Yin Hua, Mingyang Chen +4
Real-world knowledge graphs (KGs) contain not only standard triple-based facts, but also more complex, heterogeneous types of facts, such as hyper-relational facts with auxiliary k…
MKGL: Mastery of a Three-Word Language
Lingbing Guo, Zhongpu Bo, Zhuo Chen +10
Large language models (LLMs) have significantly advanced performance across a spectrum of natural language processing (NLP) tasks. Yet, their application to knowledge graphs (KGs),…
DRAK: Unlocking Molecular Insights with Domain-Specific Retrieval-Augmented Knowledge in LLMs
Jinzhe Liu, Xiangsheng Huang, Zhuo Chen +1
Large Language Models (LLMs) encounter challenges with the unique syntax of specific domains, such as biomolecules. Existing fine-tuning or modality alignment techniques struggle t…
NativE: Multi-modal Knowledge Graph Completion in the Wild
Yichi Zhang, Zhuo Chen, Lingbing Guo +5
Multi-modal knowledge graph completion (MMKGC) aims to automatically discover the unobserved factual knowledge from a given multi-modal knowledge graph by collaboratively modeling…
Multi-domain Knowledge Graph Collaborative Pre-training and Prompt Tuning for Diverse Downstream Tasks
Yichi Zhang, Binbin Hu, Zhuo Chen +6
Knowledge graphs (KGs) provide reliable external knowledge for a wide variety of AI tasks in the form of structured triples. Knowledge graph pre-training (KGP) aims to pre-train ne…
Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning
Yichi Zhang, Zhuo Chen, Lingbing Guo +5
Learning high-quality multi-modal entity representations is an important goal of multi-modal knowledge graph (MMKG) representation learning, which can enhance reasoning tasks withi…
Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation
Yichi Zhang, Zhuo Chen, Lingbing Guo +5
Multi-modal knowledge graph completion (MMKGC) aims to discover unobserved knowledge from given knowledge graphs, collaboratively leveraging structural information from the triples…
Noise-powered Multi-modal Knowledge Graph Representation Framework
Zhuo Chen, Yin Fang, Yichi Zhang +5
The rise of Multi-modal Pre-training highlights the necessity for a unified Multi-Modal Knowledge Graph (MMKG) representation learning framework. Such a framework is essential for…
Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey
Zhuo Chen, Yichi Zhang, Yin Fang +12
Knowledge Graphs (KGs) play a pivotal role in advancing various AI applications, with the semantic web community's exploration into multi-modal dimensions unlocking new avenues for…
ChatCell: Facilitating Single-Cell Analysis with Natural Language
Yin Fang, Kangwei Liu, Ningyu Zhang +7
As Large Language Models (LLMs) rapidly evolve, their influence in science is becoming increasingly prominent. The emerging capabilities of LLMs in task generalization and free-for…
ASGEA: Exploiting Logic Rules from Align-Subgraphs for Entity Alignment
Yangyifei Luo, Zhuo Chen, Lingbing Guo +4
Entity alignment (EA) aims to identify entities across different knowledge graphs that represent the same real-world objects. Recent embedding-based EA methods have achieved state-…
Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
Yuxia Geng, Jiaoyan Chen, Yuhang Zeng +5
Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC). With the success of Pre-trained Language Models (PLMs) such as BERT, they ha…
Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering
Yichi Zhang, Zhuo Chen, Yin Fang +4
Deploying large language models (LLMs) to real scenarios for domain-specific question answering (QA) is a key thrust for LLM applications, which poses numerous challenges, especial…
Universal Multi-modal Entity Alignment via Iteratively Fusing Modality Similarity Paths
Bolin Zhu, Xiaoze Liu, Xin Mao +4
The objective of Entity Alignment (EA) is to identify equivalent entity pairs from multiple Knowledge Graphs (KGs) and create a more comprehensive and unified KG. The majority of E…
Making Large Language Models Perform Better in Knowledge Graph Completion
Yichi Zhang, Zhuo Chen, Lingbing Guo +3
Large language model (LLM) based knowledge graph completion (KGC) aims to predict the missing triples in the KGs with LLMs. However, research about LLM-based KGC fails to sufficien…
MACO: A Modality Adversarial and Contrastive Framework for Modality-missing Multi-modal Knowledge Graph Completion
Yichi Zhang, Zhuo Chen, Wen Zhang
Recent years have seen significant advancements in multi-modal knowledge graph completion (MMKGC). MMKGC enhances knowledge graph completion (KGC) by integrating multi-modal entity…