Multi-modal Entity Alignment in Hyperbolic Space

Abstract

Many AI-related tasks involve the interactions of data in multiple modalities. It has been a new trend to merge multi-modal information into knowledge graph(KG), resulting in multi-modal knowledge graphs (MMKG). However, MMKGs usually suffer from low coverage and incompleteness. To mitigate this problem, a viable approach is to integrate complementary knowledge from other MMKGs. To this end, although existing entity alignment approaches could be adopted, they operate in the Euclidean space, and the resulting Euclidean entity representations can lead to large distortion of KG's hierarchical structure. Besides, the visual information has yet not been well exploited. In response to these issues, in this work, we propose a novel multi-modal entity alignment approach, Hyperbolic multi-modal entity alignment(HMEA), which extends the Euclidean representation to hyperboloid manifold. We first adopt the Hyperbolic Graph Convolutional Networks (HGCNs) to learn structural representations of entities. Regarding the visual information, we generate image embeddings using the densenet model, which are also projected into the hyperbolic space using HGCNs. Finally, we combine the structure and visual representations in the hyperbolic space and use the aggregated embeddings to predict potential alignment results. Extensive experiments and ablation studies demonstrate the effectiveness of our proposed model and its components.

arXiv comment24 pages,5 figures;

References in corpus (15)

• Semi-Supervised Classification with Graph Convolutional Networks
• Geometric deep learning: going beyond Euclidean data
• A Survey on Knowledge Graphs: Representation, Acquisition and Applications
• Hierarchical Organization in Complex Networks
• Deep Convolutional Networks on Graph-Structured Data
• Hyperbolic Geometry of Complex Networks
• Machine learning meets network science: dimensionality reduction for fast and efficient embedding of networks in the hyperbolic space
• Hyperbolic Attention Networks
• Hyperbolic Entailment Cones for Learning Hierarchical Embeddings
• Dual-Primal Graph Convolutional Networks
• Multi-Channel Graph Neural Network for Entity Alignment
• Neighborhood Matching Network for Entity Alignment
• MMKG: Multi-Modal Knowledge Graphs
• Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment
• Degree-Aware Alignment for Entities in Tail