Cell Attention Networks

Abstract

Since their introduction, graph attention networks achieved outstanding results in graph representation learning tasks. However, these networks consider only pairwise relationships among nodes and then they are not able to fully exploit higher-order interactions present in many real world data-sets. In this paper, we introduce Cell Attention Networks (CANs), a neural architecture operating on data defined over the vertices of a graph, representing the graph as the 1-skeleton of a cell complex introduced to capture higher order interactions. In particular, we exploit the lower and upper neighborhoods, as encoded in the cell complex, to design two independent masked self-attention mechanisms, thus generalizing the conventional graph attention strategy. The approach used in CANs is hierarchical and it incorporates the following steps: i) a lifting algorithm that learns {\it edge features} from {\it node features}; ii) a cell attention mechanism to find the optimal combination of edge features over both lower and upper neighbors; iii) a hierarchical {\it edge pooling} mechanism to extract a compact meaningful set of features. The experimental results show that CAN is a low complexity strategy that compares favorably with state of the art results on graph-based learning tasks.

arXiv commentPreprint, under review

References in corpus (17)

• Neural Machine Translation by Jointly Learning to Align and Translate
• Neural Message Passing for Quantum Chemistry
• Spectral Networks and Locally Connected Networks on Graphs
• Diffusion-Convolutional Neural Networks
• Self-Attention Graph Pooling
• TUDataset: A collection of benchmark datasets for learning with graphs
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Topological Signal Processing over Simplicial Complexes
• Towards Sparse Hierarchical Graph Classifiers
• Toward a Spectral Theory of Cellular Sheaves
• Simplicial Neural Networks
• Cell Complex Neural Networks
• Natural Graph Networks
• Neural Algorithmic Reasoning
• Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs
• Simplicial Attention Neural Networks
• Simplicial Attention Networks

Cited by in corpus (3)

• Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures
• Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes
• Topological Neural Networks over the Air