Learning Attributed Graph Representations with Communicative Message Passing Transformer
arXiv:2107.08773
Abstract
Constructing appropriate representations of molecules lies at the core of numerous tasks such as material science, chemistry and drug designs. Recent researches abstract molecules as attributed graphs and employ graph neural networks (GNN) for molecular representation learning, which have made remarkable achievements in molecular graph modeling. Albeit powerful, current models either are based on local aggregation operations and thus miss higher-order graph properties or focus on only node information without fully using the edge information. For this sake, we propose a Communicative Message Passing Transformer (CoMPT) neural network to improve the molecular graph representation by reinforcing message interactions between nodes and edges based on the Transformer architecture. Unlike the previous transformer-style GNNs that treat molecules as fully connected graphs, we introduce a message diffusion mechanism to leverage the graph connectivity inductive bias and reduce the message enrichment explosion. Extensive experiments demonstrated that the proposed model obtained superior performances (around 4 on average) against state-of-the-art baselines on seven chemical property datasets (graph-level tasks) and two chemical shift datasets (node-level tasks). Further visualization studies also indicated a better representation capacity achieved by our model.
arXiv commentAccepted by IJCAI2021. 7 pages, 2 figures, 3 tables, 1 appendix
References in corpus (8)
- Molecular Graph Convolutions: Moving Beyond Fingerprints
- Convolutional Networks on Graphs for Learning Molecular Fingerprints
- Massively Multitask Networks for Drug Discovery
- SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery
- Molecule Attention Transformer
- N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules
- Edge Attention-based Multi-Relational Graph Convolutional Networks
- GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation