Multiresolution Graph Transformers and Wavelet Positional Encoding for Learning Hierarchical Structures
arXiv:2302.08647 · doi:10.1063/5.0152833
Abstract
Contemporary graph learning algorithms are not well-defined for large molecules since they do not consider the hierarchical interactions among the atoms, which are essential to determine the molecular properties of macromolecules. In this work, we propose Multiresolution Graph Transformers (MGT), the first graph transformer architecture that can learn to represent large molecules at multiple scales. MGT can learn to produce representations for the atoms and group them into meaningful functional groups or repeating units. We also introduce Wavelet Positional Encoding (WavePE), a new positional encoding method that can guarantee localization in both spectral and spatial domains. Our proposed model achieves competitive results on two macromolecule datasets consisting of polymers and peptides, and one drug-like molecule dataset. Importantly, our model outperforms other state-of-the-art methods and achieves chemical accuracy in estimating molecular properties (e.g., GAP, HOMO and LUMO) calculated by Density Functional Theory (DFT) in the polymers dataset. Furthermore, the visualizations, including clustering results on macromolecules and low-dimensional spaces of their representations, demonstrate the capability of our methodology in learning to represent long-range and hierarchical structures. Our PyTorch implementation is publicly available at https://github.com/HySonLab/Multires-Graph-Transformer
References in corpus (34)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Semi-Supervised Classification with Graph Convolutional Networks
- Neural Message Passing for Quantum Chemistry
- Self-Supervised Graph Transformer on Large-Scale Molecular Data
- Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning
- Learning Structural Node Embeddings Via Diffusion Wavelets
- How Powerful are Graph Neural Networks?
- A Generalization of Transformer Networks to Graphs
- DeeperGCN: All You Need to Train Deeper GCNs
- Principal Neighbourhood Aggregation for Graph Nets
- Recipe for a General, Powerful, Scalable Graph Transformer
- Benchmarking Graph Neural Networks
- Learning from Protein Structure with Geometric Vector Perceptrons
- Graph Neural Networks Exponentially Lose Expressive Power for Node Classification
- Strategies for Pre-training Graph Neural Networks
- Cormorant: Covariant Molecular Neural Networks
- OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
- Message-passing neural networks for high-throughput polymer screening
- Hierarchical Generation of Molecular Graphs using Structural Motifs
- Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning
- Invariant and Equivariant Graph Networks
- Position-aware Graph Neural Networks
- On the Bottleneck of Graph Neural Networks and its Practical Implications
- Covariant Compositional Networks For Learning Graphs
- Rethinking Graph Transformers with Spectral Attention
- Pure Transformers are Powerful Graph Learners
- Structure-Aware Transformer for Graph Representation Learning
- Learning to Extend Molecular Scaffolds with Structural Motifs
- Graph Neural Networks with Learnable Structural and Positional Representations
- Sign and Basis Invariant Networks for Spectral Graph Representation Learning
- Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks
- Knowledge-aware Contrastive Molecular Graph Learning
- On the Connection Between MPNN and Graph Transformer
- A New Perspective on the Effects of Spectrum in Graph Neural Networks