GraphXForm: Graph transformer for computer-aided molecular design

Abstract

Generative deep learning has become pivotal in molecular design for drug discovery, materials science, and chemical engineering. A widely used paradigm is to pretrain neural networks on string representations of molecules and fine-tune them using reinforcement learning on specific objectives. However, string-based models face challenges in ensuring chemical validity and enforcing structural constraints like the presence of specific substructures. We propose to instead combine graph-based molecular representations, which can naturally ensure chemical validity, with transformer architectures, which are highly expressive and capable of modeling long-range dependencies between atoms. Our approach iteratively modifies a molecular graph by adding atoms and bonds, which ensures chemical validity and facilitates the incorporation of structural constraints. We present GraphXForm, a decoder-only graph transformer architecture, which is pretrained on existing compounds and then fine-tuned using a new training algorithm that combines elements of the deep cross-entropy method and self-improvement learning. We evaluate GraphXForm on various drug design tasks, demonstrating superior objective scores compared to state-of-the-art molecular design approaches. Furthermore, we apply GraphXForm to two solvent design tasks for liquid-liquid extraction, again outperforming alternative methods while flexibly enforcing structural constraints or initiating design from existing molecular structures.

Published in Digital Discovery, 2025

References in corpus (19)

• Automatic chemical design using a data-driven continuous representation of molecules
• GuacaMol: Benchmarking Models for De Novo Molecular Design
• Optimization of Molecules via Deep Reinforcement Learning
• MoFlow: An Invertible Flow Model for Generating Molecular Graphs
• Graph Neural Networks for Temperature-Dependent Activity Coefficient Prediction of Solutes in Ionic Liquids
• Graph Neural Networks for Molecules
• A smile is all you need: Predicting limiting activity coefficients from SMILES with natural language processing
• SPT-NRTL: A physics-guided machine learning model to predict thermodynamically consistent activity coefficients
• Scaffold-constrained molecular generation
• Group SELFIES: A Robust Fragment-Based Molecular String Representation
• Gibbs-Duhem-Informed Neural Networks for Binary Activity Coefficient Prediction
• HANNA: Hard-constraint Neural Network for Consistent Activity Coefficient Prediction
• Thermodynamics-Consistent Graph Neural Networks
• Graph Machine Learning for Design of High-Octane Fuels
• Gibbs-Helmholtz Graph Neural Network: capturing the temperature dependency of activity coefficients at infinite dilution
• Graph neural networks for the prediction of molecular structure-property relationships
• Re-evaluating sample efficiency in de novo molecule generation
• Constructions in combinatorics via neural networks
• Generative Artificial Intelligence for Navigating Synthesizable Chemical Space