State-of-the-Art Augmented NLP Transformer models for direct and single-step retrosynthesis

Abstract

We investigated the effect of different training scenarios on predicting the (retro)synthesis of chemical compounds using a text-like representation of chemical reactions (SMILES) and Natural Language Processing neural network Transformer architecture. We showed that data augmentation, which is a powerful method used in image processing, eliminated the effect of data memorization by neural networks, and improved their performance for the prediction of new sequences. This effect was observed when augmentation was used simultaneously for input and the target data simultaneously. The top-5 accuracy was 84.8% for the prediction of the largest fragment (thus identifying principal transformation for classical retro-synthesis) for the USPTO-50k test dataset and was achieved by a combination of SMILES augmentation and a beam search algorithm. The same approach provided significantly better results for the prediction of direct reactions from the single-step USPTO-MIT test set. Our model achieved 90.6% top-1 and 96.1% top-5 accuracy for its challenging mixed set and 97% top-5 accuracy for the USPTO-MIT separated set. It also significantly improved results for USPTO-full set single-step retrosynthesis for both top-1 and top-10 accuracies. The appearance frequency of the most abundantly generated SMILES was well correlated with the prediction outcome and can be used as a measure of the quality of reaction prediction.

References in corpus (3)

• Learning to Plan Chemical Syntheses
• Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction
• Transformer-CNN: Fast and Reliable tool for QSAR

Cited by in corpus (13)

• MechRetro is a chemical-mechanism-driven graph learning framework for interpretable retrosynthesis prediction and pathway planning
• Root-aligned SMILES: A Tight Representation for Chemical Reaction Prediction
• A smile is all you need: Predicting limiting activity coefficients from SMILES with natural language processing
• Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model
• Change Detection Meets Visual Question Answering
• ${\mathsf{G}}^2\mathsf{Retro}$ as a Two-Step Graph Generative Models for Retrosynthesis Prediction
• Difficulty in chirality recognition for Transformer architectures learning chemical structures from string
• RetroComposer: Composing Templates for Template-Based Retrosynthesis Prediction
• Re-evaluating Retrosynthesis Algorithms with Syntheseus
• Models Matter: The Impact of Single-Step Retrosynthesis on Synthesis Planning
• Node-Aligned Graph-to-Graph (NAG2G): Elevating Template-Free Deep Learning Approaches in Single-Step Retrosynthesis
• UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment
• DirectMultiStep: Direct Route Generation for Multistep Retrosynthesis