Transformer-CNN: Fast and Reliable tool for QSAR

Abstract

We present SMILES-embeddings derived from the internal encoder state of a Transformer [1] model trained to canonize SMILES as a Seq2Seq problem. Using a CharNN [2] architecture upon the embeddings results in higher quality interpretable QSAR/QSPR models on diverse benchmark datasets including regression and classification tasks. The proposed Transformer-CNN method uses SMILES augmentation for training and inference, and thus the prognosis is based on an internal consensus. That both the augmentation and transfer learning are based on embeddings allows the method to provide good results for small datasets. We discuss the reasons for such effectiveness and draft future directions for the development of the method. The source code and the embeddings needed to train a QSAR model are available on https://github.com/bigchem/transformer-cnn. The repository also has a standalone program for QSAR prognosis which calculates individual atoms contributions, thus interpreting the model's result. OCHEM [3] environment (https://ochem.eu) hosts the on-line implementation of the method proposed.

References in corpus (7)

• XGBoost: A Scalable Tree Boosting System
• TensorFlow: A system for large-scale machine learning
• Neural Message Passing for Quantum Chemistry
• Automatic chemical design using a data-driven continuous representation of molecules
• Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction
• Towards Explainable Artificial Intelligence
• Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks

Cited by in corpus (6)

• State-of-the-Art Augmented NLP Transformer models for direct and single-step retrosynthesis
• Generalizable, Fast, and Accurate DeepQSPR with fastprop
• Improving Compound Activity Classification via Deep Transfer and Representation Learning
• MLSolv-A: A Novel Machine Learning-Based Prediction of Solvation Free Energies from Pairwise Atomistic Interactions
• CRNNTL: convolutional recurrent neural network and transfer learning for QSAR modelling
• Beyond Chemical 1D knowledge using Transformers