Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model

Abstract

The recent success of large foundation models in artificial intelligence has prompted the emergence of chemical pre-trained models. Despite the growing interest in large molecular pre-trained models that provide informative representations for downstream tasks, attempts for multimodal pre-training approaches on the molecule domain were limited. To address this, we present a novel multimodal molecular pre-trained model that incorporates the modalities of structure and biochemical properties, drawing inspiration from recent advances in multimodal learning techniques. Our proposed model pipeline of data handling and training objectives aligns the structure/property features in a common embedding space, which enables the model to regard bidirectional information between the molecules' structure and properties. These contributions emerge synergistic knowledge, allowing us to tackle both multimodal and unimodal downstream tasks through a single model. Through extensive experiments, we demonstrate that our model shows remarkable capabilities in solving various meaningful chemical challenges, including conditional molecule generation, property prediction, molecule classification, and reaction prediction.

References in corpus (17)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Learning Transferable Visual Models From Natural Language Supervision
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Bootstrap your own latent: A new approach to self-supervised Learning
• Automatic chemical design using a data-driven continuous representation of molecules
• Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction
• Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
• ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction
• CoCa: Contrastive Captioners are Image-Text Foundation Models
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• Optimization of Molecules via Deep Reinforcement Learning
• State-of-the-Art Augmented NLP Transformer models for direct and single-step retrosynthesis
• VLP: A Survey on Vision-Language Pre-training
• polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics
• TransPolymer: a Transformer-based language model for polymer property predictions
• Scaffold-based molecular design using graph generative model
• Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model

Cited by in corpus (2)

• Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model
• A Large Encoder-Decoder Family of Foundation Models For Chemical Language