polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics

Abstract

Polymers are a vital part of everyday life. Their chemical universe is so large that it presents unprecedented opportunities as well as significant challenges to identify suitable application-specific candidates. We present a complete end-to-end machine-driven polymer informatics pipeline that can search this space for suitable candidates at unprecedented speed and accuracy. This pipeline includes a polymer chemical fingerprinting capability called polyBERT (inspired by Natural Language Processing concepts), and a multitask learning approach that maps the polyBERT fingerprints to a host of properties. polyBERT is a chemical linguist that treats the chemical structure of polymers as a chemical language. The present approach outstrips the best presently available concepts for polymer property prediction based on handcrafted fingerprint schemes in speed by two orders of magnitude while preserving accuracy, thus making it a strong candidate for deployment in scalable architectures including cloud infrastructures.

References in corpus (13)

• Molecular Contrastive Learning of Representations via Graph Neural Networks
• ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
• polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics
• Polymer Informatics: Current Status and Critical Next Steps
• Accelerated materials property predictions and design using motif-based fingerprints
• Polymer Informatics with Multi-Task Learning
• A graph representation of molecular ensembles for polymer property prediction
• ChemBERTa-2: Towards Chemical Foundation Models
• Polymer informatics at-scale with multitask graph neural networks
• Copolymer Informatics with Multi-Task Deep Neural Networks
• Bioplastic Design using Multitask Deep Neural Networks

Cited by in corpus (15)

• polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informatics
• Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture
• Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model
• De novo Design of Polymer Electrolytes with High Conductivity using GPT-based and Diffusion-based Generative Models
• Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectorie
• Machine Learning Based Prediction of Proton Conductivity in Metal-Organic Frameworks
• Tutorial: AI-assisted exploration and active design of polymers with high intrinsic thermal conductivity
• Multimodal machine learning with large language embedding model for polymer property prediction
• Benchmarking Large Language Models for Polymer Property Predictions
• Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNN
• Accelerating materials discovery for polymer solar cells: Data-driven insights enabled by natural language processing
• A fast transferable method for predicting the glass transition temperature of polymers from chemical structure
• Superconductor discovery in the emerging paradigm of Materials Informatics
• An Encoder-Decoder Foundation Chemical Language Model for Generative Polymer Design
• AI-Driven Design of poly(ethylene terephthalate)-replacement copolymers