A Bayesian Flow Network Framework for Chemistry Tasks
arXiv:2407.20294 · doi:10.1021/acs.jcim.4c01792
Abstract
In this work, we introduce ChemBFN, a language model that handles chemistry tasks based on Bayesian flow networks working on discrete data. A new accuracy schedule is proposed to improve the sampling quality by significantly reducing the reconstruction loss. We show evidence that our method is appropriate for generating molecules with satisfied diversity even when a smaller number of sampling steps is used. A classifier-free guidance method is adapted for conditional generation. It is also worthwhile to point out that after generative training, our model can be fine-tuned on regression and classification tasks with the state-of-the-art performance, which opens the gate of building all-in-one models in a single module style. Our model has been open sourced at https://github.com/Augus1999/bayesian-flow-network-for-chemistry.
7 figures, 12 tables, 27 pages
References in corpus (16)
- Decoupled Weight Decay Regularization
- Recent developments in the PySCF program package
- Deep Reinforcement Learning for De-Novo Drug Design
- GuacaMol: Benchmarking Models for De Novo Molecular Design
- Classifier-Free Diffusion Guidance
- Self-Referencing Embedded Strings (SELFIES): A 100% robust molecular string representation
- ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction
- ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
- ChemTS: An Efficient Python Library for de novo Molecular Generation
- SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery
- ChemBERTa-2: Towards Chemical Foundation Models
- DiGress: Discrete Denoising diffusion for graph generation
- Can Large Language Models Empower Molecular Property Prediction?
- Bayesian Flow Networks
- A Length-Extrapolatable Transformer
- MolKD: Distilling Cross-Modal Knowledge in Chemical Reactions for Molecular Property Prediction