Massively Multitask Networks for Drug Discovery

Abstract

Massively multitask neural architectures provide a learning framework for drug discovery that synthesizes information from many distinct biological sources. To train these architectures at scale, we gather large amounts of data from public sources to create a dataset of nearly 40 million measurements across more than 200 biological targets. We investigate several aspects of the multitask framework by performing a series of empirical studies and obtain some interesting results: (1) massively multitask networks obtain predictive accuracies significantly better than single-task methods, (2) the predictive power of multitask networks improves as additional tasks and data are added, (3) the total amount of data and the total number of tasks both contribute significantly to multitask improvement, and (4) multitask networks afford limited transferability to tasks not in the training set. Our results underscore the need for greater data sharing and further algorithmic innovation to accelerate the drug discovery process.

Preliminary work. Under review by the International Conference on Machine Learning (ICML)

References in corpus (3)

• Scikit-learn: Machine Learning in Python
• Going Deeper with Convolutions
• Multi-task Neural Networks for QSAR Predictions

Cited by in corpus (83)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• An Overview of Multi-Task Learning in Deep Neural Networks
• On the Opportunities and Risks of Foundation Models
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• Molecular Contrastive Learning of Representations via Graph Neural Networks
• AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• Precision Health Data: Requirements, Challenges and Existing Techniques for Data Security and Privacy
• TopologyNet: Topology based deep convolutional neural networks for biomolecular property predictions
• Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
• Representability of algebraic topology for biomolecules in machine learning based scoring and virtual screening
• Autonomous discovery in the chemical sciences part II: Outlook
• Limits of trust in medical AI
• Relational Graph Attention Networks
• Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry
• An Integrated Transfer Learning and Multitask Learning Approach for Pharmacokinetic Parameter Prediction
• Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Models
• Deep Confidence: A Computationally Efficient Framework for Calculating Reliable Errors for Deep Neural Networks
• Learning Deep Architectures for Interaction Prediction in Structure-based Virtual Screening
• Toxicity Prediction using Deep Learning
• Metadynamics for Training Neural Network Model Chemistries: a Competitive Assessment
• Overcoming data scarcity with transfer learning
• Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules
• Learning Graph-Level Representation for Drug Discovery
• Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space
• Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networks
• SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties
• A Survey of Deep Learning for Scientific Discovery
• MoleculeNet: A Benchmark for Molecular Machine Learning
• Constrained Bayesian Optimization for Automatic Chemical Design
• Deep Learning for Computational Chemistry
• MT-CGCNN: Integrating Crystal Graph Convolutional Neural Network with Multitask Learning for Material Property Prediction
• Multi-View Graph Neural Networks for Molecular Property Prediction
• Geodesic Convolutional Shape Optimization
• Benchmarking Accuracy and Generalizability of Four Graph Neural Networks Using Large In Vitro ADME Datasets from Different Chemical Spaces
• Path Integral Based Convolution and Pooling for Graph Neural Networks
• PaccMann: Prediction of anticancer compound sensitivity with multi-modal attention-based neural networks
• CheMixNet: Mixed DNN Architectures for Predicting Chemical Properties using Multiple Molecular Representations
• Learn molecular representations from large-scale unlabeled molecules for drug discovery
• Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks
• DeepHealth: Review and challenges of artificial intelligence in health informatics
• BERT Learns (and Teaches) Chemistry
• Meta-Learning GNN Initializations for Low-Resource Molecular Property Prediction
• Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning
• Where is the Model Looking At?--Concentrate and Explain the Network Attention
• BatmanNet: Bi-branch Masked Graph Transformer Autoencoder for Molecular Representation
• Step Change Improvement in ADMET Prediction with PotentialNet Deep Featurization
• Not to Cry Wolf: Distantly Supervised Multitask Learning in Critical Care
• Multimodal Deep Neural Networks using Both Engineered and Learned Representations for Biodegradability Prediction
• DeepCCI: End-to-end Deep Learning for Chemical-Chemical Interaction Prediction
• A Large-Scale Database for Graph Representation Learning
• Making Graph Neural Networks Worth It for Low-Data Molecular Machine Learning
• Learning Attributed Graph Representations with Communicative Message Passing Transformer
• Towards Efficiently Evaluating the Robustness of Deep Neural Networks in IoT Systems: A GAN-based Method
• Artificial Intelligence and Location Verification in Vehicular Networks
• Artificial Intelligence in Drug Discovery: Applications and Techniques
• MathNet: Haar-Like Wavelet Multiresolution-Analysis for Graph Representation and Learning
• Probabilistic Active Meta-Learning
• A Multi-task Learning Framework for Drone State Identification and Trajectory Prediction
• Deep Cross Residual Learning for Multitask Visual Recognition
• Intra-day Activity Better Predicts Chronic Conditions
• A Brief Review of Deep Multi-task Learning and Auxiliary Task Learning
• Near-optimal Representation Learning for Linear Bandits and Linear RL
• Multitask Learning On Graph Neural Networks Applied To Molecular Property Predictions
• SLAW: Scaled Loss Approximate Weighting for Efficient Multi-Task Learning
• Generative Design of Hardware-aware DNNs
• Attribution Methods Reveal Flaws in Fingerprint-Based Virtual Screening
• The Effectiveness of Multitask Learning for Phenotyping with Electronic Health Records Data
• Relational dynamic memory networks
• Network Clustering for Multi-task Learning
• Supervised Anomaly Detection via Conditional Generative Adversarial Network and Ensemble Active Learning
• Comparison of multi-task convolutional neural network (MT-CNN) and a few other methods for toxicity prediction
• Multi-Task Learning for Argumentation Mining
• Multi-task Deep Neural Networks in Automated Protein Function Prediction
• Harnessing Geometric Constraints from Emotion Labels to improve Face Verification
• Landscape of Big Medical Data: A Pragmatic Survey on Prioritized Tasks
• IL-Net: Using Expert Knowledge to Guide the Design of Furcated Neural Networks
• Fast Haar Transforms for Graph Neural Networks
• A differential neural network learns stochastic differential equations and the Black-Scholes equation for pricing multi-asset options
• Conditional Neural Architecture Search
• A Survey on Green Deep Learning