IL-Net: Using Expert Knowledge to Guide the Design of Furcated Neural Networks

Abstract

Deep neural networks (DNN) excel at extracting patterns. Through representation learning and automated feature engineering on large datasets, such models have been highly successful in computer vision and natural language applications. Designing optimal network architectures from a principled or rational approach however has been less than successful, with the best successful approaches utilizing an additional machine learning algorithm to tune the network hyperparameters. However, in many technical fields, there exist established domain knowledge and understanding about the subject matter. In this work, we develop a novel furcated neural network architecture that utilizes domain knowledge as high-level design principles of the network. We demonstrate proof-of-concept by developing IL-Net, a furcated network for predicting the properties of ionic liquids, which is a class of complex multi-chemicals entities. Compared to existing state-of-the-art approaches, we show that furcated networks can improve model accuracy by approximately 20-35%, without using additional labeled data. Lastly, we distill two key design principles for furcated networks that can be adapted to other domains.

arXiv commentSubmitted to peer-reviewed ML conference

References in corpus (7)

• An Overview of Multi-Task Learning in Deep Neural Networks
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery
• Massively Multitask Networks for Drug Discovery
• Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Models
• SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties
• Learning to SMILE(S)