Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach

Abstract

Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML pipelines is sub-optimal, since it cannot jointly optimize multiple components. To this end, we propose a framework that translates a pre-trained ML pipeline into a neural network and fine-tunes the ML models within the pipeline jointly using backpropagation. Our experiments show that fine-tuning of the translated pipelines is a promising technique able to increase the final accuracy.

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Scikit-learn: Machine Learning in Python
• The NumPy array: a structure for efficient numerical computation
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Achieving Human Parity on Automatic Chinese to English News Translation
• Deep Neural Decision Trees