Interpretable Few-Shot Learning via Linear Distillation

Abstract

It is important to develop mathematically tractable models than can interpret knowledge extracted from the data and provide reasonable predictions. In this paper, we present a Linear Distillation Learning, a simple remedy to improve the performance of linear neural networks. Our approach is based on using a linear function for each class in a dataset, which is trained to simulate the output of a teacher linear network for each class separately. We tested our model on MNIST and Omniglot datasets in the Few-Shot learning manner. It showed better results than other interpretable models such as classical Logistic Regression.

References in corpus (9)

• Distilling the Knowledge in a Neural Network
• Interpretable machine learning: definitions, methods, and applications
• Do Deep Nets Really Need to be Deep?
• Exploration by Random Network Distillation
• Deep Model Compression: Distilling Knowledge from Noisy Teachers
• Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
• Policy Distillation
• Depth Creates No Bad Local Minima
• Bootstrapped Thompson Sampling and Deep Exploration