Learning to Teach with Dynamic Loss Functions

Abstract

Teaching is critical to human society: it is with teaching that prospective students are educated and human civilization can be inherited and advanced. A good teacher not only provides his/her students with qualified teaching materials (e.g., textbooks), but also sets up appropriate learning objectives (e.g., course projects and exams) considering different situations of a student. When it comes to artificial intelligence, treating machine learning models as students, the loss functions that are optimized act as perfect counterparts of the learning objective set by the teacher. In this work, we explore the possibility of imitating human teaching behaviors by dynamically and automatically outputting appropriate loss functions to train machine learning models. Different from typical learning settings in which the loss function of a machine learning model is predefined and fixed, in our framework, the loss function of a machine learning model (we call it student) is defined by another machine learning model (we call it teacher). The ultimate goal of teacher model is cultivating the student to have better performance measured on development dataset. Towards that end, similar to human teaching, the teacher, a parametric model, dynamically outputs different loss functions that will be used and optimized by its student model at different training stages. We develop an efficient learning method for the teacher model that makes gradient based optimization possible, exempt of the ineffective solutions such as policy optimization. We name our method as "learning to teach with dynamic loss functions" (L2T-DLF for short). Extensive experiments on real world tasks including image classification and neural machine translation demonstrate that our method significantly improves the quality of various student models.

arXiv commentNIPS 2018

Cited by in corpus (32)

• Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting
• Incorporating BERT into Neural Machine Translation
• Meta-learning PINN loss functions
• Spectral Reconstruction with Deep Neural Networks
• Learning to Generalize from Sparse and Underspecified Rewards
• Deep Ice Layer Tracking and Thickness Estimation using Fully Convolutional Networks
• MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler
• Learning Surrogate Losses
• Online Meta-Critic Learning for Off-Policy Actor-Critic Methods
• Experience Replay Optimization
• Learning Adaptive Loss for Robust Learning with Noisy Labels
• AM-LFS: AutoML for Loss Function Search
• A Survey on Curriculum Learning
• Learning a Matching Model with Co-teaching for Multi-turn Response Selection in Retrieval-based Dialogue Systems
• Learning Effective Loss Functions Efficiently
• Learning to Match Jobs with Resumes from Sparse Interaction Data using Multi-View Co-Teaching Network
• Optimizing Black-box Metrics with Adaptive Surrogates
• Teaching with Commentaries
• Learning Data Teaching Strategies Via Knowledge Tracing
• A Competence-aware Curriculum for Visual Concepts Learning via Question Answering
• Leveraging Undiagnosed Data for Glaucoma Classification with Teacher-Student Learning
• VIABLE: Fast Adaptation via Backpropagating Learned Loss
• MetricOpt: Learning to Optimize Black-Box Evaluation Metrics
• Iterative Teaching by Label Synthesis
• MetaAugment: Sample-Aware Data Augmentation Policy Learning
• Adma: A Flexible Loss Function for Neural Networks
• Learning to Reweight with Deep Interactions
• Generating meta-learning tasks to evolve parametric loss for classification learning
• Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions
• A Unified Framework of Surrogate Loss by Refactoring and Interpolation
• MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding
• DCUR: Data Curriculum for Teaching via Samples with Reinforcement Learning