Learning to Actively Learn: A Robust Approach

Abstract

This work proposes a procedure for designing algorithms for specific adaptive data collection tasks like active learning and pure-exploration multi-armed bandits. Unlike the design of traditional adaptive algorithms that rely on concentration of measure and careful analysis to justify the correctness and sample complexity of the procedure, our adaptive algorithm is learned via adversarial training over equivalence classes of problems derived from information theoretic lower bounds. In particular, a single adaptive learning algorithm is learned that competes with the best adaptive algorithm learned for each equivalence class. Our procedure takes as input just the available queries, set of hypotheses, loss function, and total query budget. This is in contrast to existing meta-learning work that learns an adaptive algorithm relative to an explicit, user-defined subset or prior distribution over problems which can be challenging to define and be mismatched to the instance encountered at test time. This work is particularly focused on the regime when the total query budget is very small, such as a few dozen, which is much smaller than those budgets typically considered by theoretically derived algorithms. We perform synthetic experiments to justify the stability and effectiveness of the training procedure, and then evaluate the method on tasks derived from real data including a noisy 20 Questions game and a joke recommendation task.

References in corpus (24)

• Adam: A Method for Stochastic Optimization
• Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits
• EPOpt: Learning Robust Neural Network Policies Using Model Ensembles
• Learning Active Learning from Data
• Optimal Best Arm Identification with Fixed Confidence
• Best-Arm Identification in Linear Bandits
• Active One-shot Learning
• Learning Algorithms for Active Learning
• Learning how to Active Learn: A Deep Reinforcement Learning Approach
• Tight (Lower) Bounds for the Fixed Budget Best Arm Identification Bandit Problem
• Exploration in Structured Reinforcement Learning
• Gamification of Pure Exploration for Linear Bandits
• An Empirical Process Approach to the Union Bound: Practical Algorithms for Combinatorial and Linear Bandits
• Fully adaptive algorithm for pure exploration in linear bandits
• Efficient and Parsimonious Agnostic Active Learning
• Meta-Learning for Contextual Bandit Exploration
• Corralling a Band of Bandit Algorithms
• The End of Optimism? An Asymptotic Analysis of Finite-Armed Linear Bandits
• Adaptive Exploration in Linear Contextual Bandit
• Differentiable Bandit Exploration
• Meta-Learning Bandit Policies by Gradient Ascent
• Meta-learning with Stochastic Linear Bandits
• Playing 20 Question Game with Policy-Based Reinforcement Learning
• Improved Algorithms for Agnostic Pool-based Active Classification