Into the Unknown: Active Monitoring of Neural Networks
arXiv:2009.06429 · doi:10.1007/978-3-030-88494-9_3
Abstract
Neural-network classifiers achieve high accuracy when predicting the class of an input that they were trained to identify. Maintaining this accuracy in dynamic environments, where inputs frequently fall outside the fixed set of initially known classes, remains a challenge. The typical approach is to detect inputs from novel classes and retrain the classifier on an augmented dataset. However, not only the classifier but also the detection mechanism needs to adapt in order to distinguish between newly learned and yet unknown input classes. To address this challenge, we introduce an algorithmic framework for active monitoring of a neural network. A monitor wrapped in our framework operates in parallel with the neural network and interacts with a human user via a series of interpretable labeling queries for incremental adaptation. In addition, we propose an adaptive quantitative monitor to improve precision. An experimental evaluation on a diverse set of benchmarks with varying numbers of classes confirms the benefits of our active monitoring framework in dynamic scenarios.
published at RV 2021
References in corpus (8)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- On Calibration of Modern Neural Networks
- Selective Classification for Deep Neural Networks
- Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends
- Learning from Few Samples: A Survey
- Customizable Reference Runtime Monitoring of Neural Networks using Resolution Boxes
- Hack The Box: Fooling Deep Learning Abstraction-Based Monitors
- Monitoring Object Detection Abnormalities via Data-Label and Post-Algorithm Abstractions