Black-Box Ripper: Copying black-box models using generative evolutionary algorithms

Abstract

We study the task of replicating the functionality of black-box neural models, for which we only know the output class probabilities provided for a set of input images. We assume back-propagation through the black-box model is not possible and its training images are not available, e.g. the model could be exposed only through an API. In this context, we present a teacher-student framework that can distill the black-box (teacher) model into a student model with minimal accuracy loss. To generate useful data samples for training the student, our framework (i) learns to generate images on a proxy data set (with images and classes different from those used to train the black-box) and (ii) applies an evolutionary strategy to make sure that each generated data sample exhibits a high response for a specific class when given as input to the black box. Our framework is compared with several baseline and state-of-the-art methods on three benchmark data sets. The empirical evidence indicates that our model is superior to the considered baselines. Although our method does not back-propagate through the black-box network, it generally surpasses state-of-the-art methods that regard the teacher as a glass-box model. Our code is available at: https://github.com/antoniobarbalau/black-box-ripper.

arXiv commentAccepted as Oral at NeurIPS 2020

References in corpus (13)

• Auto-Encoding Variational Bayes
• Distilling the Knowledge in a Neural Network
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Spectral Normalization for Generative Adversarial Networks
• Do Deep Nets Really Need to be Deep?
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Thieves on Sesame Street! Model Extraction of BERT-based APIs
• Zero-Shot Knowledge Distillation in Deep Networks
• Disconnected Manifold Learning for Generative Adversarial Networks
• Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
• Towards Reverse-Engineering Black-Box Neural Networks
• Dream Distillation: A Data-Independent Model Compression Framework
• Few Shot Network Compression via Cross Distillation

Cited by in corpus (1)

• Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithm