The inverse problem for neural networks

Abstract

We study the problem of computing the preimage of a set under a neural network with piecewise-affine activation functions. We recall an old result that the preimage of a polyhedral set is again a union of polyhedral sets and can be effectively computed. We show several applications of computing the preimage for analysis and interpretability of neural networks.

References in corpus (4)

• Open- and Closed-Loop Neural Network Verification using Polynomial Zonotopes
• Verification of Neural-Network Control Systems by Integrating Taylor Models and Zonotopes
• Neural Network Compression of ACAS Xu Early Prototype is Unsafe: Closed-Loop Verification through Quantized State Backreachability
• Provably Bounding Neural Network Preimages