Reverse-Engineering Deep ReLU Networks

Abstract

It has been widely assumed that a neural network cannot be recovered from its outputs, as the network depends on its parameters in a highly nonlinear way. Here, we prove that in fact it is often possible to identify the architecture, weights, and biases of an unknown deep ReLU network by observing only its output. Every ReLU network defines a piecewise linear function, where the boundaries between linear regions correspond to inputs for which some neuron in the network switches between inactive and active ReLU states. By dissecting the set of region boundaries into components associated with particular neurons, we show both theoretically and empirically that it is possible to recover the weights of neurons and their arrangement within the network, up to isomorphism.

15 pages, 4 figures

References in corpus (5)

• Representation Benefits of Deep Feedforward Networks
• Deep ReLU Networks Have Surprisingly Few Activation Patterns
• High Accuracy and High Fidelity Extraction of Neural Networks
• Reliably Learning the ReLU in Polynomial Time
• Learning Neural Networks with Two Nonlinear Layers in Polynomial Time

Cited by in corpus (4)

• An Embedding of ReLU Networks and an Analysis of their Identifiability
• Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension
• When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence
• PDE constraints on smooth hierarchical functions computed by neural networks