Neural networks behave as hash encoders: An empirical study

Fengxiang He, Shiye Lei, Jianmin Ji +1

The input space of a neural network with ReLU-like activations is partitioned into multiple linear regions, each corresponding to a specific activation pattern of the included ReLU…