Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks

Wenlin Chen, Hong Ge

We introduce a novel approach for analyzing the training dynamics of ReLU networks by examining the characteristic activation boundaries of individual ReLU neurons. Our proposed an…