Unveiling the Training Dynamics of ReLU Networks through a Linear Lens

Longqing Ye

Deep neural networks, particularly those employing Rectified Linear Units (ReLU), are often perceived as complex, high-dimensional, non-linear systems. This complexity poses a sign…