ReLUs Are Sufficient for Learning Implicit Neural Representations

Joseph Shenouda, Yamin Zhou, Robert D. Nowak

Motivated by the growing theoretical understanding of neural networks that employ the Rectified Linear Unit (ReLU) as their activation function, we revisit the use of ReLU activati…