6 papers · 1 filter
Random ReLU Neural Networks as Non-Gaussian Processes
Rahul Parhi, Pakshal Bohra, Ayoub El Biari +2
We consider a large class of shallow neural networks with randomly initialized parameters and rectified linear unit activation functions. We prove that these random neural networks…
Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities
Rahul Parhi, Michael Unser
We investigate the function-space optimality (specifically, the Banach-space optimality) of a large class of shallow neural architectures with multivariate nonlinearities/activatio…
Weighted variation spaces and approximation by shallow ReLU networks
Ronald DeVore, Robert D. Nowak, Rahul Parhi +1
We investigate the approximation of functions on a bounded domain by the outputs of single-hidden-layer ReLU neural networks of width . This form of…
Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression
Joseph Shenouda, Rahul Parhi, Kangwook Lee +1
This paper introduces a novel theoretical framework for the analysis of vector-valued neural networks through the development of vector-valued variation spaces, a new class of repr…
Banach Space Representer Theorems for Neural Networks and Ridge Splines
Rahul Parhi, Robert D. Nowak
We develop a variational framework to understand the properties of the functions learned by neural networks fit to data. We propose and study a family of continuous-domain linear i…
The Role of Neural Network Activation Functions
Rahul Parhi, Robert D. Nowak
A wide variety of activation functions have been proposed for neural networks. The Rectified Linear Unit (ReLU) is especially popular today. There are many practical reasons that m…