229 citations · 303 across the 26 of their papers we have counts for
19 papers · 1 filter
Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform
Sho Sonoda, Luca Galimberti, Yuanchao Xu +2
Neural networks whose inputs are functions, as in operator learning, are usually shown to be universal by existence arguments, either by reduction to finite-dimensional networks or…
Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEs
Koichi Taniguchi, Sho Sonoda
Operator learning for partial differential equations (PDEs) aims to learn solution operators on infinite-dimensional function spaces from finite-resolution data. In this setting, i…
Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs
Yuka Hashimoto, Sho Sonoda, Isao Ishikawa +1
We derive a new Rademacher complexity bound for deep neural networks using Koopman operators, group representations, and reproducing kernel Hilbert spaces (RKHSs). The proposed bou…
Why and When Deep is Better than Shallow: Implementation-Agnostic State-Transition Model of Deep Learning
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We ask when adding hidden layers improves generalization, in a model that keeps the layers fixed and varies only their number. A hidden layer is a self-map of a state space, a dept…
Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, called the joint-equivariant machines, based on…
A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
To investigate neural network parameters, it is easier to study the distribution of parameters than to study the parameters in each neuron. The ridgelet transform is a pseudo-inver…
Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks
Sho Sonoda, Hideyuki Ishi, Isao Ishikawa +1
The symmetry and geometry of input data are considered to be encoded in the internal data representation inside the neural network, but the specific encoding rule has been less inv…
Deep Ridgelet Transform: Voice with Koopman Operator Proves Universality of Formal Deep Networks
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We identify hidden layers inside a deep neural network (DNN) with group actions on the data domain, and formulate a formal deep network as a dual voice transform with respect to th…
Koopman-based generalization bound: New aspect for full-rank weights
Yuka Hashimoto, Sho Sonoda, Isao Ishikawa +2
We propose a new bound for generalization of neural networks using Koopman operators. Whereas most of existing works focus on low-rank weight matrices, we focus on full-rank weight…
Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
We show the universality of depth-2 group convolutional neural networks (GCNNs) in a unified and constructive manner based on the ridgelet theory. Despite widespread use in applica…
Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
Neural network on Riemannian symmetric space such as hyperbolic space and the manifold of symmetric positive definite (SPD) matrices is an emerging subject of research in geometric…
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
Kaito Watanabe, Kotaro Sakamoto, Ryo Karakida +2
A biological neural network in the cortex forms a neural field. Neurons in the field have their own receptive fields, and connection weights between two neurons are random but high…
Differentiable Multiple Shooting Layers
Stefano Massaroli, Michael Poli, Sho Sonoda +4
We detail a novel class of implicit neural models. Leveraging time-parallel methods for differential equations, Multiple Shooting Layers (MSLs) seek solutions of initial value prob…
Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
We study parameter nonuniqueness in continuous-width depth-two fully connected neural networks. Our main contribution is a direct method for solving the neural-network equation $S[…
How Powerful are Shallow Neural Networks with Bandlimited Random Weights?
Ming Li, Sho Sonoda, Feilong Cao +2
We investigate the expressive power of depth-2 bandlimited random neural networks. A random net is a neural network where the hidden layer parameters are frozen with random assignm…
Ridge Regression with Over-Parametrized Two-Layer Networks Converge to Ridgelet Spectrum
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
Characterization of local minima draws much attention in theoretical studies of deep learning. In this study, we investigate the distribution of parameters in an over-parametrized…
Transportation analysis of denoising autoencoders: a novel method for analyzing deep neural networks
Sho Sonoda, Noboru Murata
The feature map obtained from the denoising autoencoder (DAE) is investigated by determining transportation dynamics of the DAE, which is a cornerstone for deep learning. Despite t…
Transport Analysis of Infinitely Deep Neural Network
Sho Sonoda, Noboru Murata
We investigated the feature map inside deep neural networks (DNNs) by tracking the transport map. We are interested in the role of depth (why do DNNs perform better than shallow mo…
Nonparametric Weight Initialization of Neural Networks via Integral Representation
Sho Sonoda, Noboru Murata
A new initialization method for hidden parameters in a neural network is proposed. Derived from the integral representation of the neural network, a nonparametric probability distr…