40 citations · 139 across the 37 of their papers we have counts for
18 papers · 1 filter
Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform
Sho Sonoda, Luca Galimberti, Yuanchao Xu +2
Neural networks whose inputs are functions, as in operator learning, are usually shown to be universal by existence arguments, either by reduction to finite-dimensional networks or…
Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs
Yuka Hashimoto, Sho Sonoda, Isao Ishikawa +1
We derive a new Rademacher complexity bound for deep neural networks using Koopman operators, group representations, and reproducing kernel Hilbert spaces (RKHSs). The proposed bou…
Why and When Deep is Better than Shallow: Implementation-Agnostic State-Transition Model of Deep Learning
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We ask when adding hidden layers improves generalization, in a model that keeps the layers fixed and varies only their number. A hidden layer is a self-map of a state space, a dept…
Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, called the joint-equivariant machines, based on…
A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
To investigate neural network parameters, it is easier to study the distribution of parameters than to study the parameters in each neuron. The ridgelet transform is a pseudo-inver…
Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks
Sho Sonoda, Hideyuki Ishi, Isao Ishikawa +1
The symmetry and geometry of input data are considered to be encoded in the internal data representation inside the neural network, but the specific encoding rule has been less inv…
Deep Ridgelet Transform: Voice with Koopman Operator Proves Universality of Formal Deep Networks
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We identify hidden layers inside a deep neural network (DNN) with group actions on the data domain, and formulate a formal deep network as a dual voice transform with respect to th…
Koopman-based generalization bound: New aspect for full-rank weights
Yuka Hashimoto, Sho Sonoda, Isao Ishikawa +2
We propose a new bound for generalization of neural networks using Koopman operators. Whereas most of existing works focus on low-rank weight matrices, we focus on full-rank weight…
Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
We show the universality of depth-2 group convolutional neural networks (GCNNs) in a unified and constructive manner based on the ridgelet theory. Despite widespread use in applica…
Universal approximation property of invertible neural networks
Isao Ishikawa, Takeshi Teshima, Koichi Tojo +3
Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various m…
Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
Neural network on Riemannian symmetric space such as hyperbolic space and the manifold of symmetric positive definite (SPD) matrices is an emerging subject of research in geometric…
Koopman Spectrum Nonlinear Regulators and Efficient Online Learning
Motoya Ohnishi, Isao Ishikawa, Kendall Lowrey +3
Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, beh…
Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
We study parameter nonuniqueness in continuous-width depth-two fully connected neural networks. Our main contribution is a direct method for solving the neural-network equation $S[…
Universal Approximation Property of Neural Ordinary Differential Equations
Takeshi Teshima, Koichi Tojo, Masahiro Ikeda +2
Neural ordinary differential equations (NODEs) is an invertible neural network architecture promising for its free-form Jacobian and the availability of a tractable Jacobian determ…
A global universality of two-layer neural networks with ReLU activations
Naoya Hatano, Masahiro Ikeda, Isao Ishikawa +1
In the present study, we investigate a universality of neural networks, which concerns a density of the set of two-layer neural networks in a function spaces. There are many works…
Ridge Regression with Over-Parametrized Two-Layer Networks Converge to Ridgelet Spectrum
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
Characterization of local minima draws much attention in theoretical studies of deep learning. In this study, we investigate the distribution of parameters in an over-parametrized…
Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
Takeshi Teshima, Isao Ishikawa, Koichi Tojo +3
Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning. However, their desirabl…
Krylov Subspace Method for Nonlinear Dynamical Systems with Random Noise
Yuka Hashimoto, Isao Ishikawa, Masahiro Ikeda +2
Operator-theoretic analysis of nonlinear dynamical systems has attracted much attention in a variety of engineering and scientific fields, endowed with practical estimation methods…