229 citations · 303 across the 26 of their papers we have counts for
27 papers
Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform
Sho Sonoda, Luca Galimberti, Yuanchao Xu +2
Neural networks whose inputs are functions, as in operator learning, are usually shown to be universal by existence arguments, either by reduction to finite-dimensional networks or…
Winning Lottery Tickets in Neural Networks via a Quantum-Inspired Classical Algorithm
Natsuto Isogai, Hayata Yamasaki, Sho Sonoda +1
Quantum machine learning (QML) aims to accelerate machine learning tasks by exploiting quantum computation. Previous work studied a QML algorithm for selecting sparse subnetworks f…
Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEs
Koichi Taniguchi, Sho Sonoda
Operator learning for partial differential equations (PDEs) aims to learn solution operators on infinite-dimensional function spaces from finite-resolution data. In this setting, i…
Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs
Yuka Hashimoto, Sho Sonoda, Isao Ishikawa +1
We derive a new Rademacher complexity bound for deep neural networks using Koopman operators, group representations, and reproducing kernel Hilbert spaces (RKHSs). The proposed bou…
Why and When Deep is Better than Shallow: Implementation-Agnostic State-Transition Model of Deep Learning
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We ask when adding hidden layers improves generalization, in a model that keeps the layers fixed and varies only their number. A hidden layer is a self-map of a state space, a dept…
Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, called the joint-equivariant machines, based on…
A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networks
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
To investigate neural network parameters, it is easier to study the distribution of parameters than to study the parameters in each neuron. The ridgelet transform is a pseudo-inver…
Joint Group Invariant Functions on Data-Parameter Domain Induce Universal Neural Networks
Sho Sonoda, Hideyuki Ishi, Isao Ishikawa +1
The symmetry and geometry of input data are considered to be encoded in the internal data representation inside the neural network, but the specific encoding rule has been less inv…
Deep Ridgelet Transform: Voice with Koopman Operator Proves Universality of Formal Deep Networks
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa +1
We identify hidden layers inside a deep neural network (DNN) with group actions on the data domain, and formulate a formal deep network as a dual voice transform with respect to th…
LPML: LLM-Prompting Markup Language for Mathematical Reasoning
Ryutaro Yamauchi, Sho Sonoda, Akiyoshi Sannai +1
In utilizing large language models (LLMs) for mathematical reasoning, addressing the errors in the reasoning and calculation present in the generated text by LLMs is a crucial chal…
Koopman-based generalization bound: New aspect for full-rank weights
Yuka Hashimoto, Sho Sonoda, Isao Ishikawa +2
We propose a new bound for generalization of neural networks using Koopman operators. Whereas most of existing works focus on low-rank weight matrices, we focus on full-rank weight…
Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation
Hayata Yamasaki, Sathyawageeswar Subramanian, Satoshi Hayakawa +1
A significant challenge in the field of quantum machine learning (QML) is to establish applications of quantum computation to accelerate common tasks in machine learning such as th…
Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
We show the universality of depth-2 group convolutional neural networks (GCNNs) in a unified and constructive manner based on the ridgelet theory. Despite widespread use in applica…
Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
Neural network on Riemannian symmetric space such as hyperbolic space and the manifold of symmetric positive definite (SPD) matrices is an emerging subject of research in geometric…
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
Kaito Watanabe, Kotaro Sakamoto, Ryo Karakida +2
A biological neural network in the cortex forms a neural field. Neurons in the field have their own receptive fields, and connection weights between two neurons are random but high…
Differentiable Multiple Shooting Layers
Stefano Massaroli, Michael Poli, Sho Sonoda +4
We detail a novel class of implicit neural models. Leveraging time-parallel methods for differential equations, Multiple Shooting Layers (MSLs) seek solutions of initial value prob…
Exponential Error Convergence in Data Classification with Optimized Random Features: Acceleration by Quantum Machine Learning
Hayata Yamasaki, Sho Sonoda
Classification is a common task in machine learning. Random features (RFs) stand as a central technique for scalable learning algorithms based on kernel methods, and more recently…
Ghosts in Neural Networks: Existence, Structure and Role of Infinite-Dimensional Null Space
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
We study parameter nonuniqueness in continuous-width depth-two fully connected neural networks. Our main contribution is a direct method for solving the neural-network equation $S[…
How Powerful are Shallow Neural Networks with Bandlimited Random Weights?
Ming Li, Sho Sonoda, Feilong Cao +2
We investigate the expressive power of depth-2 bandlimited random neural networks. A random net is a neural network where the hidden layer parameters are frozen with random assignm…
Ridge Regression with Over-Parametrized Two-Layer Networks Converge to Ridgelet Spectrum
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
Characterization of local minima draws much attention in theoretical studies of deep learning. In this study, we investigate the distribution of parameters in an over-parametrized…
Learning with Optimized Random Features: Exponential Speedup by Quantum Machine Learning without Sparsity and Low-Rank Assumptions
Hayata Yamasaki, Sathyawageeswar Subramanian, Sho Sonoda +1
Kernel methods augmented with random features give scalable algorithms for learning from big data. But it has been computationally hard to sample random features according to a pro…
Fast Approximation and Estimation Bounds of Kernel Quadrature for Infinitely Wide Models
Sho Sonoda
An infinitely wide model is a weighted integration of feature maps. This model excels at handling an infinite number of features, and thus it has been adopted…
The global optimum of shallow neural network is attained by ridgelet transform
Sho Sonoda, Isao Ishikawa, Masahiro Ikeda +4
We prove that the global minimum of the backpropagation (BP) training problem of neural networks with an arbitrary nonlinear activation is given by the ridgelet transform. A series…
Transportation analysis of denoising autoencoders: a novel method for analyzing deep neural networks
Sho Sonoda, Noboru Murata
The feature map obtained from the denoising autoencoder (DAE) is investigated by determining transportation dynamics of the DAE, which is a cornerstone for deep learning. Despite t…