5 citations · 15 across the 8 of their papers we have counts for
6 papers · 1 filter
Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation
Li'ang Li, Yifei Duan, Guanghua Ji +1
The study of universal approximation properties (UAP) for neural networks (NN) has a long history. When the network width is unlimited, only a single hidden layer is sufficient for…
Vocabulary for Universal Approximation: A Linguistic Perspective of Mapping Compositions
Yongqiang Cai
In recent years, deep learning-based sequence modelings, such as language models, have received much attention and success, which pushes researchers to explore the possibility of t…
Achieve the Minimum Width of Neural Networks for Universal Approximation
Yongqiang Cai
The universal approximation property (UAP) of neural networks is fundamental for deep learning, and it is well known that wide neural networks are universal approximators of contin…
Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems
Yifei Duan, Li'ang Li, Guanghua Ji +1
Deep learning has made significant applications in the field of data science and natural science. Some studies have linked deep neural networks to dynamic systems, but the network…
Optimization in Machine Learning: A Distribution Space Approach
Yongqiang Cai, Qianxiao Li, Zuowei Shen
We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space, but with a non…
A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent
Yongqiang Cai, Qianxiao Li, Zuowei Shen
Despite its empirical success and recent theoretical progress, there generally lacks a quantitative analysis of the effect of batch normalization (BN) on the convergence and stabil…