12 citations · 20 across the 7 of their papers we have counts for
8 papers
Deep learning and the rate of approximation by flows
Jingpu Cheng, Qianxiao Li, Ting Lin +1
We investigate the dependence of the approximation capacity of deep residual networks on its depth in a continuous dynamical systems setting. This can be formulated as the general…
A unified framework for establishing the universal approximation of transformer-type architectures
Jingpu Cheng, Ting Lin, Zuowei Shen +1
We investigate the universal approximation property (UAP) of transformer-type architectures, providing a unified theoretical framework that extends prior results on residual networ…
Interpolation, Approximation and Controllability of Deep Neural Networks
Jingpu Cheng, Qianxiao Li, Ting Lin +1
We investigate the expressive power of deep residual neural networks idealized as continuous dynamical systems through control theory. Specifically, we consider two properties that…
On the Universal Approximation Property of Deep Fully Convolutional Neural Networks
Ting Lin, Zuowei Shen, Qianxiao Li
We study the approximation of shift-invariant or equivariant functions by deep fully convolutional networks from the dynamical systems perspective. We prove that deep residual full…
Deep Neural Network Approximation of Invariant Functions through Dynamical Systems
Qianxiao Li, Ting Lin, Zuowei Shen
We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems. Such invariant functions…
Optimization in Machine Learning: A Distribution Space Approach
Yongqiang Cai, Qianxiao Li, Zuowei Shen
We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space, but with a non…
Deep Learning via Dynamical Systems: An Approximation Perspective
Qianxiao Li, Ting Lin, Zuowei Shen
We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective. In…
A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent
Yongqiang Cai, Qianxiao Li, Zuowei Shen
Despite its empirical success and recent theoretical progress, there generally lacks a quantitative analysis of the effect of batch normalization (BN) on the convergence and stabil…