1 citations · 1 across the 3 of their papers we have counts for
5 papers
Deep learning and the rate of approximation by flows
Jingpu Cheng, Qianxiao Li, Ting Lin +1
We investigate the dependence of the approximation capacity of deep residual networks on its depth in a continuous dynamical systems setting. This can be formulated as the general…
A unified framework for establishing the universal approximation of transformer-type architectures
Jingpu Cheng, Ting Lin, Zuowei Shen +1
We investigate the universal approximation property (UAP) of transformer-type architectures, providing a unified theoretical framework that extends prior results on residual networ…
Optimization in Machine Learning: A Distribution Space Approach
Yongqiang Cai, Qianxiao Li, Zuowei Shen
We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space, but with a non…
Deep Learning via Dynamical Systems: An Approximation Perspective
Qianxiao Li, Ting Lin, Zuowei Shen
We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective. In…
A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent
Yongqiang Cai, Qianxiao Li, Zuowei Shen
Despite its empirical success and recent theoretical progress, there generally lacks a quantitative analysis of the effect of batch normalization (BN) on the convergence and stabil…