371 citations · 617 across the 6 of their papers we have counts for
4 papers · 1 filter
Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations
Qianxiao Li, Cheng Tai, Weinan E
We develop the mathematical foundations of the stochastic modified equations (SME) framework for analyzing the dynamics of stochastic gradient algorithms, where the latter is appro…
Maximum Principle Based Algorithms for Deep Learning
Qianxiao Li, Long Chen, Cheng Tai +1
The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and…
Stochastic modified equations and adaptive stochastic gradient algorithms
Qianxiao Li, Cheng Tai, Weinan E
We develop the method of stochastic modified equations (SME), in which stochastic gradient algorithms are approximated in the weak sense by continuous-time stochastic differential…
Convolutional neural networks with low-rank regularization
Cheng Tai, Tong Xiao, Yi Zhang +2
Large CNNs have delivered impressive performance in various computer vision applications. But the storage and computation requirements make it problematic for deploying these model…