4 papers · 1 filter
Stable initialization without the CLT
Simon Kuang, Kyle Chickering, Xinfan Lin
Successful training of deep neural networks is highly dependent on the distribution of the initial weights. If the weights are too large, network training blows up; if they are too…
Incremental stability in and : classification and synthesis
Simon Kuang, Xinfan Lin
All Lipschitz dynamics with the weak infinitesimal contraction (WIC) property can be expressed as a Lipschitz nonlinear system in proportional negative feedback -- this statement,…
Demystifying Lipschitz verification: positive matrices, negative results
Simon Kuang, Yuezhu Xu, S. Sivaranjani +1
The global Lipschitz constant of a neural network is related to robustness and generalization, yet unlike in many classical models, it is not plainly legible from the parameters. T…
Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method
Simon Kuang, Xinfan Lin
We study the problem of propagating the mean and covariance of a general multivariate Gaussian distribution through a deep (residual) neural network using layer-by-layer moment mat…