11 papers
Stable initialization without the CLT
Simon Kuang, Kyle Chickering, Xinfan Lin
Successful training of deep neural networks is highly dependent on the distribution of the initial weights. If the weights are too large, network training blows up; if they are too…
Incremental stability in and : classification and synthesis
Simon Kuang, Xinfan Lin
All Lipschitz dynamics with the weak infinitesimal contraction (WIC) property can be expressed as a Lipschitz nonlinear system in proportional negative feedback -- this statement,…
Demystifying Lipschitz verification: positive matrices, negative results
Simon Kuang, Yuezhu Xu, S. Sivaranjani +1
The global Lipschitz constant of a neural network is related to robustness and generalization, yet unlike in many classical models, it is not plainly legible from the parameters. T…
Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method
Simon Kuang, Xinfan Lin
We study the problem of propagating the mean and covariance of a general multivariate Gaussian distribution through a deep (residual) neural network using layer-by-layer moment mat…
Instrumental variables system identification with consistency
Simon Kuang, Xinfan Lin
Instrumental variables (eliminate the bias that afflicts least-squares identification of dynamical systems through noisy data, yet traditionally relies on external instruments that…
Assumed Density Filtering and Smoothing with Neural Network Surrogate Models
Simon Kuang, Xinfan Lin
The Kalman filter and Rauch-Tung-Striebel (RTS) smoother are optimal for state estimation in linear dynamic systems. With nonlinear systems, the challenge consists in how to propag…
Debiasing Continuous-time Nonlinear Autoregressions
Simon Kuang, Xinfan Lin
We study how to identify a class of continuous-time nonlinear systems defined by an ordinary differential equation affine in the unknown parameter. We define a notion of asymptotic…
Structural Constraints for Physics-augmented Learning
Simon Kuang, Xinfan Lin
When the physics is wrong, physics-informed machine learning becomes physics-misinformed machine learning. A powerful black-box model should not be able to conceal misconceived phy…
A New Type of Nonlinear Disturbance Rejection
Simon Kuang, Xinfan Lin
Asymptotic disturbance rejection (equivalently tracking) for nonlinear systems has been studied only in qualitative terms (the state is asymptotically stable under bounded disturba…
Estimation Sample Complexity of a Class of Nonlinear Continuous-time Systems
Simon Kuang, Xinfan Lin
We present a method of parameter estimation for large class of nonlinear systems, namely those in which the state consists of output derivatives and the flow is linear in the param…
Dirty derivative stability in the frequency domain
Simon Kuang
Differentiation filters can be made proper by composition with a first-order low-pass filter at a desired bandwidth, resulting in a "dirty derivative." A stable closed-loop system…