18 citations · 68 across the 27 of their papers we have counts for
8 papers · 1 filter
Certifying Residual Architectures from Their Primitives: A Sharp Stability Threshold
Hyemin Gu, Michael Tyrrell, Tuhin Sahai +1
Whether a deep residual architecture trains stably is usually determined by training it, which is expensive and answers the question only for the architecture, depth, and floating-…
Vocabulary-size-independent Convergence of Discrete Diffusion Models: adjoint equations induce the right space
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
Discrete diffusion has become a leading framework for generative modeling in various applications including language, vision, and biology. Existing convergence theory, however, exh…
Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions
Noah Schwartz, Chandra Kanth Nagesh, Sriram Sankaranarayanan +3
We present a generalized framework for the range verification of neural networks featuring non-linear activation functions. Our approach first constructs an ``optimized piecewise a…
Stability of Transformers under Layer Normalization
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +4
Despite their widespread use, training deep Transformers can be unstable. Layer normalization, a standard component, improves training stability, but its placement has often been a…
Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations
Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur +2
We study the problem of learning neural network models for Ordinary Differential Equations (ODEs) with parametric uncertainties. Such neural network models capture the solution to…
Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture…