From the 1 of 8 linked papers with an AI index.
8 papers
Sharp Stability Threshold and Certification for Designing Stable Residual Architectures
Hyemin Gu, Michael Tyrrell, Tuhin Sahai +1
The paper introduces a sublinear‑growth principle that gives a sharp stability condition (input‑magnitude exponent q ≤ 1) for deep residual networks, and provides a method to certi…
Dimension-Free Convergence of Discrete Diffusion Models: Adjoint Equations Induce the Right Space
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
Discrete diffusion has become a leading framework for generative modeling in various applications including language, vision, and biology. Existing convergence theory, however, exh…
ISOMORPH: A Supply Chain Digital Twin for Simulation, Dataset Generation, and Forecasting Benchmarks
Zhizhen Zhang, Hyemin Gu, Benjamin J. Zhang +6
Open time-series forecasting (TSF) benchmarks cover retail, energy, weather, and traffic, but supply-chain logistics remains underserved. We introduce ISOMORPH, the first public di…
Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions
Noah Schwartz, Chandra Kanth Nagesh, Sriram Sankaranarayanan +3
We present a generalized framework for the range verification of neural networks featuring non-linear activation functions. Our approach first constructs an ``optimized piecewise a…
Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture…
Stability of Transformers under Layer Normalization
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +4
Despite their widespread use, training deep Transformers can be unstable. Layer normalization, a standard component, improves training stability, but its placement has often been a…