48 citations · 413 across the 59 of their papers we have counts for
8 papers · 1 filter
Certifying Residual Architectures from Their Primitives: Growth Exponents and Criticality
Hyemin Gu, Michael Tyrrell, Tuhin Sahai +1
Whether a deep residual architecture avoids numerical blow-up in its forward and backward dynamics is typically determined through costly trial-and-error training. Existing stabili…
Vocabulary-size-independent Convergence of Discrete Diffusion Models: adjoint equations induce the right space
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
Discrete diffusion has become a leading framework for generative modeling in various applications including language, vision, and biology. Existing convergence theory, however, exh…
Stability of Transformers under Layer Normalization
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +4
Despite their widespread use, training deep Transformers can be unstable. Layer normalization, a standard component, improves training stability, but its placement has often been a…
Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture…
Structure-preserving GANs
Jeremiah Birrell, Markos A. Katsoulakis, Luc Rey-Bellet +1
Generative adversarial networks (GANs), a class of distribution-learning methods based on a two-player game between a generator and a discriminator, can generally be formulated as…
Mutual Information for Explainable Deep Learning of Multiscale Systems
Søren Taverniers, Eric J. Hall, Markos A. Katsoulakis +1
Timely completion of design cycles for complex systems ranging from consumer electronics to hypersonic vehicles relies on rapid simulation-based prototyping. The latter typically i…
Optimizing Variational Representations of Divergences and Accelerating their Statistical Estimation
Jeremiah Birrell, Markos A. Katsoulakis, Yannis Pantazis
Variational representations of divergences and distances between high-dimensional probability distributions offer significant theoretical insights and practical advantages in numer…
Cumulant GAN
Yannis Pantazis, Dipjyoti Paul, Michail Fasoulakis +2
In this paper, we propose a novel loss function for training Generative Adversarial Networks (GANs) aiming towards deeper theoretical understanding as well as improved stability an…