3 citations · 5 across the 11 of their papers we have counts for
11 papers
A mixed residual method for biharmonic equations in spectral Barron spaces
Mengjia Bai, Kuo Gai, Shuai Lu
We propose a mixed residual method (MIM) for numerically solving the biharmonic equation with nonhomogeneous clamped boundary conditions. By establishing the well-posedness of the…
Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic
Hu Tan, Kuo Gai, Shihua Zhang
While neural collapse (NC) predicts that a -class-balanced classifier should organize terminal representations as a -dimensional simplex equiangular tight frame (ETF), mo…
Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction
Hu Tan, Kuo Gai, Shihua Zhang
Grokking suggests that fitting the training data and learning a simple underlying rule may occur on different time scales. We formalize this phenomenon by separating the fast decay…
Deciphering Shortcut Learning from an Evolutionary Game Theory Perspective
Xiayang Li, Kuo Gai, Shihua Zhang
Shortcut learning causes deep learning models to rely on non-essential features within the data. However, its formation in deep neural network training still lacks theoretical unde…
Feature Dynamics as Implicit Data Augmentation: A Depth-Decomposed View on Deep Neural Network Generalization
Tianyu Ruan, Kuo Gai, Shihua Zhang
Why do deep networks generalize well? In contrast to classical generalization theory, we approach this fundamental question by examining not only inputs and outputs, but the evolut…
OTAD: An Optimal Transport-Induced Robust Model for Agnostic Adversarial Attack
Kuo Gai, Sicong Wang, Shihua Zhang
Deep neural networks (DNNs) are vulnerable to small adversarial perturbations of the inputs, posing a significant challenge to their reliability and robustness. Empirical methods s…
Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function
Kuo Gai, Shihua Zhang
In practice, deeper networks tend to be more powerful than shallow ones, but this has not been understood theoretically. In this paper, we find the analytical solution of a three-l…
Progressive Feedforward Collapse of ResNet Training
Sicong Wang, Kuo Gai, Shihua Zhang
Neural collapse (NC) is a simple and symmetric phenomenon for deep neural networks (DNNs) at the terminal phase of training, where the last-layer features collapse to their class m…
Deep Residual Networks Learn the Geodesic Curve in the Wasserstein Space
Kuo Gai, Shihua Zhang
Recent studies revealed the mathematical connection between deep neural networks (DNNs) and dynamic systems. However, the specific dynamics that DNNs, especially deep residual netw…
Tessellated Wasserstein Auto-Encoders
Kuo Gai, Shihua Zhang
Non-adversarial generative models such as variational auto-encoder (VAE), Wasserstein auto-encoders with maximum mean discrepancy (WAE-MMD), sliced-Wasserstein auto-encoder (SWAE)…
Matrix Normal PCA for Interpretable Dimension Reduction and Graphical Noise Modeling
Chihao Zhang, Kuo Gai, Shihua Zhang
Principal component analysis (PCA) is one of the most widely used dimension reduction and multivariate statistical techniques. From a probabilistic perspective, PCA seeks a low-dim…