2 citations · 2 across the 5 of their papers we have counts for
5 papers
Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
Hongru Yang, Bhavya Kailkhura, Zhangyang Wang +1
Understanding the training dynamics of transformers is important to explain the impressive capabilities behind large language models. In this work, we study the dynamics of trainin…
Pruning Before Training May Improve Generalization, Provably
Hongru Yang, Yingbin Liang, Xiaojie Guo +2
It has been observed in practice that applying pruning-at-initialization methods to neural networks and training the sparsified networks can not only retain the testing performance…
Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK
Hongru Yang, Ziyu Jiang, Ruizhe Zhang +2
We study training one-hidden-layer ReLU networks in the neural tangent kernel (NTK) regime, where the networks' biases are initialized to some constant rather than zero. We prove t…
On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks
Hongru Yang, Zhangyang Wang
Motivated by both theory and practice, we study how random pruning of the weights affects a neural network's neural tangent kernel (NTK). In particular, this work establishes an eq…
Continuous Regular Functions
Alexi Block Gorman, Philipp Hieronymi, Elliot Kaplan +5
Following Chaudhuri, Sankaranarayanan, and Vardi, we say that a function is -regular if there is a Büchi automaton that accepts precisely the set of base $r…