15 citations · 20 across the 12 of their papers we have counts for
6 papers · 1 filter
A General Theory for Compositional Generalization
Jingwen Fu, Zhizheng Zhang, Yan Lu +1
Compositional Generalization (CG) embodies the ability to comprehend novel combinations of familiar concepts, representing a significant cognitive leap in human intellectual advanc…
Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity
Bohan Wang, Jingwen Fu, Huishuai Zhang +2
Recently, Arjevani et al. [1] established a lower bound of iteration complexity for the first-order optimization under an -smooth condition and a bounded noise variance assumpti…
Generalization error bounds for iterative learning algorithms with bounded updates
Jingwen Fu, Nanning Zheng
This paper explores the generalization characteristics of iterative learning algorithms with bounded updates for non-convex loss functions, employing information-theoretic techniqu…
Breaking through the learning plateaus of in-context learning in Transformer
Jingwen Fu, Tao Yang, Yuwang Wang +2
In-context learning, i.e., learning from context examples, is an impressive ability of Transformer. Training Transformers to possess this in-context learning skill is computational…
When and Why Momentum Accelerates SGD:An Empirical Study
Jingwen Fu, Bohan Wang, Huishuai Zhang +3
Momentum has become a crucial component in deep learning optimizers, necessitating a comprehensive understanding of when and why it accelerates stochastic gradient descent (SGD). T…
Learning Trajectories are Generalization Indicators
Jingwen Fu, Zhizheng Zhang, Dacheng Yin +2
This paper explores the connection between learning trajectories of Deep Neural Networks (DNNs) and their generalization capabilities when optimized using (stochastic) gradient des…