1 citations · 3 across the 5 of their papers we have counts for
Showing cs.LGShow all
3 papers · 1 filter
cs.LG2024★ 1 cited
How Sparse Attention Approximates Exact Attention? Your Attention is Naturally -Sparse
Yichuan Deng, Zhao Song, Jing Xiong +1
Sparse Attention is a technique that approximates standard attention computation with sub-quadratic complexity. This is achieved by selectively ignoring smaller entries in the atte…
cs.LG2024
Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
Yichuan Deng, Zhao Song, Chiwun Yang
Based on SGD, previous works have proposed many algorithms that have improved convergence speed and generalization in stochastic optimization, such as SGDm, AdaGrad, Adam, etc. How…
cs.LG2023★ 1 cited
Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights
Yichuan Deng, Zhao Song, Shenghao Xie +1
In the realm of deep learning, transformers have emerged as a dominant architecture, particularly in natural language processing tasks. However, with their widespread adoption, con…