86 citations · 92 across the 4 of their papers we have counts for
4 papers
Higher-order Linear Attention
Yifan Zhang, Zhen Qin, Mengdi Wang +1
The quadratic cost of scaled dot-product attention is a central obstacle to scaling autoregressive language models to long contexts. Linear-time attention and State Space Models (S…
A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks
Jinghui Chen, Dongruo Zhou, Jinfeng Yi +1
Depending on how much information an adversary can access to, adversarial attacks can be classified as white-box attack and black-box attack. For white-box attack, optimization-bas…
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
Jinghui Chen, Dongruo Zhou, Yiqi Tang +3
Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed…
Statistical Limits of Convex Relaxations
Zhaoran Wang, Quanquan Gu, Han Liu
Many high dimensional sparse learning problems are formulated as nonconvex optimization. A popular approach to solve these nonconvex optimization problems is through convex relaxat…