3 citations · 7 across the 9 of their papers we have counts for
4 papers · 1 filter
Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences
Yifan Chen, Qi Zeng, Dilek Hakkani-Tur +3
Transformer-based models are not efficient in processing long sequences due to the quadratic space and time complexity of the self-attention modules. To address this limitation, Li…
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method
Yifan Chen, Qi Zeng, Heng Ji +1
Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism. On the other hand, although kernel machines suffer from the same…
Fast Statistical Leverage Score Approximation in Kernel Ridge Regression
Yifan Chen, Yun Yang
Nyström approximation is a fast randomized method that rapidly solves kernel ridge regression (KRR) problems through sub-sampling the n-by-n empirical kernel matrix appearing in th…
Accumulation of Sub-Sampling Matrices with Applications to Statistical Computation
Yifan Chen, Yun Yang
With appropriately chosen sampling probabilities, sampling-based random projection can be used to implement large-scale statistical methods, substantially reducing computational co…