50 citations · 302 across the 44 of their papers we have counts for
7 papers · 1 filter
Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences
Yifan Chen, Qi Zeng, Dilek Hakkani-Tur +3
Transformer-based models are not efficient in processing long sequences due to the quadratic space and time complexity of the self-attention modules. To address this limitation, Li…
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method
Yifan Chen, Qi Zeng, Heng Ji +1
Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism. On the other hand, although kernel machines suffer from the same…
Learning Topic Models: Identifiability and Finite-Sample Analysis
Yinyin Chen, Shishuang He, Yun Yang +1
Topic models provide a useful text-mining tool for learning, extracting, and discovering latent structures in large text corpora. Although a plethora of methods have been proposed…
Regret Lower Bound and Optimal Algorithm for High-Dimensional Contextual Linear Bandit
Ke Li, Yun Yang, Naveen N. Narisetty
In this paper, we consider the multi-armed bandit problem with high-dimensional features. First, we prove a minimax lower bound, $\mathcal{O}\big((\log d)^{\frac{α+1}{2}}T^{\frac{1…
Statistical Inference for Bayesian Risk Minimization via Exponentially Tilted Empirical Likelihood
Rong Tang, Yun Yang
The celebrated Bernstein von-Mises theorem ensures that credible regions from Bayesian posterior are well-calibrated when the model is correctly-specified, in the frequentist sense…
Fast Statistical Leverage Score Approximation in Kernel Ridge Regression
Yifan Chen, Yun Yang
Nyström approximation is a fast randomized method that rapidly solves kernel ridge regression (KRR) problems through sub-sampling the n-by-n empirical kernel matrix appearing in th…