3 citations · 7 across the 9 of their papers we have counts for
9 papers
MLP Fusion: Towards Efficient Fine-tuning of Dense and Mixture-of-Experts Language Models
Mengting Ai, Tianxin Wei, Yifan Chen +2
Fine-tuning a pre-trained language model (PLM) emerges as the predominant strategy in many natural language processing applications. However, this process is known to be expensive,…
A Gromov--Wasserstein Geometric View of Spectrum-Preserving Graph Coarsening
Yifan Chen, Rentian Yao, Yun Yang +1
Graph coarsening is a technique for solving large-scale graph problems by working on a smaller version of the original graph, and possibly interpolating the results back to the ori…
Inducer-tuning: Connecting Prefix-tuning and Adapter-tuning
Yifan Chen, Devamanyu Hazarika, Mahdi Namazifar +3
Prefix-tuning, or more generally continuous prompt tuning, has become an essential paradigm of parameter-efficient transfer learning. Using a large pre-trained language model (PLM)…
Calibrate and Debias Layer-wise Sampling for Graph Convolutional Networks
Yifan Chen, Tianning Xu, Dilek Hakkani-Tur +3
Multiple sampling-based methods have been developed for approximating and accelerating node embedding aggregation in graph convolutional networks (GCNs) training. Among them, a lay…
Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention
Yifan Chen, Devamanyu Hazarika, Mahdi Namazifar +3
The massive amount of trainable parameters in the pre-trained language models (PLMs) makes them hard to be deployed to multiple downstream tasks. To address this issue, parameter-e…
Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences
Yifan Chen, Qi Zeng, Dilek Hakkani-Tur +3
Transformer-based models are not efficient in processing long sequences due to the quadratic space and time complexity of the self-attention modules. To address this limitation, Li…
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method
Yifan Chen, Qi Zeng, Heng Ji +1
Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism. On the other hand, although kernel machines suffer from the same…
Fast Statistical Leverage Score Approximation in Kernel Ridge Regression
Yifan Chen, Yun Yang
Nyström approximation is a fast randomized method that rapidly solves kernel ridge regression (KRR) problems through sub-sampling the n-by-n empirical kernel matrix appearing in th…
Accumulation of Sub-Sampling Matrices with Applications to Statistical Computation
Yifan Chen, Yun Yang
With appropriately chosen sampling probabilities, sampling-based random projection can be used to implement large-scale statistical methods, substantially reducing computational co…