4 citations · 7 across the 8 of their papers we have counts for
5 papers · 1 filter
Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection
Yun Zhu, Jia-Chen Gu, Caitlin Sikora +8
Large language models (LLMs) augmented with retrieval exhibit robust performance and extensive versatility by incorporating external contexts. However, the input length grows linea…
Inducing Generalization across Languages and Tasks using Featurized Low-Rank Mixtures
Chu-Cheng Lin, Xinyi Wang, Jonathan H. Clark +4
Adapting pretrained large language models (LLMs) to various downstream tasks in tens or hundreds of human languages is computationally expensive. Parameter-efficient fine-tuning (P…
SiRA: Sparse Mixture of Low Rank Adaptation
Yun Zhu, Nevan Wichers, Chu-Cheng Lin +8
Parameter Efficient Tuning has been an prominent approach to adapt the Large Language Model to downstream tasks. Most previous works considers adding the dense trainable parameters…
UT5: Pretraining Non autoregressive T5 with unrolled denoising
Mahmoud G. Salem, Jiayu Ye, Chu-Cheng Lin +1
Recent advances in Transformer-based Large Language Models have made great strides in natural language generation. However, to decode K tokens, an autoregressive model needs K sequ…
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study
Chenxi Whitehouse, Fantine Huot, Jasmijn Bastings +3
Although the advancements of pre-trained Large Language Models have significantly accelerated recent progress in NLP, their ever-increasing size poses significant challenges for co…