3 citations · 5 across the 5 of their papers we have counts for
5 papers
MixFormer: Linear Transformer with Mixture of Memory Experts
Yu Guo, Lei Duan
State Space Models (SSMs), as a mainstream research direction of linear Transformers, aim to achieve higher efficiency than standard Transformers in long-context modeling. However,…
Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMs
Shuyuan Lin, Lei Duan, Philip Hughes +1
Conversational Information Retrieval (CIR) systems, while offering intuitive access to information, face a significant challenge: reliably handling unanswerable questions to preven…
DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
Dengchun Li, Naizheng Wang, Zihao Zhang +4
Instruction-based fine-tuning of large language models (LLMs) has achieved remarkable success in various natural language processing (NLP) tasks. Parameter-efficient fine-tuning (P…
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
Dengchun Li, Yingzi Ma, Naizheng Wang +8
Fine-tuning Large Language Models (LLMs) is a common practice to adapt pre-trained models for specific applications. While methods like LoRA have effectively addressed GPU memory c…
mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs
Zhengmao Ye, Dengchun Li, Zetao Hu +8
Transformer-based, pre-trained large language models (LLMs) have demonstrated outstanding performance across diverse domains, particularly in the emerging {\em pretrain-then-finetu…