30 citations · 89 across the 21 of their papers we have counts for
Showing 2023 · cs.LGShow all
3 papers · 2 filters
cs.LG2023★ 4 cited
LLM-Assisted Code Cleaning For Training Accurate Code Generators
Naman Jain, Tianjun Zhang, Wei-Lin Chiang +3
Natural language to code generation is an important application area of LLMs and has received wide attention from the community. The majority of relevant studies have exclusively c…
cs.LG2023★ 7 cited
S-LoRA: Serving Thousands of Concurrent LoRA Adapters
Ying Sheng, Shiyi Cao, Dacheng Li +9
The "pretrain-then-finetune" paradigm is commonly adopted in the deployment of large language models. Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method, is often…
cs.LG2023★ 1 cited
DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training
Dacheng Li, Rulin Shao, Anze Xie +5
FlashAttention (Dao, 2023) effectively reduces the quadratic peak memory usage to linear in training transformer-based large language models (LLMs) on a single GPU. In this paper,…