6 citations · 13 across the 15 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.CL2024
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
Yansheng Mao, Jiaqi Li, Fanxu Meng +3
Long context understanding remains challenging for large language models due to their limited context windows. This paper introduces Long Input Fine-Tuning (LIFT) for long context…
cs.LG2024
CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuning
Fanxu Meng, Pingzhi Tang, Fan jiang +1
Decoder-only models generate tokens autoregressively by caching key/value vectors, but as the cache grows, inference becomes memory-bound. To address this issue, we introduce CLOVE…
cs.LG2024★ 6 cited
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
Fanxu Meng, Zhaohui Wang, Muhan Zhang
To parameter-efficiently fine-tune (PEFT) large language models (LLMs), the low-rank adaptation (LoRA) method approximates the model changes throug…