1 citations · 2 across the 7 of their papers we have counts for
8 papers
ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression
Yuechuan Chen, Yanda Guo, Xiaoxi Liu +4
Some observation-window-based KV cache compressors use prompt attention to identify important tokens but allocate uniform or predefined budgets across heads, overlooking input-depe…
Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
Chengqian Zhang, Wei Zhu, Kyumin Lee
Post-training has become essential for adapting large language models (LLMs) to complex downstream behaviors, including instruction following, preference alignment, and multi-step…
ShapLoRA: Allocation of Low-rank Adaption on Large Language Models via Shapley Value Inspired Importance Estimation
Yi Zhao, Qinghua Yao, Xinyuan song +1
Low-rank adaption (LoRA) is a representative method in the field of parameter-efficient fine-tuning (PEFT), and is key to Democratizating the modern large language models (LLMs). T…
ML-Triton, A Multi-Level Compilation and Language Extension to Triton GPU Programming
Dewei Wang, Wei Zhu, Liyang Ling +5
In the era of LLMs, dense operations such as GEMM and MHA are critical components. These operations are well-suited for parallel execution using a tilebased approach. While traditi…
PARA: Parameter-Efficient Fine-tuning with Prompt Aware Representation Adjustment
Zequan Liu, Yi Zhao, Ming Tan +2
In the realm of parameter-efficient fine-tuning (PEFT) methods, while options like LoRA are available, there is a persistent demand in the industry for a PEFT approach that excels…
MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning
Jingfan Zhang, Yi Zhao, Dan Chen +3
Low-rank adaptation (LoRA) and its mixture-of-experts (MOE) variants are highly effective parameter-efficient fine-tuning (PEFT) methods. However, they introduce significant latenc…
PEDRO: Parameter-Efficient Fine-tuning with Prompt DEpenDent Representation MOdification
Tianfang Xie, Tianjing Li, Wei Zhu +2
Due to their substantial sizes, large language models (LLMs) are typically deployed within a single-backbone multi-tenant framework. In this setup, a single instance of an LLM back…
Efficient LLM inference solution on Intel GPU
Hui Wu, Yi Gan, Feng Yuan +8
Transformer based Large Language Models (LLMs) have been widely used in many fields, and the efficiency of LLM inference becomes hot topic in real applications. However, LLMs are u…