7 citations · 9 across the 4 of their papers we have counts for
4 papers
Optimised Grouped-Query Attention Mechanism for Transformers
Yuang Chen, Cheng Zhang, Xitong Gao +3
Grouped-query attention (GQA) has been widely adopted in LLMs to mitigate the complexity of multi-head attention (MHA). To transform an MHA to a GQA, neighbour queries in MHA are e…
Unlocking the Global Synergies in Low-Rank Adapters
Zixi Zhang, Cheng Zhang, Xitong Gao +3
Low-rank Adaption (LoRA) has been the de-facto parameter-efficient fine-tuning technique for large language models. We present HeteroLoRA, a light-weight search algorithm that leve…
APBench: A Unified Benchmark for Availability Poisoning Attacks and Defenses
Tianrui Qin, Xitong Gao, Juanjuan Zhao +2
The efficacy of availability poisoning, a method of poisoning data by injecting imperceptible perturbations to prevent its use in model training, has been a hot subject of investig…
Learning the Unlearnable: Adversarial Augmentations Suppress Unlearnable Example Attacks
Tianrui Qin, Xitong Gao, Juanjuan Zhao +2
Unlearnable example attacks are data poisoning techniques that can be used to safeguard public data against unauthorized use for training deep learning models. These methods add st…