66 citations · 108 across the 9 of their papers we have counts for
3 papers · 1 filter
GAIN: Multiplicative Modulation for Domain Adaptation
Hengshuai Yao, Xing Chen, Ahmed Murtadha +1
Adapting LLMs to new domains causes forgetting because standard methods (e.g., full fine-tuning, LoRA) inject new directions into the weight space. We show that forgetting is gover…
Thin Keys, Full Values: Reducing KV Cache via Low-Dimensional Attention Selection
Hengshuai Yao, Xing Chen, Ahmed Murtadha +1
Standard Transformer attention uses identical dimensionality for queries, keys, and values, yet these components serve different roles: queries and keys produce scalar attention we…
ZLPR: A Novel Loss for Multi-label Classification
Jianlin Su, Mingren Zhu, Ahmed Murtadha +3
In the era of deep learning, loss functions determine the range of tasks available to models and algorithms. To support the application of deep learning in multi-label classificati…