66 citations · 108 across the 9 of their papers we have counts for
4 papers · 1 filter
Praxist: From Experimental Artifacts to Solution Lineages
Jin Li, Ahmed Murtadha, Zhiyu Wang +14
Autonomous R&D agents now write, run, and improve executable artifacts under automated evaluation-but largely as laboratory instruments: shown on curated benchmarks, with gains tha…
Why Attend to Everything? Focus is the Key
Hengshuai Yao, Xing Chen, Ahmed Murtadha +8
Standard attention scales quadratically with sequence length. Efficient attention methods reduce this O(n^2) cost, but when retrofitted into pretrained models, they often degrade p…
GAIN: Multiplicative Modulation for Domain Adaptation
Hengshuai Yao, Xing Chen, Ahmed Murtadha +1
Adapting LLMs to new domains causes forgetting because standard methods (e.g., full fine-tuning, LoRA) inject new directions into the weight space. We show that forgetting is gover…
Thin Keys, Full Values: Reducing KV Cache via Low-Dimensional Attention Selection
Hengshuai Yao, Xing Chen, Ahmed Murtadha +1
Standard Transformer attention uses identical dimensionality for queries, keys, and values, yet these components serve different roles: queries and keys produce scalar attention we…