Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection
Anusha Madan Gopal, Aras Pirbadian, Kristofor D. Carlson +2
Retrieval-augmented generation (RAG) imposes a prefill cost proportional to retrieved context length, and -- with Transformer backbones -- a KV-cache that grows with each generated…
cs.LG2026
Do Transformers Need Three Projections? Systematic Study of QKV Variants
Ali Kayyam, Anusha Madan Gopal, M Anthony Lewis
Transformers have become the standard solution for various AI tasks, with the query, key, and value (QKV) attention formulation playing a central role. However, the individual cont…