3 citations · 5 across the 6 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.AR2026
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Vishesh Tripathi, Abhay Kumar, Ramsha Khan
The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with sequence length. Grouped-query attention (GQA) reduces this cos…
cs.LG2026
Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention
Vishesh Tripathi, Abhay Kumar
Self-attention is central to Transformer performance and is often the most expensive part of the Transformer at long context lengths because its pairwise token interactions scale q…