3 papers
cs.AI2026
Inference Time Context Sparsity: Illusion or Opportunity?
Sahil Joshi, Prithvi Dixit, Agniva Chowdhury +5
Sparsity has long been a central theme in LLM efficiency, but its role in context processing remains unresolved. As LLM workloads shift toward longer contexts and agentic interacti…
cs.LG2026
SOCKET: SOft Collision Kernel EsTimator for Sparse Attention
Sahil Joshi, Agniva Chowdhury, Wyatt Bellinger +5
Exploiting sparsity is key to efficient long-context inference, as attention dominates the cost of autoregressive decoding. Sparse attention reduces this cost by restricting comput…
cs.LG2025
RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts
Sahil Joshi, Agniva Chowdhury, Amar Kanakamedala +3
Softmax Attention has a quadratic time complexity in sequence length, which becomes prohibitive to run at long contexts, even with highly optimized GPU kernels. For example, FlashA…