4 papers
Geak: Introducing Triton Kernel AI Agent & Evaluation Benchmarks
Jianghui Wang, Vinay Joshi, Saptarshi Majumder +7
The demand for AI-generated GPU kernels is rapidly growing, influenced by the need for scalable, hardware-optimized solutions in both industry and academia. As deep learning worklo…
TaDA: Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centering
Vinay Joshi, Pratik Prabhanjan Brahma, Zicheng Liu +1
The key-value (KV) cache in transformer models is a critical component for efficient decoding or inference, yet its memory demands scale poorly with sequence length, posing a major…
QCQA: Quality and Capacity-aware grouped Query Attention
Vinay Joshi, Prashant Laddha, Shambhavi Sinha +2
Excessive memory requirements of key and value features (KV-cache) present significant challenges in the autoregressive inference of large language models (LLMs), restricting both…
CiMNet: Towards Joint Optimization for DNN Architecture and Configuration for Compute-In-Memory Hardware
Souvik Kundu, Anthony Sarah, Vinay Joshi +2
With the recent growth in demand for large-scale deep neural networks, compute in-memory (CiM) has come up as a prominent solution to alleviate bandwidth and on-chip interconnect b…