36 citations · 84 across the 17 of their papers we have counts for
3 papers · 1 filter
LLM-Aided Compilation for Tensor Accelerators
Charles Hong, Sahil Bhatia, Altan Haan +4
Hardware accelerators, in particular accelerators for tensor processing, have many potential application domains. However, they currently lack the software infrastructure to suppor…
Virgo: Cluster-level Matrix Unit Integration in GPUs for Scalability and Energy Efficiency
Hansung Kim, Ruohan Richard Yan, Joshua You +2
Modern GPUs incorporate specialized matrix units such as Tensor Cores to accelerate GEMM operations, which are central to deep learning workloads. However, existing matrix unit des…
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh +4
LLMs are seeing growing use for applications which require large context windows, and with these large context windows KV cache activations surface as the dominant contributor to m…