5 citations · 16 across the 10 of their papers we have counts for
10 papers
How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee +9
Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-level objectives: from chunked-pre…
Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
Abhimanyu Bambhaniya, Geonhwa Jeong, Jason Park +6
Most recent state-of-the-art (SOTA) large language models (LLMs) use Mixture-of-Experts (MoE) architectures to scale model capacity without proportional per-token compute, enabling…
MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inference
Abhimanyu Rajeshkumar Bambhaniya, Hanjiang Wu, Suvinay Subramanian +8
Modern LLM serving now spans multi-stage pipelines including RAG retrieval and KV cache reuse, each with distinct compute, memory, and latency demands. Inference engines expose a l…
Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
Abhimanyu Bambhaniya, Ritik Raj, Geonhwa Jeong +6
Large language models (LLMs) have shown remarkable performance across a wide range of applications, often outperforming human experts. However, deploying these gigantic models effi…
Enabling Unstructured Sparse Acceleration on Structured Sparse Accelerators
Geonhwa Jeong, Po-An Tsai, Abhimanyu R. Bambhaniya +2
Exploiting sparsity in deep neural networks (DNNs) has been a promising area for meeting the growing computation requirements. To minimize the overhead of sparse acceleration, hard…
Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers
Abhimanyu Rajeshkumar Bambhaniya, Amir Yazdanbakhsh, Suvinay Subramanian +4
N:M Structured sparsity has garnered significant interest as a result of relatively modest overhead and improved efficiency. Additionally, this form of sparsity holds considerable…
Subgraph Stationary Hardware-Software Inference Co-Design
Payman Behnam, Jianming Tong, Alind Khare +6
A growing number of applications depend on Machine Learning (ML) functionality and benefits from both higher quality ML predictions and better timeliness (latency) at the same time…
VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs
Geonhwa Jeong, Sana Damani, Abhimanyu Rajeshkumar Bambhaniya +5
Deep Learning (DL) acceleration support in CPUs has recently gained a lot of traction, with several companies (Arm, Intel, IBM) announcing products with specialized matrix engines…
COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training
Divya Kiran Kadiyala, Saeed Rashidi, Taekyung Heo +3
Modern Deep Learning (DL) models have grown to sizes requiring massive clusters of specialized, high-end nodes to train. Designing such clusters to maximize both performance and ut…
Enabling Flexibility for Sparse Tensor Acceleration via Heterogeneity
Eric Qin, Raveesh Garg, Abhimanyu Bambhaniya +5
Recently, numerous sparse hardware accelerators for Deep Neural Networks (DNNs), Graph Neural Networks (GNNs), and scientific computing applications have been proposed. A common ch…