4 papers
BIDENT: Heterogeneous Operator-level Mapping for Efficient Edge Inference
Hoseok Kim, Arghadip Das, Soumendu Ghosh +2
Modern edge System-on-Chips (SoCs) integrate heterogeneous processing units (PUs) such as CPUs, GPUs, and NPUs, yet current inference stacks map entire models to a single PU, leavi…
SPARQLe: Sub-Precision Activation Representation for Quantized LLM Inference
Aradhana Mohan Parvathy, Soumendu Kumar Ghosh, Shamik Kundu +4
The rapid growth in sizes of Large language models (LLMs) results in high compute and memory costs during inference. Quantization has been a significant pathway to addressing this…
XAMBA: Enabling Efficient State Space Models on Resource-Constrained Neural Processing Units
Arghadip Das, Arnab Raha, Shamik Kundu +3
State-Space Models (SSMs) have emerged as efficient alternatives to transformers for sequential data tasks, offering linear or near-linear scalability with sequence length, making…
GraNNite: Enabling High-Performance Execution of Graph Neural Networks on Resource-Constrained Neural Processing Units
Arghadip Das, Shamik Kundu, Arnab Raha +3
Graph Neural Networks (GNNs) are vital for learning from graph-structured data, enabling applications in network analysis, recommendation systems, and speech analytics. Deploying t…