9 papers
HYDRA: A Heterogeneous Chiplet DSE Framework for Serving Dynamic Hybrid LLM Workloads
Jiahao Lin, Alish Kanani, Sangwan Lee +2
Hybrid Transformer-Mamba large language models (LLMs) enhance long-context efficiency, but their heterogeneous computation and communication patterns complicate efficient hardware…
APEX: Adaptive Expert Prefetching for Memory-Efficient Edge MoE Inference
Alish Kanani, Layan Badawi, Umit Y. Ogras
Mixture-of-Experts (MoE) models are attractive for edge deployment because they provide high model capacity while activating only a small subset of parameters per token, improving…
ReVolt: Power Delivery Network-Aware Voltage Droop Control for 2.5D PIM Chiplet Architectures
Vibhanshu Sharma, Alish Kanani, Miao Sun +3
Processing-in-memory (PIM)-based 2.5D multi-chiplet platforms are enablers for machine learning (ML) workloads. However, their performance is affected by the power delivery network…
LEXI: Lossless Exponent Coding for Efficient Inter-Chiplet Communication in Hybrid LLMs
Miao Sun, Alish Kanani, Kaushik Shroff +1
Data movement overheads increase the inference latency of state-of-the-art large language models (LLMs). These models commonly use the bfloat16 (BF16) format for stable training. F…
DUET: Disaggregated Hybrid Mamba-Transformer LLMs with Prefill and Decode-Specific Packages
Alish Kanani, Sangwan Lee, Han Lyu +3
Large language models operate in distinct compute-bound prefill followed by memory bandwidth-bound decode phases. Hybrid Mamba-Transformer models inherit this asymmetry while addin…
CHIPSIM: A Co-Simulation Framework for Deep Learning on Chiplet-Based Systems
Lukas Pfromm, Alish Kanani, Harsh Sharma +3
Due to reduced manufacturing yields, traditional monolithic chips cannot keep up with the compute, memory, and communication demands of data-intensive applications, such as rapidly…