3 papers
cs.AR2026
HYDRA: A Heterogeneous Chiplet DSE Framework for Serving Dynamic Hybrid LLM Workloads
Jiahao Lin, Alish Kanani, Sangwan Lee +2
Hybrid Transformer-Mamba large language models (LLMs) enhance long-context efficiency, but their heterogeneous computation and communication patterns complicate efficient hardware…
cs.AR2026
DUET: Disaggregated Hybrid Mamba-Transformer LLMs with Prefill and Decode-Specific Packages
Alish Kanani, Sangwan Lee, Han Lyu +3
Large language models operate in distinct compute-bound prefill followed by memory bandwidth-bound decode phases. Hybrid Mamba-Transformer models inherit this asymmetry while addin…
cs.DC2025
MatKV: Trading Compute for Flash Storage in LLM Inference
Kun-Woo Shin, Jay H. Park, Moonwook Oh +3
We observe two major trends in LLM-based generative AI: (1) inference is becoming the dominant factor in terms of cost and power consumption, surpassing training, and (2) retrieval…