4 papers
Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4
Sparse Mixture-of-Experts (MoE) allows scaling of language and vision models efficiently by activating only a small subset of experts per input. While this reduces computation, the…
Robust Heterogeneous Analog-Digital Computing for Mixture-of-Experts Models with Theoretical Generalization Guarantees
Mohammed Nowaz Rabbani Chowdhury, Hsinyu Tsai, Geoffrey W. Burr +3
Sparse Mixture-of-Experts (MoE) models enable efficient scalability by activating only a small sub-set of experts per input, yet their massive parameter counts lead to substantial…
Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
Zehao Fan, Zhenyu Liu, Yunzhen Liu +4
Mixture-of-Experts (MoE) models scale large language models through conditional computation, but inference becomes memory-bound once expert weights exceed the capacity of GPU memor…
SparseST: Exploiting Data Sparsity in Spatiotemporal Modeling and Prediction
Junfeng Wu, Hadjer Benmeziane, Kaoutar El Maghraoui +2
Spatiotemporal data mining (STDM) has a wide range of applications in various complex physical systems (CPS), i.e., transportation, manufacturing, healthcare, etc. Among all the pr…