5 papers
Efficient Quantization of Mixture-of-Experts with Theoretical Generalization Guarantees
Mohammed Nowaz Rabbani Chowdhury, Kaoutar El Maghraoui, Hsinyu Tsai +4
Sparse Mixture-of-Experts (MoE) allows scaling of language and vision models efficiently by activating only a small subset of experts per input. While this reduces computation, the…
Visual prompting reimagined: The power of the Activation Prompts
Yihua Zhang, Hongkang Li, Yuguang Yao +5
Visual prompting (VP) has emerged as a popular method to repurpose pretrained vision models for adaptation to downstream tasks. Unlike conventional model fine-tuning techniques, VP…
Robust Heterogeneous Analog-Digital Computing for Mixture-of-Experts Models with Theoretical Generalization Guarantees
Mohammed Nowaz Rabbani Chowdhury, Hsinyu Tsai, Geoffrey W. Burr +3
Sparse Mixture-of-Experts (MoE) models enable efficient scalability by activating only a small sub-set of experts per input, yet their massive parameter counts lead to substantial…
Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
Yunhua Fang, Rui Xie, Asad Ul Haq +6
Large Language Model (LLM) inference is increasingly constrained by memory bandwidth, with frequent access to the key-value (KV) cache dominating data movement. While attention spa…
A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts
Mohammed Nowaz Rabbani Chowdhury, Meng Wang, Kaoutar El Maghraoui +3
The sparsely gated mixture of experts (MoE) architecture sends different inputs to different subnetworks, i.e., experts, through trainable routers. MoE reduces the training computa…