11 papers
S2-MoE: Enabling Efficient Self-Speculative Decoding for Mixture-of-Experts on Edge Devices
Haochen Huang, Shengxuan Qiu, Meng Li
Deploying large language models (LLMs) for inference on edge devices is challenging due to severe memory and bandwidth constraints. While speculative decoding and Mixture-of-Expert…
HyPER: Bridging Exploration and Exploitation for Scalable LLM Reasoning with Hypothesis Path Expansion and Reduction
Shengxuan Qiu, Haochen Huang, Shuzhang Zhong +2
Scaling test-time compute with multi-path chain-of-thought improves reasoning accuracy, but its effectiveness depends critically on the exploration-exploitation trade-off. Existing…
EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrieval
Zebin Yang, Sunjian Zheng, Tong Xie +6
Object-goal navigation (ObjNav) tasks an agent with navigating to the location of a specific object in an unseen environment. Embodied agents equipped with large language models (L…
Cachemir: Fully Homomorphic Encrypted Inference of Generative Large Language Model with KV Cache
Ye Yu, Yifan Zhou, Yi Chen +3
Generative large language models (LLMs) have revolutionized multiple domains. Modern LLMs predominantly rely on an autoregressive decoding strategy, which generates output tokens s…
IntraSlice: Towards High-Performance Structural Pruning with Block-Intra PCA for LLMs
Meng Li, Peisong Wang, Yuantian Shao +5
Large Language Models (LLMs) achieve strong performance across diverse tasks but face deployment challenges due to their massive size. Structured pruning offers acceleration benefi…
What Gets Activated: Uncovering Domain and Driver Experts in MoE Language Models
Guimin Hu, Meng Li, Qiwei Peng +3
Most interpretability work focuses on layer- or neuron-level mechanisms in Transformers, leaving expert-level behavior in MoE LLMs underexplored. Motivated by functional specializa…