most citedMoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts

1 citations · 1 across the 3 of their papers we have counts for

collaborators
Showing cs.ARShow all

2 papers · 1 filter

cs.AR2025

From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing

Yushu Zhao, Yubin Qin, Yang Wang +5

Large language models achieve impressive performance across diverse tasks but exhibit high inference latency due to their large parameter sizes. While quantization reduces model si…

cs.AR2025

MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness

Huizheng Wang, Zichuan Wang, Zhiheng Yue +8

Large language models (LLMs) face significant inference latency due to inefficiencies in GEMM operations, weight access, and KV cache access, especially in real-time scenarios. Thi…