5 papers
Benchmarking Composable Compression Techniques in Mixture-of-Experts LLMs
Afsara Benazir, Chen Chen, Rongxiao Qu +3
Mixture-of-Experts (MoE) LLMs scale model capacity efficiently through sparse activation, but their large expert parameter footprint, routing imbalance, and long-context KV-cache g…
Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs
Afsara Benazir, Felix Xiaozhu Lin
Apple Neural Engine (ANE) is a dedicated neural processing unit (NPU) present in every Apple Silicon chip. Mixture-of-Experts (MoE) LLMs improve inference efficiency via sparse act…
Safeguarding Privacy in Edge Speech Understanding with Tiny Foundation Models
Afsara Benazir, Felix Xiaozhu Lin
Robust speech recognition systems rely on cloud service providers for inference. It needs to ensure that an untrustworthy provider cannot deduce the sensitive content in speech. Sa…
Proto: A Guided Journey through Modern OS Construction
Wonkyo Choe, Rongxiang Wang, Afsara Benazir +1
Proto is a new instructional OS that runs on commodity, portable hardware. It showcases modern features, including per-app address spaces, threading, commodity filesystems, USB, DM…
Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective
Afsara Benazir, Felix Xiaozhu Lin
A systematic understanding of Apple Silicon is lacking in the current landscape of hardware efficiency; research focus is largely centered on accelerating GPUs for large-scale trai…