1 citations · 2 across the 5 of their papers we have counts for
5 papers
Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs
Afsara Benazir, Felix Xiaozhu Lin
Apple Neural Engine (ANE) is a dedicated neural processing unit (NPU) present in every Apple Silicon chip. Mixture-of-Experts (MoE) LLMs improve inference efficiency via sparse act…
Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective
Afsara Benazir, Felix Xiaozhu Lin
A systematic understanding of Apple Silicon is lacking in the current landscape of hardware efficiency; research focus is largely centered on accelerating GPUs for large-scale trai…
Safeguarding Privacy in Edge Speech Understanding with Tiny Foundation Models
Afsara Benazir, Felix Xiaozhu Lin
Robust speech recognition systems rely on cloud service providers for inference. It needs to ensure that an untrustworthy provider cannot deduce the sensitive content in speech. Sa…
Speech Understanding on Tiny Devices with A Learning Cache
Afsara Benazir, Zhiming Xu, Felix Xiaozhu Lin
This paper addresses spoken language understanding (SLU) on microcontroller-like embedded devices, integrating on-device execution with cloud offloading in a novel fashion. We leve…
SelectFormer: Private and Practical Data Selection for Transformers
Xu Ouyang, Felix Xiaozhu Lin, Yangfeng Ji
Critical to a free data market is , i.e. the model owner selects and then appraises training data from the data owner before both parties commit to…