5 papers
Enabling Dynamic Sparsity in Quantized LLM Inference
Rongxiang Wang, Kangyuan Shu, Felix Xiaozhu Lin
Deploying large language models (LLMs) on end-user devices is gaining importance due to benefits in responsiveness, privacy, and operational cost. Yet the limited memory and comput…
Proto: A Guided Journey through Modern OS Construction
Wonkyo Choe, Rongxiang Wang, Afsara Benazir +1
Proto is a new instructional OS that runs on commodity, portable hardware. It showcases modern features, including per-app address spaces, threading, commodity filesystems, USB, DM…
WhisperFlow: speech foundation models in real time
Rongxiang Wang, Zhiming Xu, Felix Xiaozhu Lin
Speech foundation models, such as OpenAI's Whisper, become the state of the art in speech understanding due to their strong accuracy and generalizability. Yet, their applications a…
Profiling Apple Silicon Performance for ML Training
Dahua Feng, Zhiming Xu, Rongxiang Wang +1
Apple Silicon has attracted much attention for its performance and role in machine learning (ML) training. Unlike NVIDIA GPUs, which have traditionally dominated ML training, Apple…
Turbocharge Speech Understanding with Pilot Inference
Rongxiang Wang, Felix Xiaozhu Lin
Modern speech understanding (SU) runs a sophisticated pipeline: ingesting streaming voice input, the pipeline executes encoder-decoder based deep neural networks repeatedly; by doi…