11 citations · 15 across the 8 of their papers we have counts for
11 papers
BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs
Hangyu Zheng, Zhihao Shu, Kunxiong Zhu +4
The rapid scaling of large language models (LLMs) has unlocked remarkable capabilities across various domains. However, deploying LLMs on mobile devices remains challenging because…
EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models
Kunxiong Zhu, Zhihao Shu, Hangyu Zheng +4
Disaggregating the two stages, Prefill and Decode, onto separate GPU pools is now a standard optimization for (text-only) LLM serving. However, multimodal LLMs (MLLMs), which add a…
Mask-Aware Execution for Efficient JEPA Training
Md Musfiqur Rahman Sanim, Zhihao Shu, Bahram Afsharmanesh +3
Joint Embedding Predictive Architectures (JEPAs) are becoming a core representation-learning primitive and a building block for latent world models across vision, video, audio, bra…
TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching
Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu +4
Large language models (LLMs) are moving onto mobile devices for increasingly diverse workloads over text, images, video, and audio. These applications often require long contexts,…
GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents
Zhe Ren, Yibo Yang, Yimeng Chen +7
Memory benchmarks for LLM agents largely assume single-user settings, leaving shared assistants for hospitals, workplaces, campuses, and households understudied. In these deploymen…
FlashMem: Supporting Modern DNN Workloads on Mobile with GPU Memory Hierarchy Optimizations
Zhihao Shu, Md Musfiqur Rahman Sanim, Hangyu Zheng +4
The increasing size and complexity of modern deep neural networks (DNNs) pose significant challenges for on-device inference on mobile GPUs, with limited memory and computational r…
Optimizing 3D Gaussian Splattering for Mobile GPUs
Md Musfiqur Rahman Sanim, Zhihao Shu, Bahram Afsharmanesh +5
Image-based 3D scene reconstruction, which transforms multi-view images into a structured 3D representation of the surrounding environment, is a common task across many modern appl…
LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers
Xuan Shen, Zhao Song, Yufa Zhou +12
Diffusion Transformers have emerged as the preeminent models for a wide array of generative tasks, demonstrating superior performance and efficacy across various applications. The…
Data Overfitting for On-Device Super-Resolution with Dynamic Algorithm and Compiler Co-Design
Gen Li, Zhihao Shu, Jie Ji +4
Deep neural networks (DNNs) are frequently employed in a variety of computer vision applications. Nowadays, an emerging trend in the current video distribution system is to take ad…
SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile
Wei Niu, Md Musfiqur Rahman Sanim, Zhihao Shu +5
This work is motivated by recent developments in Deep Neural Networks, particularly the Transformer architectures underlying applications such as ChatGPT, and the need for performi…
Squat: Quant Small Language Models on the Edge
Xuan Shen, Peiyan Dong, Zhenglun Kong +9
A growing trend has emerged in designing high-quality Small Language Models (SLMs) with a few million parameters. This trend is driven by the increasing concerns over cloud costs,…