11 citations · 17 across the 7 of their papers we have counts for
9 papers
EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models
Kunxiong Zhu, Zhihao Shu, Hangyu Zheng +4
Disaggregating the two stages, Prefill and Decode, onto separate GPU pools is now a standard optimization for (text-only) LLM serving. However, multimodal LLMs (MLLMs), which add a…
Mask-Aware Execution for Efficient JEPA Training
Md Musfiqur Rahman Sanim, Zhihao Shu, Bahram Afsharmanesh +3
Joint Embedding Predictive Architectures (JEPAs) are becoming a core representation-learning primitive and a building block for latent world models across vision, video, audio, bra…
TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching
Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu +4
Large language models (LLMs) are moving onto mobile devices for increasingly diverse workloads over text, images, video, and audio. These applications often require long contexts,…
Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training
Yangming Zhang, Jian Xu, Chaojian Li +7
3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis with high-quality rendering through continuous aggregations of millions of 3D Gaussian primitives. However, it…
FlashMem: Supporting Modern DNN Workloads on Mobile with GPU Memory Hierarchy Optimizations
Zhihao Shu, Md Musfiqur Rahman Sanim, Hangyu Zheng +4
The increasing size and complexity of modern deep neural networks (DNNs) pose significant challenges for on-device inference on mobile GPUs, with limited memory and computational r…
From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs
Kaiyuan Deng, Hangyu Zheng, Minghai Qing +11
Deploying models, especially large language models (LLMs), is becoming increasingly attractive to a broader user base, including those without specialized expertise. However, due t…
Optimizing 3D Gaussian Splattering for Mobile GPUs
Md Musfiqur Rahman Sanim, Zhihao Shu, Bahram Afsharmanesh +5
Image-based 3D scene reconstruction, which transforms multi-view images into a structured 3D representation of the surrounding environment, is a common task across many modern appl…
SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile
Wei Niu, Md Musfiqur Rahman Sanim, Zhihao Shu +5
This work is motivated by recent developments in Deep Neural Networks, particularly the Transformer architectures underlying applications such as ChatGPT, and the need for performi…
SoD: Statically Optimizing Dynamic Deep Neural Network
Wei Niu, Gagan Agrawal, Bin Ren
Though many compilation and runtime systems have been developed for DNNs in recent years, the focus has largely been on static DNNs. Dynamic DNNs, where tensor shapes and sizes and…