11 citations · 17 across the 7 of their papers we have counts for
4 papers · 1 filter
TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching
Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu +4
Large language models (LLMs) are moving onto mobile devices for increasingly diverse workloads over text, images, video, and audio. These applications often require long contexts,…
From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs
Kaiyuan Deng, Hangyu Zheng, Minghai Qing +11
Deploying models, especially large language models (LLMs), is becoming increasingly attractive to a broader user base, including those without specialized expertise. However, due t…
SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile
Wei Niu, Md Musfiqur Rahman Sanim, Zhihao Shu +5
This work is motivated by recent developments in Deep Neural Networks, particularly the Transformer architectures underlying applications such as ChatGPT, and the need for performi…
SoD: Statically Optimizing Dynamic Deep Neural Network
Wei Niu, Gagan Agrawal, Bin Ren
Though many compilation and runtime systems have been developed for DNNs in recent years, the focus has largely been on static DNNs. Dynamic DNNs, where tensor shapes and sizes and…