3 citations · 5 across the 5 of their papers we have counts for
6 papers
Accelerating Cross-Model LLM Inference via Attention Map Reuse
Zhan Jin, Yuze Luo, Bin Cui +1
LLM-augmented data workflows often involve multiple heterogeneous models operating over substantial shared context, such as schemas, tool specifications, interaction history, and i…
LobRA: Multi-tenant Fine-tuning over Heterogeneous Data
Sheng Lin, Fangcheng Fu, Haoyang Li +5
With the breakthrough of Transformer-based pre-trained models, the demand for fine-tuning (FT) to adapt the base pre-trained models to downstream applications continues to grow, so…
Theseus: Exploring Efficient Wafer-Scale Chip Design for Large Language Models
Jingchen Zhu, Chenhao Xue, Yiqi Chen +13
The emergence of the large language model~(LLM) poses an exponential growth of demand for computation throughput, memory capacity, and communication bandwidth. Such a demand growth…
CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models
Hailin Zhang, Zirui Liu, Boxuan Chen +4
Recently, the growing memory demands of embedding tables in Deep Learning Recommendation Models (DLRMs) pose great challenges for model training and deployment. Existing embedding…
SpotServe: Serving Generative Large Language Models on Preemptible Instances
Xupeng Miao, Chunan Shi, Jiangfei Duan +4
The high computational and memory requirements of generative large language models (LLMs) make it challenging to serve them cheaply. This paper aims to reduce the monetary cost for…
Experimental Analysis of Large-scale Learnable Vector Storage Compression
Hailin Zhang, Penghao Zhao, Xupeng Miao +4
Learnable embedding vector is one of the most important applications in machine learning, and is widely used in various database-related domains. However, the high dimensionality o…