8 papers
To Store or To Regenerate? A Cost Model for AI-Generated Content at Scale
Yunjia Zheng, Zirui Wang, Haoran Ni +4
AI-generated content is becoming a rapidly growing class of digital artifacts. Because these artifacts accumulate over time, their exponential growth creates a substantial storage,…
LatentStore: An Efficient Latent-First Storage System for AI-Generated Images
Zirui Wang, Yunjia Zheng, Tingfeng Lan +4
The explosive growth of AI-generated images has created a sustainability challenge for storage infrastructure. Platforms like Midjourney and Adobe Firefly already host billions of…
TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models
Tingfeng Lan, Zirui Wang, Yunjia Zheng +3
Modern model hubs store hundreds of petabytes of large language models (LLMs), with fine-tuned variants dominating the storage footprint. These variants contain substantial cross-m…
ViviDoc: Generating Interactive Documents through Human-Agent Collaboration
Yinghao Tang, Yupeng Xie, Yingchaojie Feng +4
Interactive documents help readers engage with complex ideas through dynamic visualization, interactive animations, and exploratory interfaces. However, creating such documents rem…
MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
Zhaoyuan Su, Zeyu Zhang, Tingfeng Lan +4
Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model…
ZenFlow: Enabling Stall-Free Offloading for LLM Training
Tingfeng Lan, Yusen Wu, Bin Ma +7
Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks l…
ZipLLM: Efficient LLM Storage via Model-Aware Synergistic Data Deduplication and Compression
Zirui Wang, Tingfeng Lan, Zhaoyuan Su +2
Modern model hubs, such as Hugging Face, store tens of petabytes of LLMs, with fine-tuned variants vastly outnumbering base models and dominating storage consumption. Existing stor…
λScale: Enabling Fast Scaling for Serverless Large Language Model Inference
Minchen Yu, Rui Yang, Chaobo Jia +9
Serverless computing has emerged as a compelling solution for cloud-based model inference. However, as modern large language models (LLMs) continue to grow in size, existing server…