5 papers · 1 filter
LatentBox: Storing AI-Generated Images at Scale via a Latent-First Design
Zirui Wang, Yunjia Zheng, Tingfeng Lan +4
The explosive growth of AI-generated images has created a sustainability challenge for storage infrastructure. Platforms like Midjourney and Adobe Firefly already host billions of…
TStore: Rethinking AI Model Hub with Tensor-Centric Compression
Tingfeng Lan, Zirui Wang, Yunjia Zheng +3
Modern AI models are growing rapidly in size and redundancy, leading to significant storage and distribution challenges in model hubs. We present TStore, a tensor-centric system fo…
MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
Zhaoyuan Su, Zeyu Zhang, Tingfeng Lan +4
Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model…
Clock2Q+: A Simple and Efficient Replacement Algorithm for Metadata Cache in VMware vSAN
Yiyan Zhai, Bintang Dwi Marthen, Sarath Balivada +8
Cache replacement algorithms are critical building blocks of storage systems. This paper examines the characteristics of metadata caches and argues that they inherently exhibit cor…
Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI
Samyam Rajbhandari, Mert Hidayetoglu, Aurick Qiao +5
Inference is now the dominant AI workload, yet existing systems force trade-offs between latency, throughput, and cost. Arctic Inference, an open-source vLLM plugin from Snowflake…