9 papers
LatentBox: Storing AI-Generated Images at Scale via a Latent-First Design
Zirui Wang, Yunjia Zheng, Tingfeng Lan +4
The explosive growth of AI-generated images has created a sustainability challenge for storage infrastructure. Platforms like Midjourney and Adobe Firefly already host billions of…
MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
Zhaoyuan Su, Olatunji Ruwase, Karthik Ganesan +5
Production LLM workloads increasingly serve discriminative tasks, such as classification, recommendation, and verification, whose answers are read from the logits of a single prefi…
TStore: Rethinking AI Model Hub with Tensor-Centric Compression
Tingfeng Lan, Zirui Wang, Yunjia Zheng +3
Modern AI models are growing rapidly in size and redundancy, leading to significant storage and distribution challenges in model hubs. We present TStore, a tensor-centric system fo…
λScale: Enabling Fast Scaling for Serverless Large Language Model Inference
Minchen Yu, Rui Yang, Chaobo Jia +9
Serverless computing has emerged as a compelling solution for cloud-based model inference. However, as modern large language models (LLMs) continue to grow in size, existing server…
TopoSZp: Lightweight Topology-Aware Error-controlled Compression for Scientific Data
Tripti Agarwal, Sheng Di, Xin Liang +5
Error-bounded lossy compression is essential for managing the massive data volumes produced by large-scale HPC simulations. While state-of-the-art compressors such as SZ and ZFP pr…
MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
Zhaoyuan Su, Zeyu Zhang, Tingfeng Lan +4
Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model…