6 papers
Beyond Prediction: Tail-Aware Scheduling for LLM Inference
Yueying Li, Yuanfan Chen, Jiayang Chen +6
LLM serving exhibits extreme length variability, making size-based scheduling difficult in practice. Recent LLM schedulers approximate SJF/SRPT using predicted decode lengths or ra…
VIPIR: A Versatile GPU Framework for Integrating Private Information Retrieval Protocols
Jongmin Kim, Hyesung Ji, Jean-Luc Watson +3
While private information retrieval (PIR) enables private database services by fully concealing access patterns, it simultaneously requires high computational throughput, large mem…
GPIR: Enabling Practical Private Information Retrieval with GPUs
Hyesung Ji, Hyunah Yu, Jongmin Kim +3
Private information retrieval (PIR) allows private database queries; however, it is hindered by intense server-side computation and memory traffic. Numerous modern lattice-based PI…
Onyx: Cost-Efficient Disk-Oblivious ANN Search
Deevashwer Rathee, Jean-Luc Watson, Zirui Neil Zhao +2
Approximate nearest neighbor (ANN) search in AI systems increasingly handles sensitive data on third-party infrastructure. Trusted execution environments (TEEs) offer protection, b…
Leveraging ASIC AI Chips for Homomorphic Encryption
Jianming Tong, Tianhao Huang, Jingtian Dang +9
Homomorphic Encryption (HE) provides strong data privacy for cloud services but at the cost of prohibitive computational overhead. While GPUs have emerged as a practical platform f…
EcoServe: Designing Carbon-Aware AI Inference Systems
Yueying Li, Zhanqiu Hu, Esha Choukse +3
The rapid increase in LLM ubiquity and scale levies unprecedented demands on computing infrastructure. These demands not only incur large compute and memory resources but also sign…