1 citations · 2 across the 5 of their papers we have counts for
5 papers
GPU Acceleration of TFHE-Based High-Precision Nonlinear Layers for Encrypted LLM Inference
Guoci Chen, Xiurui Pan, Qiao Li +5
Deploying large language models (LLMs) as cloud services raises privacy concerns as inference may leak sensitive data. Fully Homomorphic Encryption (FHE) allows computation on encr…
LouisKV: Efficient KV Cache Retrieval for Long Input-Output Sequences
Wenbo Wu, Qingyi Si, Xiurui Pan +2
While Key-Value (KV) cache succeeds in reducing redundant computations in auto-regressive models, it introduces significant memory overhead, limiting its practical deployment in lo…
XBOF: A Cost-Efficient CXL JBOF with Inter-SSD Compute Resource Sharing
Shushu Yi, Yuda An, Li Peng +11
Enterprise SSDs integrate numerous computing resources (e.g., ARM processor and onboard DRAM) to satisfy the ever-increasing performance requirements of I/O bursts. While these res…
InstInfer: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference
Xiurui Pan, Endian Li, Qiao Li +6
The widespread of Large Language Models (LLMs) marks a significant milestone in generative AI. Nevertheless, the increasing context length and batch size in offline LLM inference e…
A Survey on User-Space Storage and Its Implementations
Junzhe Li, Xiurui Pan, Shushu Yi +1
The storage stack in the traditional operating system is primarily optimized towards improving the CPU utilization and hiding the long I/O latency imposed by the slow I/O devices s…