8 papers
Scaling up FHE-based Privacy-Preserving ML: Higher Throughput, Longer Inputs for LLama-3-8B
Jaiyoung Park, Sejin Park, Jai Hyun Park +6
As large language models (LLMs) become ubiquitous, privacy concerns pertaining to inference keep growing. Fully homomorphic encryption (FHE) has emerged as a primary cryptographic…
DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention
Younjoo Lee, Seungkyun Dan, Junghoo Lee +2
Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative d…
From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
Gunjun Lee, Jiwon Kim, Jaiyoung Park +2
Large Language Model (LLM) inference in production must meet stringent service-level objectives for both time-to-first-token (TTFT) and time-between-token (TBT) while maximizing th…
Theodosian: A Deep Dive into Memory-Hierarchy-Centric FHE Acceleration
Wonseok Choi, Hyunah Yu, Jongmin Kim +3
Fully homomorphic encryption (FHE) enables secure computation on encrypted data, mitigating privacy concerns in cloud and edge environments. However, due to its high compute and me…
Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
Sungmin Yun, Seonyong Park, Hwayong Nam +10
Computational workloads composing traditional transformer models are starkly bifurcated. Multi-Head Attention (MHA) and Grouped-Query Attention are memory-bound due to low arithmet…
IVE: An Accelerator for Single-Server Private Information Retrieval Using Versatile Processing Elements
Sangpyo Kim, Hyesung Ji, Jongmin Kim +3
Private information retrieval (PIR) is an essential cryptographic protocol for privacy-preserving applications, enabling a client to retrieve a record from a server's database with…