20 citations · 26 across the 10 of their papers we have counts for
10 papers
SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization
Gunjun Lee, Sehwan Son, Younjoo Lee +2
Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2-3 bits. Existing backprop…
DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention
Younjoo Lee, Seungkyun Dan, Junghoo Lee +2
Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative d…
Scaling up FHE-based Privacy-Preserving ML: Higher Throughput, Longer Inputs for LLama-3-8B
Jaiyoung Park, Sejin Park, Jai Hyun Park +6
As large language models (LLMs) become ubiquitous, privacy concerns pertaining to inference keep growing. Fully homomorphic encryption (FHE) has emerged as a primary cryptographic…
Theodosian: A Deep Dive into Memory-Hierarchy-Centric FHE Acceleration
Wonseok Choi, Hyunah Yu, Jongmin Kim +3
Fully homomorphic encryption (FHE) enables secure computation on encrypted data, mitigating privacy concerns in cloud and edge environments. However, due to its high compute and me…
IVE: An Accelerator for Single-Server Private Information Retrieval Using Versatile Processing Elements
Sangpyo Kim, Hyesung Ji, Jongmin Kim +3
Private information retrieval (PIR) is an essential cryptographic protocol for privacy-preserving applications, enabling a client to retrieve a record from a server's database with…
From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
Gunjun Lee, Jiwon Kim, Jaiyoung Park +2
Large Language Model (LLM) inference in production must meet stringent service-level objectives for both time-to-first-token (TTFT) and time-between-token (TBT) while maximizing th…
Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
Sungmin Yun, Seonyong Park, Hwayong Nam +10
Computational workloads composing traditional transformer models are starkly bifurcated. Multi-Head Attention (MHA) and Grouped-Query Attention are memory-bound due to low arithmet…
Cheddar: A Swift Fully Homomorphic Encryption Library Designed for GPU Architectures
Wonseok Choi, Jongmin Kim, Jung Ho Ahn
Fully homomorphic encryption (FHE) frees cloud computing from privacy concerns by enabling secure computation on encrypted data. However, its substantial computational and memory o…
NeuJeans: Private Neural Network Inference with Joint Optimization of Convolution and FHE Bootstrapping
Jae Hyung Ju, Jaiyoung Park, Jongmin Kim +4
Fully homomorphic encryption (FHE) is a promising cryptographic primitive for realizing private neural network inference (PI) services by allowing a client to fully offload the inf…
Toward Practical Privacy-Preserving Convolutional Neural Networks Exploiting Fully Homomorphic Encryption
Jaiyoung Park, Donghwan Kim, Jongmin Kim +4
Incorporating fully homomorphic encryption (FHE) into the inference process of a convolutional neural network (CNN) draws enormous attention as a viable approach for achieving priv…