7 papers
From Volume to Value: Preference-Aligned Memory Construction for On-Device RAG
Changmin Lee, Jaemin Kim, Taesik Gong
With the rapid emergence of personal AI agents based on Large Language Models (LLMs), implementing them on-device has become essential for privacy and responsiveness. To handle the…
Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate
Jaemin Kim, Sungkyun Kim, Junyeol Lee +1
Large Language Models (LLMs) are widely used across many domains, but their scale makes deployment challenging. Post-Training Quantization (PTQ) reduces memory footprint without re…
When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models
Jiho Choi, Jaemin Kim, Sanghwan Kim +2
Attention sinks are defined as tokens that attract disproportionate attention. While these have been studied in single modality transformers, their cross-modal impact in Large Visi…
Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models
Jae O Lee, Jaemin Kim, Sumyeong Ahn +1
Retrieval-Augmented Language Models (RALMs) have shown strong potential in knowledge-intensive tasks, yet they remain vulnerable when retrieved contexts are noisy or irrelevant. Ro…
FlexiQ: Adaptive Mixed-Precision Quantization for Latency/Accuracy Trade-Offs in Deep Neural Networks
Jaemin Kim, Hongjun Um, Sungkyun Kim +2
Neural networks commonly execute on hardware accelerators such as NPUs and GPUs for their size and computation overhead. These accelerators are costly and it is hard to scale their…
Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
Sungkyun Kim, Jaemin Kim, Dogyung Yoon +3
LLMs have low GPU efficiency and high latency due to autoregressive decoding. Speculative decoding (SD) mitigates this using a small draft model to speculatively generate multiple…
ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference
Hyungjun Oh, Kihong Kim, Jaemin Kim +4
This paper presents ExeGPT, a distributed system designed for constraint-aware LLM inference. ExeGPT finds and runs with an optimal execution schedule to maximize inference through…