5 papers
QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention
Sehyeon Oh, Yongin Kwon, Jemin Lee
FlashAttention improves efficiency through tiling, but its online softmax still relies on floating-point arithmetic for numerical stability, making full quantization difficult. We…
Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant
Jemin Lee, Sihyeong Park, Jinse Kwon +2
Quantization has gained attention as a promising solution for the cost-effective deployment of large and small language models. However, most prior work has been limited to perplex…
Mixed Non-linear Quantization for Vision Transformers
Gihwan Kim, Jemin Lee, Sihyeong Park +2
The majority of quantization methods have been proposed to reduce the model size of Vision Transformers, yet most of them have overlooked the quantization of non-linear operations.…
QuantuneV2: Compiler-Based Local Metric-Driven Mixed Precision Quantization for Practical Embedded AI Applications
Jeongseok Kim, Jemin Lee, Yongin Kwon +1
Mixed-precision quantization methods have been proposed to reduce model size while minimizing accuracy degradation. However, existing studies require retraining and do not consider…
MLTuner: Efficient Code Tuning via Multi-Level Machine Learning Models
JooHyoung Cha, Munyoung Lee, Jinse Kwon +3
The increasing complexity of deep learning models necessitates specialized hardware and software optimizations, particularly for deep learning accelerators. Existing autotuning met…