6 papers
EXAONE 4.5 Technical Report
Eunbi Choi, Kibong Choi, Sehyun Chun +55
This technical report introduces EXAONE 4.5, the first open-weight vision language model released by LG AI Research. EXAONE 4.5 is architected by integrating a dedicated visual enc…
PruneCD: Contrasting Pruned Self Model to Improve Decoding Factuality
Byeongho Yu, Changhun Lee, Jungyu Jin +1
To mitigate the hallucination problem in large language models, DoLa exploits early exit logits from the same model as a contrastive prior. However, we found that these early exit…
AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
Sangjun Lee, Seung-taek Woo, Jungyu Jin +2
To enable broader deployment of Large Language Models (LLMs), it is essential to identify the best-performing model under strict memory constraints. We present AMQ, Automated Mixed…
SEAL: Scaling to Emphasize Attention for Long-Context Retrieval
Changhun Lee, Minsang Seok, Jun-gyu Jin +2
While many advanced LLMs are designed to handle long sequence data, we can still observe notable quality degradation even within the sequence limit. In this work, we introduce a no…
PTQ4VM: Post-Training Quantization for Visual Mamba
Younghyun Cho, Changhun Lee, Seonggon Kim +1
Visual Mamba is an approach that extends the selective space state model, Mamba, to vision tasks. It processes image tokens sequentially in a fixed order, accumulating information…
QEFT: Quantization for Efficient Fine-Tuning of LLMs
Changhun Lee, Jun-gyu Jin, Younghyun Cho +1
With the rapid growth in the use of fine-tuning for large language models (LLMs), optimizing fine-tuning while keeping inference efficient has become highly important. However, thi…