8 papers
Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding
Dohyun Kim, Sungjun Han, Hyungguk Kim +4
Autoregressive OCR vision-language models accurately convert document images into text and structured markup, but require one sequential decoding step per output token, limiting in…
TheBioCollection: Unified Pre-Training Scale LLM Corpus for Biology
Hyunjin Seo, Hyeon Hwang, Gyubok Lee +7
The push toward large language models for biology (BioLM) has created a need for training corpora that can endow models with a genuine understanding of biology. However, existing b…
VibeProteinBench: An Evaluation Benchmark for Language-interfaced Vibe Protein Design
Hyunjin Seo, Hongjoon Ahn, Jimin Park +16
Protein design aims to compose amino-acid sequences that fold into stable three-dimensional structures while satisfying targeted functional properties. The field is increasingly sh…
Generative Visual Code Mobile World Models
Woosung Koh, Sungjun Han, Segyu Lee +2
Mobile Graphical User Interface (GUI) World Models (WMs) offer a promising path for improving mobile GUI agent performance at train- and inference-time. However, current approaches…
Predicting LLM Reasoning Performance with Small Proxy Model
Woosung Koh, Juyoung Suk, Sungjun Han +2
Given the prohibitive cost of pre-training large language models, it is essential to leverage smaller proxy models to optimize datasets before scaling up. However, this approach be…
Trillion 7B Technical Report
Sungjun Han, Juyoung Suk, Suyeong An +5
We introduce Trillion-7B, the most token-efficient Korean-centric multilingual LLM available. Our novel Cross-lingual Document Attention (XLDA) mechanism enables highly efficient a…
Generalized Gaussian Temporal Difference Error for Uncertainty-aware Reinforcement Learning
Seyeon Kim, Joonhun Lee, Namhoon Cho +2
Conventional uncertainty-aware temporal difference (TD) learning often models TD errors as zero-mean Gaussian. This assumption can miss the heavy-tailed and heteroscedastic residua…
Towards Understanding the Relationship between In-context Learning and Compositional Generalization
Sungjun Han, Sebastian Padó
According to the principle of compositional generalization, the meaning of a complex expression can be understood as a function of the meaning of its parts and of how they are comb…