9 papers
Intrinsic Task Symmetry Drives Generalization in Algorithmic Tasks
Hyeonbin Hwang, Yeachan Park
Grokking, the sudden transition from memorization to generalization, is characterized by the emergence of low-dimensional representations, yet the mechanism underlying this organiz…
Characterizing Pattern Matching and Its Limits on Compositional Task Structures
Hoyeon Chang, Jinho Park, Hanseul Cho +7
Despite impressive capabilities, LLMs' successes often rely on pattern-matching behaviors, yet these are also linked to OOD generalization failures in compositional tasks. However,…
Understanding Human Perception of Music Plagiarism Through a Computational Approach
Daeun Hwang, Hyeonbin Hwang
There is a wide variety of music similarity detection algorithms, while discussions about music plagiarism in the real world are often based on audience perceptions. Therefore, we…
Latent Reasoning via Sentence Embedding Prediction
Hyeonbin Hwang, Byeongguk Jeon, Seungone Kim +7
Autoregressive language models (LMs) generate one token at a time, yet human reasoning operates over higher-level abstractions - sentences, propositions, and concepts. This contras…
Differential Information Distribution: A Bayesian Perspective on Direct Preference Optimization
Yunjae Won, Hyunji Lee, Hyeonbin Hwang +1
Direct Preference Optimization (DPO) has been widely used for aligning language models with human preferences in a supervised manner. However, several key questions remain unresolv…
The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think
Seongyun Lee, Seungone Kim, Minju Seo +9
Long chain-of-thought (CoT) is an essential ingredient in effective usage of modern large language models, but our understanding of the reasoning strategies underlying these capabi…