5 papers
Learning Rate Transfer for Hybrid Transformer-SSM Architectures
Jimin Seo, Gyubok Lee, Yeonsik Jo +11
We study learning rate (LR) scaling for hybrid architectures combining Transformer and State-Space Model (SSM) blocks, a class adopted by several recent production language models.…
Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective
Jaehyeon Kim, Suhwan Kim, Nakyung Lee +4
Pause-token methods improve LLM reasoning by inserting special tokens into sequences. Prior work explains these gains through computational expressivity. However, there is relative…
Function-Level Execution Feedback for Code Preference Optimization
Idris Nechnech, Sehwan Kim, Jimin Seo +4
Process supervision has improved mathematical reasoning, where intermediate steps are naturally expressed as chains of thought. In code generation, however, process supervision rem…
Self-EvolveRec: Self-Evolving Recommender Systems with LLM-based Directional Feedback
Sein Kim, Sangwu Park, Hongseok Kang +6
Traditional methods for automating recommender system design, such as Neural Architecture Search (NAS), are often constrained by a fixed search space defined by human priors, limit…
Target Circuit Matching in Large-Scale Netlists using GNN-Based Region Prediction
Sangwoo Seo, Jimin Seo, Yoonho Lee +4
Subgraph matching plays an important role in electronic design automation (EDA) and circuit verification. Traditional rule-based methods have limitations in generalizing to arbitra…