From the 1 of 7 linked papers with an AI index.
7 papers
AMUSE: Anytime Muon with Stable Gradient Evaluation
Jueun Kim, Baekrok Shin, Jihun Yun +3
The paper introduces AMUSE, an optimization method that combines Muon's orthogonal momentum updates with Schedule-Free iterate averaging to achieve fast, stable training without le…
Pruning and Distilling Mixture-of-Experts into Dense Language Models
Junhyuck Kim, Jihun Yun, Haechan Kim +3
Mixture-of-Experts (MoE) is now the dominant architecture for frontier language models, yet it requires all expert parameters to be loaded in memory, making it less preferable for…
Beyond RLHF: A Unified Theoretical Framework of Alignment
Jihun Yun, Juno Kim, Jongho Park +4
Alignment via reinforcement learning from human feedback (RLHF) has become the dominant paradigm for controlling the quality of outputs from large language models (LLMs). However,…
THINKSAFE: Self-Generated Safety Alignment for Reasoning Models
Seanie Lee, Sangwoo Park, Yumin Choi +6
Large reasoning models (LRMs) achieve remarkable performance by leveraging reinforcement learning (RL) on reasoning tasks to generate long chain-of-thought (CoT) reasoning. However…
Raon-Speech Technical Report
Beomsoo Kim, Changho Choi, Dohyun Kim +23
We present Raon-Speech, a top-performing 9B-parameter speech language model (SpeechLM) for English and Korean speech understanding, answering, and generation, and Raon-SpeechChat,…
Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization
Changmin Kang, Jihun Yun, Baekrok Shin +2
Spectral gradient descent (SpecGD) orthogonalizes the matrix parameter updates and has inspired practical optimizers such as Muon. They often perform well in large language model (…