6 papers
CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry
Huzama Ahmad, Cao Viet Hai Nam, Se-Young Yun
Deep Transformers are composed of uniformly stacked residual blocks, yet their deepest layers often add little value. We present two efficiency methods that exploit this asymmetry.…
SpotAttention: Plug-In Block-Sparse Routing for Pretrained Long-Context Transformers
Huzama Ahmad, Se-Young Yun
Long contexts have become standard in pretrained LLMs, yet they remain expensive to run: prefill compute grows quadratically with sequence length, and every decode step re-reads a…
Process-Verified Reinforcement Learning for Theorem Proving via Lean
Minsu Kim, Se-Young Yun
While reinforcement learning from verifiable rewards (RLVR) typically has relied on a single binary verification signal, symbolic proof assistants in formal reasoning offer rich, f…
Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferences
Junghyun Lee, Minju Hong, Kwang-Sung Jun +2
We consider the problem of regularized best-response max-regret minimization in online RLHF under general preferences and bandit feedback. While various regularizers are utilized t…
An Adaptive Approach for Infinitely Many-armed Bandits under Generalized Rotting Constraints
Jung-hun Kim, Milan Vojnovic, Se-Young Yun
In this study, we consider the infinitely many-armed bandit problems in a rested rotting setting, where the mean reward of an arm may decrease with each pull, while otherwise, it r…
What is the Alignment Objective of GRPO?
Milan Vojnovic, Se-Young Yun
In this note, we examine the aggregation of preferences achieved by the Group Policy Optimisation (GRPO) algorithm, a reinforcement learning method used to train advanced artificia…