5 citations · 6 across the 8 of their papers we have counts for
9 papers
MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems
Yuki Ichihara, Yuu Jinnai, Tetsuro Morimura +3
Group Relative Policy Optimization (GRPO) has been shown to be an effective algorithm when an accurate reward model is available. However, such a highly reliable reward model is no…
Asymmetric Perturbation in Solving Bilinear Saddle-Point Optimization
Kenshi Abe, Mitsuki Sakamoto, Kaito Ariu +1
This paper proposes asymmetric perturbation, where only one player's payoff function is perturbed, for solving bilinear saddle-point optimization problems, commonly arising in mini…
Evaluation of Best-of-N Sampling Strategies for Language Model Alignment
Yuki Ichihara, Yuu Jinnai, Tetsuro Morimura +4
Best-of-N (BoN) sampling with a reward model has been shown to be an effective strategy for aligning Large Language Models (LLMs) with human preferences at the time of decoding. Bo…
On the Power of Perturbation under Sampling in Solving Extensive-Form Games
Wataru Masaka, Mitsuki Sakamoto, Kenshi Abe +3
We investigate how perturbation does and does not improve the Follow-the-Regularized-Leader (FTRL) algorithm in solving imperfect-information extensive-form games under sampling, w…
Boosting Perturbed Gradient Ascent for Last-Iterate Convergence in Games
Kenshi Abe, Mitsuki Sakamoto, Kaito Ariu +1
This paper presents a payoff perturbation technique, introducing a strong convexity to players' payoff functions in games. This technique is specifically designed for first-order m…
Filtered Direct Preference Optimization
Tetsuro Morimura, Mitsuki Sakamoto, Yuu Jinnai +2
Reinforcement learning from human feedback (RLHF) plays a crucial role in aligning language models with human preferences. While the significance of dataset quality is generally re…
Adaptively Perturbed Mirror Descent for Learning in Games
Kenshi Abe, Kaito Ariu, Mitsuki Sakamoto +1
This paper proposes a payoff perturbation technique for the Mirror Descent (MD) algorithm in games where the gradient of the payoff functions is monotone in the strategy profile sp…
Last-Iterate Convergence with Full and Noisy Feedback in Two-Player Zero-Sum Games
Kenshi Abe, Kaito Ariu, Mitsuki Sakamoto +2
This paper proposes Mutation-Driven Multiplicative Weights Update (M2WU) for learning an equilibrium in two-player zero-sum normal-form games and proves that it exhibits the last-i…
Mutation-Driven Follow the Regularized Leader for Last-Iterate Convergence in Zero-Sum Games
Kenshi Abe, Mitsuki Sakamoto, Atsushi Iwasaki
In this study, we consider a variant of the Follow the Regularized Leader (FTRL) dynamics in two-player zero-sum games. FTRL is guaranteed to converge to a Nash equilibrium when ti…