10 papers
Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation
Ku Onoda, Paavo Parmas, Hiroki Furuta +4
Text-to-image (T2I) models can synthesize realistic, prompt-aligned images, yet samples generated for the same prompt often cover only a small subset of visually distinct modes. Th…
OrderGrad: Optimizing Beyond the Mean with Order-Statistic Policy Gradient Estimation
Paavo Parmas, Yongmin Kim, Kohsei Matsutani +5
Policy-gradient methods usually optimize expected return, but many real world applications care about distributional properties of returns: tail risk, outlier robustness, or best-o…
On Advantage Estimates for Max@K Policy Gradients
Shota Takashiro, Soichiro Nishimori, Paavo Parmas +6
Reinforcement learning with verifiable rewards is widely used for post-training reasoning models, but sparse outcome rewards make exploration difficult. A complementary approach is…
Retry Policy Gradients in Continuous Action Spaces
Soichiro Nishimori, Paavo Parmas
Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration wi…
Finite-Time Regret Analysis of Retry-Aware Bandits
Bingkui Tong, Junpei Komiyama, Soichiro Nishimori +1
We study a stochastic bandit algorithm motivated by retry-aware objectives that value the best outcome among multiple attempts, such as pass@ and max@. Given a posterior over…
Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying
Soichiro Nishimori, Paavo Parmas, Sotetsu Koyamada +4
In reinforcement learning (RL), agents benefit from exploration only because they repeatedly encounter similar states: trying different actions can improve performance or reduce un…