53 citations · 87 across the 25 of their papers we have counts for
30 papers
Dissimilarity-Based Markov Decision Processes for Stabilizing State Distributions
Xun Shen, Yuepeng Wang, Daniyal Sajedi-Hosseini +6
Reinforcement learning for stochastic systems typically optimizes expected cumulative rewards, while reliable and repeatable behavior also depends on how the state distribution evo…
Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models
Shokichi Takakura, Akifumi Wachi, Rei Higuchi +2
Inference-time alignment of flow and diffusion-based models is critical for achieving flexible generative modeling. Theoretically, Doob's -transform provides an elegant solution…
Safety Hacking in Constrained Best-of- Inference-time Scaling
Akifumi Wachi, Takumi Tanabe, Youhei Akimoto
Inference-time pipelines often sample multiple outputs, filter them with a learned safety model, and return the proxy-feasible output with the highest learned reward. We show that…
Interaction-Limited Safe Continuous-Time RL for Dynamical Medical Treatment
Xun Shen, Yuepeng Wang, Akifumi Wachi +13
Dynamic medical treatment requires deciding treatment intensity and intervention timing, while patient states evolve continuously and adverse events may occur between clinical inte…
A Unified Benchmark for Dynamic Medical Treatment Reinforcement Learning
Yuepeng Wang, Ken Kawano, Yongqi Zhou +12
Medical treatment recommendation poses several challenges to reinforcement learning (RL): patient physiology evolves in continuous time, measurements and interventions are performe…
How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
Rei Higuchi, Ryotaro Kawata, Akifumi Wachi +3
Reward modeling is not only a prediction problem: in KL-regularized policy optimization, the learned reward is exponentiated to define the deployed policy, so downstream value depe…
Sample-Efficient Hypergradient Estimation for Decentralized Bi-Level Reinforcement Learning
Mikoto Kudo, Takumi Tanabe, Akifumi Wachi +1
Many strategic decision-making problems, such as environment design for warehouse robots, can be naturally formulated as bi-level reinforcement learning (RL), where a leader agent…
Inference-Aware Meta-Alignment of LLMs via Non-Linear GRPO
Shokichi Takakura, Akifumi Wachi, Rei Higuchi +2
Aligning large language models (LLMs) to diverse human preferences is fundamentally challenging since criteria can often conflict with each other. Inference-time alignment methods…
A Relative-Budget Theory for Reinforcement Learning with Verifiable Rewards in Large Language Model Reasoning
Akifumi Wachi, Hirota Kinoshita, Shokichi Takakura +2
Reinforcement learning (RL) is a dominant paradigm for improving the reasoning abilities of large language models, yet its effectiveness varies across tasks and compute budgets. We…
Cost-Minimized Label-Flipping Poisoning Attack to LLM Alignment
Shigeki Kusaka, Keita Saito, Mikoto Kudo +3
Large language models (LLMs) are increasingly deployed in real-world systems, making it critical to understand their vulnerabilities. While data poisoning attacks during RLHF/DPO a…
A Provable Approach for End-to-End Safe Reinforcement Learning
Akifumi Wachi, Kohei Miyaguchi, Takumi Tanabe +2
A longstanding goal in safe reinforcement learning (RL) is a method to ensure the safety of a policy throughout the entire process, from learning to operation. However, existing sa…
Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies
Runze Yan, Xun Shen, Akifumi Wachi +3
When applying offline reinforcement learning (RL) in healthcare scenarios, the out-of-distribution (OOD) issues pose significant risks, as inappropriate generalization beyond clini…
Target Return Optimizer for Multi-Game Decision Transformer
Kensuke Tatematsu, Akifumi Wachi
Achieving autonomous agents with robust generalization capabilities across diverse games and tasks remains one of the ultimate goals in AI research. Recent advancements in transfor…
Vulnerability Mitigation for Safety-Aligned Language Models via Debiasing
Thien Q. Tran, Akifumi Wachi, Rei Sato +2
Safety alignment is an essential research topic for real-world AI applications. Despite the multifaceted nature of safety and trustworthiness in AI, current safety alignment method…
Flipping-based Policy for Chance-Constrained Markov Decision Processes
Xun Shen, Shuo Jiang, Akifumi Wachi +2
Safe reinforcement learning (RL) is a promising approach for many real-world decision-making problems where ensuring safety is a critical necessity. In safe RL research, while expe…
Stepwise Alignment for Constrained Language Model Policy Optimization
Akifumi Wachi, Thien Q. Tran, Rei Sato +2
Safety and trustworthiness are indispensable requirements for real-world applications of AI systems using large language models (LLMs). This paper formulates human value alignment…
A Survey of Constraint Formulations in Safe Reinforcement Learning
Akifumi Wachi, Xun Shen, Yanan Sui
Safety is critical when applying reinforcement learning (RL) to real-world problems. As a result, safe RL has emerged as a fundamental and powerful paradigm for optimizing an agent…
Long-term Safe Reinforcement Learning with Binary Feedback
Akifumi Wachi, Wataru Hashimoto, Kazumune Hashimoto
Safety is an indispensable requirement for applying reinforcement learning (RL) to real problems. Although there has been a surge of safe RL algorithms proposed in recent years, mo…
Verbosity Bias in Preference Labeling by Large Language Models
Keita Saito, Akifumi Wachi, Koki Wataoka +1
In recent years, Large Language Models (LLMs) have witnessed a remarkable surge in prevalence, altering the landscape of natural language processing and machine learning. One key f…
Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms
Akifumi Wachi, Wataru Hashimoto, Xun Shen +1
Safe exploration is essential for the practical use of reinforcement learning (RL) in many real-world scenarios. In this paper, we present a generalized safe exploration (GSE) prob…
Bayesian Meta-Learning on Control Barrier Functions with Data from On-Board Sensors
Wataru Hashimoto, Kazumune Hashimoto, Akifumi Wachi +3
In this paper, we consider a way to safely navigate the robots in unknown environments using measurement data from sensory devices. The control barrier function (CBF) is one of the…
Safe Policy Optimization with Local Generalized Linear Function Approximations
Akifumi Wachi, Yunyue Wei, Yanan Sui
Safe exploration is a key to applying reinforcement learning (RL) in safety-critical systems. Existing safe exploration methods guaranteed safety under the assumption of regularity…
LOA: Logical Optimal Actions for Text-based Interaction Games
Daiki Kimura, Subhajit Chaudhury, Masaki Ono +6
We present Logical Optimal Actions (LOA), an action decision architecture of reinforcement learning applications with a neuro-symbolic framework which is a combination of neural ne…
Neuro-Symbolic Reinforcement Learning with First-Order Logic
Daiki Kimura, Masaki Ono, Subhajit Chaudhury +6
Deep reinforcement learning (RL) methods often require many trials before convergence, and no direct interpretability of trained policies is provided. In order to achieve fast conv…