70 citations · 129 across the 10 of their papers we have counts for
7 papers · 1 filter
ABC: Advantage-Based Control Variates for Reinforcement Learning with Verifiable Rewards
Hsiao-Ru Pan, Florent Draye, Bernhard Schölkopf
Recent progress in reinforcement learning with verifiable rewards (RLVR) has highlighted the effectiveness of simple critic-free policy-gradient methods such as Group Relative Poli…
Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learning
Hsiao-Ru Pan, Bernhard Schölkopf
Direct Advantage Estimation (DAE) has been shown to improve the sample efficiency of deep reinforcement learning algorithms. However, its reliance on full environment observability…
On the Variance of Temporal Difference Learning and its Reduction Using Control Variates
Hsiao-Ru Pan, Bernhard Schölkopf
We analyze the variance of temporal difference (TD) learning using the phased setting with tabular representation, and show that one of the mechanisms behind its ability to reduce…
Intrinsically Interpretable Attention via Sparse Post-Training
Florent Draye, Anson Lei, Hsiao-Ru Pan +2
We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsity regularisation under a constrained-…
Skill or Luck? Return Decomposition via Advantage Functions
Hsiao-Ru Pan, Bernhard Schölkopf
Learning from off-policy data is essential for sample-efficient reinforcement learning. In the present work, we build on the insight that the advantage function can be understood a…
Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitions
Hamza Keurti, Hsiao-Ru Pan, Michel Besserve +2
How can agents learn internal models that veridically represent interactions with the real world is a largely open question. As machine learning is moving towards representations c…
Direct Advantage Estimation
Hsiao-Ru Pan, Nico Gürtler, Alexander Neitz +1
The predominant approach in reinforcement learning is to assign credit to actions based on the expected return. However, we show that the return may depend on the policy in a way w…