5 papers · 1 filter
Uncertainty-Normalized Margins for Direct Preference Optimization
Sadegh Khorasani, Petrus Mikkola, Matthias Grossglauser
Direct preference optimization (DPO) models binary preferences through a Bradley-Terry model with a common noise scale, without explicitly accounting for preference strength or pro…
Fusing Rewards and Preferences in Reinforcement Learning
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +1
We present Dual-Feedback Actor (DFA), a reinforcement learning algorithm that fuses both individual rewards and pairwise preferences (if available) into a single update rule. DFA u…
Hierarchical Reinforcement Learning with Targeted Causal Interventions
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +1
Hierarchical reinforcement learning (HRL) improves the efficiency of long-horizon reinforcement-learning tasks with sparse rewards by decomposing the task into a hierarchy of subgo…
Causal Effect Identification in a Sub-Population with Latent Variables
Amir Mohammad Abouei, Ehsan Mokhtarian, Negar Kiyavash +1
The s-ID problem seeks to compute a causal effect in a specific sub-population from the observational data pertaining to the same sub population (Abouei et al., 2023). This problem…
Efficiently Escaping Saddle Points for Policy Optimization
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +2
Policy gradient (PG) is widely used in reinforcement learning due to its scalability and good performance. In recent years, several variance-reduced PG methods have been proposed w…