7 papers
Uncertainty-Normalized Margins for Direct Preference Optimization
Sadegh Khorasani, Petrus Mikkola, Matthias Grossglauser
Direct preference optimization (DPO) models binary preferences through a Bradley-Terry model with a common noise scale, without explicitly accounting for preference strength or pro…
Inference Time Causal Probing in LLMs
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +1
Causal probing methods aim to test and control how internal representations influence the behavior of generative models. In causal probing, an intervention modifies hidden states s…
Fusing Rewards and Preferences in Reinforcement Learning
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +1
We present Dual-Feedback Actor (DFA), a reinforcement learning algorithm that fuses both individual rewards and pairwise preferences (if available) into a single update rule. DFA u…
Hierarchical Reinforcement Learning with Targeted Causal Interventions
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +1
Hierarchical reinforcement learning (HRL) improves the efficiency of long-horizon reinforcement-learning tasks with sparse rewards by decomposing the task into a hierarchy of subgo…
Causal Effect Identification in a Sub-Population with Latent Variables
Amir Mohammad Abouei, Ehsan Mokhtarian, Negar Kiyavash +1
The s-ID problem seeks to compute a causal effect in a specific sub-population from the observational data pertaining to the same sub population (Abouei et al., 2023). This problem…
Efficiently Escaping Saddle Points for Policy Optimization
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash +2
Policy gradient (PG) is widely used in reinforcement learning due to its scalability and good performance. In recent years, several variance-reduced PG methods have been proposed w…
Studying Lobby Influence in the European Parliament
Aswin Suresh, Lazar Radojevic, Francesco Salvi +3
We present a method based on natural language processing (NLP), for studying the influence of interest groups (lobbies) in the law-making process in the European Parliament (EP). W…