5 papers · 1 filter
Asymptotically Optimal Best Arm Identification with Fixed-Budget under Differential Privacy
Keqin Chen, Jie Bian, Yulian Wu +1
Best arm identification under differential privacy is a pure-exploration problem in which both statistical efficiency and privacy protection must be achieved simultaneously. We stu…
Offline and Online KL-Regularized RLHF under Differential Privacy
Yulian Wu, Rushil Thareja, Praneeth Vepakomma +1
In this paper, we study the offline and online settings of reinforcement learning from human feedback (RLHF) with KL-regularization -- a widely used objective function in large lan…
SquarePO: Differentially Private and Robust -Preference Optimization in Offline Direct Alignment
Xingyu Zhou, Yulian Wu, Wenqian Weng +1
In this paper, we theoretically study the offline alignment of language models with human preference feedback, under both preference label corruption and privacy protections. To th…
A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
Xingyu Zhou, Yulian Wu, Francesco Orabona
In this paper, we theoretically investigate the effects of noisy labels in offline alignment, with a focus on the interplay between privacy and robustness against adversarial corru…
Better-than-KL PAC-Bayes Bounds
Ilja Kuzborskij, Kwang-Sung Jun, Yulian Wu +2
Let be a sequence of random elements, where is a fixed scalar function, are independent random variables (data), and i…