Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Learning a Pessimistic Reward Model in RLHF
Yinglun Xu, Hangoo Kang, Tarun Suresh +2
This work proposes `PET', a novel pessimistic reward fine-tuning method, to learn a pessimistic reward model robust against reward hacking in offline reinforcement learning from hu…
cs.LG2024
Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignment
Jason Vega, Junsheng Huang, Gaokai Zhang +3
Safety alignment of Large Language Models (LLMs) has recently become a critical objective of model developers. In response, a growing body of work has been investigating how safety…