Inference-Time Nash Alignment

Hadi Hosseini, Debmalya Mandal, Duohan Zhang

Preference-based fine-tuning methods such as RLHF and DPO require substantial compute and large preference datasets. They also need direct access to the model parameters which are…