2 papers
cs.CL2026
Agentic Reinforcement Learning for Search Misaligns Instruction-Tuning
Yushi Yang, Shreyansh Padarha, Sarah Ball +2
Agentic reinforcement learning (RL) trains large language models to use tools, but its impact on alignment is poorly understood. We study how agentic RL for search affects the alig…
cs.LG2025
How Does DPO Reduce Toxicity? A Mechanistic Neuron-Level Analysis
Yushi Yang, Filip Sondej, Harry Mayne +2
Safety fine-tuning algorithms reduce harmful outputs in language models, yet their mechanisms remain under-explored. Direct Preference Optimization (DPO) is a popular choice of alg…