Forty Shades of Blue: Quality-Diversity Alignment via Mode-Conditioned Reinforcement Learning

Jiayi Yuan, Hangoo Kang, James Jihao Liu +4

A notable byproduct of LLM alignment training is mode collapse: the progressive loss of output diversity that narrows a model's expressivity at inference time. This degradation is…