Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steering

Dongxuan Zhu, Ly Tran Ho Khanh, Andy Yat-Ming Cheung +2

Language models often default to a narrow set of high-probability outputs, leaving their generation paths homogeneous and prone to mode collapse. Sampling-based strategies inject r…