Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models

Alexandrine Fortier, Hazel Chen, Peter West

The lack of diversity in LM content is widely attributed to the alignment process, but how and where exactly in the pipeline this collapse begins is unknown. We argue that output h…