Quieter Than the Room: Representation Drift and Task Robustness in Speech Encoders

Vsevolod Kovalev, Pranay Manocha

Non-speech interference can change a speech representation without causing comparable task loss. We test eight frozen encoders on four tasks, adding non-speech sounds throughout re…