21 papers
Clustering Unsupervised Representations as Defense against Poisoning Attacks on Speech Commands Classification System
Thomas Thebaud, Sonal Joshi, Henry Li +4
Poisoning attacks entail attackers intentionally tampering with training data. In this paper, we consider a dirty-label poisoning attack scenario on a speech commands classificatio…
HK-LegiCoST: Leveraging Non-Verbatim Transcripts for Speech Translation
Cihan Xiao, Henry Li Xinyuan, Jinyi Yang +4
We introduce HK-LegiCoST, a new three-way parallel corpus of Cantonese-English translations, containing 600+ hours of Cantonese audio, its standard traditional Chinese transcript,…
Universal Speech Content Factorization
Henry Li Xinyuan, Zexin Cai, Lin Zhang +5
We propose Universal Speech Content Factorization (USCF), a simple and invertible linear method for extracting a low-rank speech representation in which speaker timbre is suppresse…
Escape the Language Prior: Mitigating Late-Stage Modality Collapse in Audio Reasoning via Modality-Aware Policy Optimization
Cihan Xiao, Yiwen Shao, Chenxing Li +5
Audio and omni-modal large language models exhibit impressive cross-modal reasoning capabilities. However, applying standard reinforcement learning post-training algorithms to thes…
Dimensionality-Aware Anomaly Detection in Learned Representations of Self-Supervised Speech Models
Sandra Arcos-Holzinger, Sarah M. Erfani, James Bailey +1
Self-supervised speech models (S3Ms) achieve strong downstream performance, yet their learned representations remain poorly understood under natural and adversarial perturbations.…
Modeling Overlapped Speech with Shuffles
Matthew Wiesner, Samuele Cornell, Alexander Polok +3
We propose to model parallel streams of data, such as overlapped speech, using shuffles. Specifically, this paper shows how the shuffle product and partial order finite-state autom…