9 papers
Whisper-UT: A Unified Translation Framework for Speech and Text
Cihan Xiao, Matthew Wiesner, Debashish Chakraborty +7
Encoder-decoder models have achieved remarkable success in speech and text tasks, yet efficiently adapting these models to diverse uni/multi-modal scenarios remains an open challen…
Rapidly Adapting to New Voice Spoofing: Few-Shot Detection of Synthesized Speech Under Distribution Shifts
Ashi Garg, Zexin Cai, Henry Li Xinyuan +5
We address the challenge of detecting synthesized speech under distribution shifts -- arising from unseen synthesis methods, speakers, languages, or audio conditions -- relative to…
Scalable Controllable Accented TTS
Henry Li Xinyuan, Zexin Cai, Ashi Garg +5
We tackle the challenge of scaling accented TTS systems, expanding their capabilities to include much larger amounts of training data and a wider variety of accent labels, even for…
Uncertainty Distillation: Teaching Language Models to Express Semantic Confidence
Sophia Hager, David Mueller, Kevin Duh +1
As large language models (LLMs) are increasingly used for factual question-answering, it becomes more important for LLMs to have the capability to communicate the likelihood that t…
ShiftySpeech: A Large-Scale Synthetic Speech Dataset with Distribution Shifts
Ashi Garg, Zexin Cai, Lin Zhang +6
The problem of synthetic speech detection has enjoyed considerable attention, with recent methods achieving low error rates across several established benchmarks. However, to what…
GenVC: Self-Supervised Zero-Shot Voice Conversion
Zexin Cai, Henry Li Xinyuan, Ashi Garg +5
Most current zero-shot voice conversion methods rely on externally supervised components, particularly speaker encoders, for training. To explore alternatives that eliminate this d…
SpeechQE: Estimating the Quality of Direct Speech Translation
HyoJung Han, Kevin Duh, Marine Carpuat
Recent advances in automatic quality estimation for machine translation have exclusively focused on written language, leaving the speech modality underexplored. In this work, we fo…
HLTCOE JHU Submission to the Voice Privacy Challenge 2024
Henry Li Xinyuan, Zexin Cai, Ashi Garg +5
We present a number of systems for the Voice Privacy Challenge, including voice conversion based systems such as the kNN-VC method and the WavLM voice Conversion method, and text-t…
Privacy versus Emotion Preservation Trade-offs in Emotion-Preserving Speaker Anonymization
Zexin Cai, Henry Li Xinyuan, Ashi Garg +5
Advances in speech technology now allow unprecedented access to personally identifiable information through speech. To protect such information, the differential privacy field has…