1 citations · 1 across the 9 of their papers we have counts for
11 papers
Asymmetric Classifier-Free Guidance for Target-Speaker ASR
Yiwen Guan, Jacob Whitehill
Target-speaker automatic speech recognition (TS-ASR) must identify and transcribe a desired speaker under varying overlap and noise conditions. These changes alter the acoustic evi…
Efficient One-to-Many Translation with Joint Multi-Stream Diffusion
Yiwen Guan, Jacob Whitehill
One-to-many machine translation (MT) is computationally expensive for autoregressive (AR) systems, which suffer from linear latency scaling with both sequence length and the number…
Learning to Translate from Soft to Hard LLM Prompts
Pitipat Kongsomjit, Suryansh Goyal, Jacob Whitehill
Soft prompting, also known as continuous prompting, is a parameter-efficient method for tuning LLMs to specific tasks. Like other machine learning techniques, its parameters encode…
Quantization of Spiking Neural Networks Beyond Accuracy
Evan Gibson Smith, Jacob Whitehill, Fatemeh Ganji
Quantization is a natural complement to the sparse, event-driven computation of Spiking Neural Networks, reducing memory bandwidth and arithmetic cost for deployment on resource-co…
Interactive In-Meeting Speaker Correction with Human Feedback
Xinlu He, Yiwen Guan, Badrivishal Paurana +3
Most automatic speech processing systems operate in ``open loop'' mode without user feedback about who said what, yet human-in-the-loop workflows can potentially enable higher accu…
Transformer-Encoder Trees for Efficient Multilingual Machine Translation and Speech Translation
Yiwen Guan, Jacob Whitehill
Multilingual translation suffers from computational redundancy, especially when translating into multiple languages simultaneously. In addition, translation quality can suffer for…
MLLM-based Speech Recognition: When and How is Multimodality Beneficial?
Yiwen Guan, Viet Anh Trinh, Vivek Voleti +1
Recent advances in multi-modal large language models (MLLMs) have opened new possibilities for unified modeling of speech, text, images, and other modalities. Building on our prior…
Improving Speech Recognition of Named Entities in Classroom Speech with LLM Revision and Phonetic-Semantic Context
Viet Anh Trinh, Xinlu He, Jacob Whitehill
Classroom speech and lectures often contain named entities (NEs) such as names of people and special terminology. While automatic speech recognition (ASR) systems have achieved rem…
Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audio
Xinlu He, Jacob Whitehill
Monaural multi-speaker automatic speech recognition (ASR) remains challenging due to data scarcity and the intrinsic difficulty of recognizing and attributing words to individual s…
Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy?
Yiwen Guan, Viet Anh Trinh, Vivek Voleti +1
Decoder-only discrete-token language models have recently achieved significant success in automatic speech recognition. However, systematic analyses of how different modalities imp…
Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing
Viet Anh Trinh, Rosy Southwell, Yiwen Guan +3
Recent work on discrete speech tokenization has paved the way for models that can seamlessly perform multiple tasks across modalities, e.g., speech recognition, text to speech, spe…