43 citations · 45 across the 7 of their papers we have counts for
7 papers
Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities
Vicky Zayats, Peter Chen, Melissa Ferrari +1
Integrating multiple generative foundation models, especially those trained on different modalities, into something greater than the sum of its parts poses significant challenges.…
AudioPaLM: A Large Language Model That Can Speak and Listen
Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen +27
We introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and…
MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup
Hua Shen, Vicky Zayats, Johann C. Rocholl +2
Current disfluency detection models focus on individual utterances each from a single speaker. However, numerous discontinuity phenomena in spoken conversational transcripts occur…
Chronological Self-Training for Real-Time Speaker Diarization
Dirk Padfield, Daniel J. Liebling
Diarization partitions an audio stream into segments based on the voices of the speakers. Real-time diarization systems that include an enrollment step should limit enrollment trai…
Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection
Angelica Chen, Vicky Zayats, Daniel D. Walker +1
In modern interactive speech-based systems, speech is consumed and transcribed incrementally prior to having disfluencies removed. This post-processing step is crucial for producin…
Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speech
Katrin Tomanek, Vicky Zayats, Dirk Padfield +2
Automatic Speech Recognition (ASR) systems are often optimized to work best for speakers with canonical speech patterns. Unfortunately, these systems perform poorly when tested on…
Sentence Boundary Augmentation For Neural Machine Translation Robustness
Daniel Li, Te I, Naveen Arivazhagan +2
Neural Machine Translation (NMT) models have demonstrated strong state of the art performance on translation tasks where well-formed training and evaluation data are provided, but…