1.9k citations · 2k across the 21 of their papers we have counts for
22 papers
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1
With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associatio…
NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription
Alon Vinnikov, Amir Ivry, Aviv Hurvitz +16
We introduce the first Natural Office Talkers in Settings of Far-field Audio Recordings (``NOTSOFAR-1'') Challenge alongside datasets and baseline system. The challenge focuses on…
Profile-Error-Tolerant Target-Speaker Voice Activity Detection
Dongmei Wang, Xiong Xiao, Naoyuki Kanda +3
Target-Speaker Voice Activity Detection (TS-VAD) utilizes a set of speaker profiles alongside an input audio signal to perform speaker diarization. While its superiority over conve…
Speaker Change Detection for Transformer Transducer ASR
Jian Wu, Zhuo Chen, Min Hu +2
Speaker change detection (SCD) is an important feature that improves the readability of the recognized words from an automatic speech recognition (ASR) system by breaking the word…
Target Speaker Voice Activity Detection with Transformers and Its Integration with End-to-End Neural Diarization
Dongmei Wang, Xiong Xiao, Naoyuki Kanda +2
This paper describes a speaker diarization model based on target speaker voice activity detection (TS-VAD) using transformers. To overcome the original TS-VAD model's drawback of b…
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
Naoyuki Kanda, Jian Wu, Yu Wu +7
This paper presents a streaming speaker-attributed automatic speech recognition (SA-ASR) model that can recognize ``who spoke what'' with low latency even when multiple people are…
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
Naoyuki Kanda, Jian Wu, Yu Wu +7
This paper proposes a token-level serialized output training (t-SOT), a novel framework for streaming multi-talker automatic speech recognition (ASR). Unlike existing streaming mul…
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Wangyou Zhang, Zhuo Chen, Naoyuki Kanda +8
Multi-talker conversational speech processing has drawn many interests for various applications such as meeting transcription. Speech separation is often required to handle overlap…
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Sanyuan Chen, Chengyi Wang, Zhengyang Chen +15
Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks. As speech signal contai…
Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR
Naoyuki Kanda, Xiong Xiao, Yashesh Gaur +4
This paper presents Transcribe-to-Diarize, a new approach for neural speaker diarization that uses an end-to-end (E2E) speaker-attributed automatic speech recognition (SA-ASR). The…
A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio
Naoyuki Kanda, Xiong Xiao, Jian Wu +6
Speaker-attributed automatic speech recognition (SA-ASR) is a task to recognize "who spoke what" from multi-talker recordings. An SA-ASR system usually consists of multiple modules…
Speaker attribution with voice profiles by graph-based semi-supervised learning
Jixuan Wang, Xiong Xiao, Jian Wu +3
Speaker attribution is required in many real-world applications, such as meeting transcription, where speaker identity is assigned to each utterance according to speaker voice prof…
Microsoft Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2020
Xiong Xiao, Naoyuki Kanda, Zhuo Chen +10
This paper describes the Microsoft speaker diarization system for monaural multi-talker recordings in the wild, evaluated at the diarization track of the VoxCeleb Speaker Recogniti…
Speaker diarization with session-level speaker embedding refinement using graph neural networks
Jixuan Wang, Xiong Xiao, Jian Wu +3
Deep speaker embedding models have been commonly used as a building block for speaker diarization systems; however, the speaker embedding model is usually trained according to a gl…
Advances in Online Audio-Visual Meeting Transcription
Takuya Yoshioka, Igor Abramovski, Cem Aksoylar +23
This paper describes a system that generates speaker-annotated transcripts of meetings by using a microphone array and a 360-degree camera. The hallmark of the system is its abilit…
PyKaldi2: Yet another speech toolkit based on Kaldi and PyTorch
Liang Lu, Xiong Xiao, Zhuo Chen +1
We introduce PyKaldi2 speech recognition toolkit implemented based on Kaldi and PyTorch. While similar toolkits are available built on top of the two, a key feature of PyKaldi2 is…
Low-Latency Speaker-Independent Continuous Speech Separation
Takuya Yoshioka, Zhuo Chen, Changliang Liu +3
Speaker independent continuous speech separation (SI-CSS) is a task of converting a continuous audio stream, which may contain overlapping voices of unknown speakers, into a fixed…
Recognizing Overlapped Speech in Meetings: A Multichannel Separation Approach Using Neural Networks
Takuya Yoshioka, Hakan Erdogan, Zhuo Chen +2
The goal of this work is to develop a meeting transcription system that can recognize speech even when utterances of different speakers are overlapped. While speech overlaps have b…
Developing Far-Field Speaker System Via Teacher-Student Learning
Jinyu Li, Rui Zhao, Zhuo Chen +4
In this study, we develop the keyword spotting (KWS) and acoustic model (AM) components in a far-field speaker system. Specifically, we use teacher-student (T/S) learning to adapt…
Cracking the cocktail party problem by multi-beam deep attractor network
Zhuo Chen, Jinyu Li, Xiong Xiao +4
While recent progresses in neural network approaches to single-channel speech separation, or more generally the cocktail party problem, achieved significant improvement, their perf…
Noise Robust Speech Recognition Using Multi-Channel Based Channel Selection And ChannelWeighting
Zhaofeng Zhang, Xiong Xiao, Longbiao Wang +2
In this paper, we study several microphone channel selection and weighting methods for robust automatic speech recognition (ASR) in noisy conditions. For channel selection, we inve…
Fantastic 4 system for NIST 2015 Language Recognition Evaluation
Kong Aik Lee, Ville Hautamäki, Anthony Larcher +14
This article describes the systems jointly submitted by Institute for Infocomm (IR), the Laboratoire d'Informatique de l'Université du Maine (LIUM), Nanyang Technology Universi…