4 citations · 18 across the 28 of their papers we have counts for
28 papers
All-in-One ASR: Unifying Encoder-Decoder Models of CTC, Attention, and Transducer in Dual-Mode ASR
Takafumi Moriya, Masato Mimura, Tomohiro Tanaka +3
This paper proposes a unified framework, All-in-One ASR, that allows a single model to support multiple automatic speech recognition (ASR) paradigms, including connectionist tempor…
Generic Speech Enhancement with Self-Supervised Representation Space Loss
Hiroshi Sato, Tsubasa Ochiai, Marc Delcroix +3
Single-channel speech enhancement is utilized in various tasks to mitigate the effect of interfering signals. Conventionally, to ensure the speech enhancement performs optimally, t…
Alignment-Free Training for Transducer-based Multi-Talker ASR
Takafumi Moriya, Shota Horiguchi, Marc Delcroix +5
Extending the RNN Transducer (RNNT) to recognize multi-talker speech is essential for wider automatic speech recognition (ASR) applications. Multi-talker RNNT (MT-RNNT) aims to ach…
Factor-Conditioned Speaking-Style Captioning
Atsushi Ando, Takafumi Moriya, Shota Horiguchi +1
This paper presents a novel speaking-style captioning method that generates diverse descriptions while accurately predicting speaking-style information. Conventional learning crite…
Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff
Satoshi Suzuki, Shin'ya Yamaguchi, Shoichiro Takeda +4
This paper addresses the tradeoff between standard accuracy on clean examples and robustness against adversarial examples in deep neural networks (DNNs). Although adversarial train…
End-to-End Joint Target and Non-Target Speakers ASR
Ryo Masumura, Naoki Makishima, Taiga Yamane +12
This paper proposes a novel automatic speech recognition (ASR) system that can transcribe individual speaker's speech while identifying whether they are target or non-target speake…
Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data
Takafumi Moriya, Hiroshi Sato, Tsubasa Ochiai +7
Neural transducer (RNNT)-based target-speaker speech recognition (TS-RNNT) directly transcribes a target speaker's voice from a multi-talker mixture. It is a promising approach for…
Improving Scheduled Sampling for Neural Transducer-based ASR
Takafumi Moriya, Takanori Ashihara, Hiroshi Sato +3
The recurrent neural network-transducer (RNNT) is a promising approach for automatic speech recognition (ASR) with the introduction of a prediction network that autoregressively co…
Downstream Task Agnostic Speech Enhancement with Self-Supervised Representation Loss
Hiroshi Sato, Ryo Masumura, Tsubasa Ochiai +8
Self-supervised learning (SSL) is the latest breakthrough in speech processing, especially for label-scarce downstream tasks by leveraging massive unlabeled audio data. The noise r…
Leveraging Large Text Corpora for End-to-End Speech Summarization
Kohei Matsuura, Takanori Ashihara, Takafumi Moriya +4
End-to-end speech summarization (E2E SSum) is a technique to directly generate summary sentences from speech. Compared with the cascade approach, which combines automatic speech re…
On the Use of Modality-Specific Large-Scale Pre-Trained Encoders for Multimodal Sentiment Analysis
Atsushi Ando, Ryo Masumura, Akihiko Takashima +6
This paper investigates the effectiveness and implementation of modality-specific large-scale pre-trained encoders for multimodal sentiment analysis~(MSA). Although the effectivene…
Speaker consistency loss and step-wise optimization for semi-supervised joint training of TTS and ASR using unpaired text data
Naoki Makishima, Satoshi Suzuki, Atsushi Ando +1
In this paper, we investigate the semi-supervised joint training of text to speech (TTS) and automatic speech recognition (ASR), where a small amount of paired data and a large amo…
Strategies to Improve Robustness of Target Speech Extraction to Enrollment Variations
Hiroshi Sato, Tsubasa Ochiai, Marc Delcroix +6
Target speech extraction is a technique to extract the target speaker's voice from mixture signals using a pre-recorded enrollment utterance that characterize the voice characteris…
Audio Visual Scene-Aware Dialog Generation with Transformer-based Video Representations
Yoshihiro Yamazaki, Shota Orihashi, Ryo Masumura +2
There have been many attempts to build multimodal dialog systems that can respond to a question about given audio-visual information, and the representative task for such systems i…
Utilizing Resource-Rich Language Datasets for End-to-End Scene Text Recognition in Resource-Poor Languages
Shota Orihashi, Yoshihiro Yamazaki, Naoki Makishima +4
This paper presents a novel training method for end-to-end scene text recognition. End-to-end scene text recognition offers high recognition accuracy, especially when using the enc…
Hierarchical Knowledge Distillation for Dialogue Sequence Labeling
Shota Orihashi, Yoshihiro Yamazaki, Naoki Makishima +4
This paper presents a novel knowledge distillation method for dialogue sequence labeling. Dialogue sequence labeling is a supervised learning task that estimates labels for each ut…
End-to-End Rich Transcription-Style Automatic Speech Recognition with Semi-Supervised Learning
Tomohiro Tanaka, Ryo Masumura, Mana Ihori +3
We propose a semi-supervised learning method for building end-to-end rich transcription-style automatic speech recognition (RT-ASR) systems from small-scale rich transcription-styl…
Cross-Modal Transformer-Based Neural Correction Models for Automatic Speech Recognition
Tomohiro Tanaka, Ryo Masumura, Mana Ihori +5
We propose a cross-modal transformer-based neural correction models that refines the output of an automatic speech recognition (ASR) system so as to exclude ASR errors. Generally,…
Unified Autoregressive Modeling for Joint End-to-End Multi-Talker Overlapped Speech Recognition and Speaker Attribute Estimation
Ryo Masumura, Daiki Okamura, Naoki Makishima +4
In this paper, we present a novel modeling method for single-channel multi-talker overlapped automatic speech recognition (ASR) systems. Fully neural network based end-to-end model…
Enrollment-less training for personalized voice activity detection
Naoki Makishima, Mana Ihori, Tomohiro Tanaka +3
We present a novel personalized voice activity detection (PVAD) learning method that does not require enrollment data during training. PVAD is a task to detect the speech segments…
Zero-Shot Joint Modeling of Multiple Spoken-Text-Style Conversion Tasks using Switching Tokens
Mana Ihori, Naoki Makishima, Tomohiro Tanaka +3
In this paper, we propose a novel spoken-text-style conversion method that can simultaneously execute multiple style conversion modules such as punctuation restoration and disfluen…
Audio-Visual Speech Separation Using Cross-Modal Correspondence Loss
Naoki Makishima, Mana Ihori, Akihiko Takashima +3
We present an audio-visual speech separation learning method that considers the correspondence between the separated signals and the visual signals to reflect the speech characteri…
End-to-End Automatic Speech Recognition with Deep Mutual Learning
Ryo Masumura, Mana Ihori, Akihiko Takashima +2
This paper is the first study to apply deep mutual learning (DML) to end-to-end ASR models. In DML, multiple models are trained simultaneously and collaboratively by mimicking each…
Large-Context Conversational Representation Learning: Self-Supervised Learning for Conversational Documents
Ryo Masumura, Naoki Makishima, Mana Ihori +3
This paper presents a novel self-supervised learning method for handling conversational documents consisting of transcribed text of human-to-human conversations. One of the key tec…