261 citations · 682 across the 43 of their papers we have counts for
24 papers · 1 filter
IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages
Tahir Javed, Janki Atul Nawale, Eldho Ittan George +18
We present INDICVOICES, a dataset of natural and spontaneous speech containing a total of 7348 hours of read (9%), extempore (74%) and conversational (17%) audio from 16237 speaker…
Scaling Speech Technology to 1,000+ Languages
Vineel Pratap, Andros Tjandra, Bowen Shi +13
Expanding the language coverage of speech technology has the potential to improve access to information for many more people. However, current speech technology is restricted to ab…
SpeeChain: A Speech Toolkit for Large-Scale Machine Speech Chain
Heli Qi, Sashi Novitasari, Andros Tjandra +2
This paper introduces SpeeChain, an open-source Pytorch-based toolkit designed to develop the machine speech chain for large-scale use. This first release focuses on the TTS-to-ASR…
Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities
Andros Tjandra, Nayan Singhal, David Zhang +4
End-to-end multilingual ASR has become more appealing because of several reasons such as simplifying the training and deployment process and positive performance transfer from high…
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
Arun Babu, Changhan Wang, Andros Tjandra +10
This paper presents XLS-R, a large-scale model for cross-lingual speech representation learning based on wav2vec 2.0. We train models with up to 2B parameters on nearly half a mill…
Improved Language Identification Through Cross-Lingual Self-Supervised Learning
Andros Tjandra, Diptanu Gon Choudhury, Frank Zhang +6
Language identification greatly impacts the success of downstream tasks such as automatic speech recognition. Recently, self-supervised speech representations learned by wav2vec 2.…
Cross-Lingual Machine Speech Chain for Javanese, Sundanese, Balinese, and Bataks Speech Recognition and Synthesis
Sashi Novitasari, Andros Tjandra, Sakriani Sakti +1
Even though over seven hundred ethnic languages are spoken in Indonesia, the available technology remains limited that could support communication within indigenous communities as…
Sequence-to-Sequence Learning via Attention Transfer for Incremental Speech Recognition
Sashi Novitasari, Andros Tjandra, Sakriani Sakti +1
Attention-based sequence-to-sequence automatic speech recognition (ASR) requires a significant delay to recognize long utterances because the output is generated after receiving en…
Incremental Machine Speech Chain Towards Enabling Listening while Speaking in Real-time
Sashi Novitasari, Andros Tjandra, Tomoya Yanagita +2
Inspired by a human speech chain mechanism, a machine speech chain framework based on deep learning was recently proposed for the semi-supervised development of automatic speech re…
Augmenting Images for ASR and TTS through Single-loop and Dual-loop Multimodal Chain Framework
Johanes Effendi, Andros Tjandra, Sakriani Sakti +1
Previous research has proposed a machine speech chain to enable automatic speech recognition (ASR) and text-to-speech synthesis (TTS) to assist each other in semi-supervised learni…
Unsupervised Learning of Disentangled Speech Content and Style Representation
Andros Tjandra, Ruoming Pang, Yu Zhang +1
We present an approach for unsupervised learning of speech representation disentangling contents and styles. Our model consists of: (1) a local encoder that captures per-frame info…
Transformer VQ-VAE for Unsupervised Unit Discovery and Speech Synthesis: ZeroSpeech 2020 Challenge
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
In this paper, we report our submitted system for the ZeroSpeech 2020 challenge on Track 2019. The main theme in this challenge is to build a speech synthesizer without any textual…
Speech-to-speech Translation between Untranscribed Unknown Languages
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
In this paper, we explore a method for training speech-to-speech translation tasks without any transcription or linguistic supervision. Our proposed method consists of two steps: F…
Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
Andros Tjandra, Chunxi Liu, Frank Zhang +5
Deep acoustic models typically receive features in the first layer of the network, and process increasingly abstract representations in the subsequent layers. Here, we propose to f…
Transformer-based Acoustic Modeling for Hybrid Speech Recognition
Yongqiang Wang, Abdelrahman Mohamed, Duc Le +10
We propose and evaluate transformer-based acoustic models (AMs) for hybrid speech recognition. Several modeling choices are discussed in this work, including various positional emb…
Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain
Johanes Effendi, Andros Tjandra, Sakriani Sakti +1
Previously, a machine speech chain, which is based on sequence-to-sequence deep learning, was proposed to mimic speech perception and production behavior. Such chains separately pr…
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019
Andros Tjandra, Berrak Sisman, Mingyang Zhang +3
We describe our submitted system for the ZeroSpeech Challenge 2019. The current challenge theme addresses the difficulty of constructing a speech synthesizer without any text or ph…
End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimator
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
The speech chain mechanism integrates automatic speech recognition (ASR) and text-to-speech synthesis (TTS) modules into a single cycle during training. In our previous work, we ap…
Multi-scale Alignment and Contextual History for Attention Mechanism in Sequence-to-sequence Model
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
A sequence-to-sequence model is a neural network module for mapping two sequences of different lengths. The sequence-to-sequence model has three core modules: encoder, decoder, and…
Machine Speech Chain with One-shot Speaker Adaptation
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
In previous work, we developed a closed-loop speech chain model based on deep learning, in which the architecture enabled the automatic speech recognition (ASR) and text-to-speech…
Sequence-to-Sequence ASR Optimization via Reinforcement Learning
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Despite the success of sequence-to-sequence approaches in automatic speech recognition (ASR) systems, the models still suffer from several problems, mainly due to the mismatch betw…
Attention-based Wav2Text with Feature Transfer Learning
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Conventional automatic speech recognition (ASR) typically performs multi-level pattern recognition tasks that map the acoustic speech waveform into a hierarchy of speech units. But…
Listening while Speaking: Speech Chain by Deep Learning
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Despite the close relationship between speech perception and production, research in automatic speech recognition (ASR) and text-to-speech synthesis (TTS) has progressed more or le…
Local Monotonic Attention Mechanism for End-to-End Speech and Language Processing
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Recently, encoder-decoder neural networks have shown impressive performance on many sequence-related tasks. The architecture commonly uses an attentional mechanism which allows the…