111 citations · 262 across the 21 of their papers we have counts for
21 papers
Efficient Neural Music Generation
Max W. Y. Lam, Qiao Tian, Tang Li +10
Recent progress in music generation has been remarkably advanced by the state-of-the-art MusicLM, which comprises a hierarchy of three LMs, respectively, for semantic, coarse acous…
Language-universal phonetic encoder for low-resource speech recognition
Siyuan Feng, Ming Tu, Rui Xia +2
Multilingual training is effective in improving low-resource ASR, which may partially be explained by phonetic representation sharing between languages. In end-to-end (E2E) ASR sys…
Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition
Siyuan Feng, Ming Tu, Rui Xia +2
We improve low-resource ASR by integrating the ideas of multilingual training and self-supervised learning. Concretely, we leverage an International Phonetic Alphabet (IPA) multili…
Conditional Energy-Based Models for Implicit Policies: The Gap between Theory and Practice
Duy-Nguyen Ta, Eric Cousineau, Huihua Zhao +1
We present our findings in the gap between theory and practice of using conditional energy-based models (EBM) as an implicit representation for behavior-cloned policies. We also cl…
Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition
Piotr Żelasko, Siyuan Feng, Laureano Moro Velazquez +5
The high cost of data acquisition makes Automatic Speech Recognition (ASR) model training problematic for most existing languages, including languages that do not even have a writt…
The Effectiveness of Time Stretching for Enhancing Dysarthric Speech for Improved Dysarthric Speech Recognition
Luke Prananta, Bence Mark Halpern, Siyuan Feng +1
In this paper, we investigate several existing and a new state-of-the-art generative adversarial network-based (GAN) voice conversion method for enhancing dysarthric speech for imp…
Unsupervised Acoustic Unit Discovery by Leveraging a Language-Independent Subword Discriminative Feature Representation
Siyuan Feng, Piotr Żelasko, Laureano Moro-Velázquez +1
This paper tackles automatically discovering phone-like acoustic units (AUD) from unlabeled speech data. Past studies usually proposed single-step approaches. We propose a two-stag…
Quantifying Bias in Automatic Speech Recognition
Siyuan Feng, Olya Kudina, Bence Mark Halpern +1
Automatic speech recognition (ASR) systems promise to deliver objective interpretation of human speech. Practice and recent evidence suggests that the state-of-the-art (SotA) ASRs…
The effectiveness of unsupervised subword modeling with autoregressive and cross-lingual phone-aware networks
Siyuan Feng, Odette Scharenborg
This study addresses unsupervised subword modeling, i.e., learning acoustic feature representations that can distinguish between subword units of a language. We propose a two-stage…
Show and Speak: Directly Synthesize Spoken Description of Images
Xinsheng Wang, Siyuan Feng, Jihua Zhu +2
This paper proposes a new model, referred to as the show and speak (SAS) model that, for the first time, is able to directly synthesize spoken descriptions of images, bypassing the…
The CUHK-TUDELFT System for The SLT 2021 Children Speech Recognition Challenge
Si-Ioi Ng, Wei Liu, Zhiyuan Peng +4
This technical report describes our submission to the 2021 SLT Children Speech Recognition Challenge (CSRC) Track 1. Our approach combines the use of a joint CTC-attention end-to-e…
Joint predictions of multi-modal ride-hailing demands: a deep multi-task multigraph learning-based approach
Jintao Ke, Siyuan Feng, Zheng Zhu +2
Ride-hailing platforms generally provide various service options to customers, such as solo ride services, shared ride services, etc. It is generally expected that demands for diff…
Unsupervised Pattern Discovery from Thematic Speech Archives Based on Multilingual Bottleneck Features
Man-Ling Sung, Siyuan Feng, Tan Lee
The present study tackles the problem of automatically discovering spoken keywords from untranscribed audio archives without requiring word-by-word speech transcription by automati…
How Phonotactics Affect Multilingual and Zero-shot ASR Performance
Siyuan Feng, Piotr Żelasko, Laureano Moro-Velázquez +4
The idea of combining multiple languages' recordings to train a single automatic speech recognition (ASR) model brings the promise of the emergence of universal speech representati…
Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling
Siyuan Feng, Odette Scharenborg
This study addresses unsupervised subword modeling, i.e., learning feature representations that can distinguish subword units of a language. The proposed approach adopts a two-stag…
Exploiting Cross-Lingual Knowledge in Unsupervised Acoustic Modeling for Low-Resource Languages
Siyuan Feng
(Short version of Abstract) This thesis describes an investigation on unsupervised acoustic modeling (UAM) for automatic speech recognition (ASR) in the zero-resource scenario, whe…
Mixture factorized auto-encoder for unsupervised hierarchical deep factorization of speech signal
Zhiyuan Peng, Siyuan Feng, Tan Lee
Speech signal is constituted and contributed by various informative factors, such as linguistic content and speaker characteristic. There have been notable recent studies attemptin…
Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling
Siyuan Feng, Tan Lee
This research addresses the problem of acoustic modeling of low-resource languages for which transcribed training data is absent. The goal is to learn robust frame-level feature re…
Improving Unsupervised Subword Modeling via Disentangled Speech Representation Learning and Transformation
Siyuan Feng, Tan Lee
This study tackles unsupervised subword modeling in the zero-resource scenario, learning frame-level speech representation that is phonetically discriminative and speaker-invariant…
Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling
Siyuan Feng, Tan Lee, Zhiyuan Peng
This study addresses the problem of unsupervised subword unit discovery from untranscribed speech. It forms the basis of the ultimate goal of ZeroSpeech 2019, building text-to-spee…
CoT: Cooperative Training for Generative Modeling of Discrete Data
Sidi Lu, Lantao Yu, Siyuan Feng +3
In this paper, we study the generative models of sequential discrete data. To tackle the exposure bias problem inherent in maximum likelihood estimation (MLE), generative adversari…