12 citations · 22 across the 23 of their papers we have counts for
24 papers
Kepler-Encoder-v0.1: Towards a Multimodal Embedding Model for Robots
Ishneet Sukhvinder Singh, Dhanoosh Pooranakumaran, Alex Nguyen +1
A robot must understand the state of its own body, but a camera sees only part of it. Force and contact leave almost no trace in a single frame, and raw vision features read force…
Bona fide Cross Testing Reveals Weak Spot in Audio Deepfake Detection Systems
Chin Yuen Kwok, Jia Qi Yip, Zhen Qiu +2
Audio deepfake detection (ADD) models are commonly evaluated using datasets that combine multiple synthesizers, with performance reported as a single Equal Error Rate (EER). Howeve…
Improving Synthetic Data Training for Contextual Biasing Models with a Keyword-Aware Cost Function
Chin Yuen Kwok, Jia Qi Yip, Eng Siong Chng
Rare word recognition can be improved by adapting ASR models to synthetic data that includes these words. Further improvements can be achieved through contextual biasing, which tra…
Efficient Trie-based Biasing using K-step Prediction for Rare Word Recognition
Chin Yuen Kwok, Jia Qi yip
Contextual biasing improves rare word recognition of ASR models by prioritizing the output of rare words during decoding. A common approach is Trie-based biasing, which gives "bonu…
Speechless: Speech Instruction Training Without Speech for Low Resource Languages
Alan Dao, Dinh Bach Vu, Huy Hoang Ha +6
The rapid growth of voice assistants powered by large language models (LLM) has highlighted a need for speech instruction data to train these systems. Despite the abundance of spee…
Continual Learning with Embedding Layer Surgery and Task-wise Beam Search using Whisper
Chin Yuen Kwok, Jia Qi Yip, Eng Siong Chng
Current Multilingual ASR models only support a fraction of the world's languages. Continual Learning (CL) aims to tackle this problem by adding new languages to pre-trained models…
VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music
Jiatong Shi, Hye-jin Shim, Jinchuan Tian +14
In this work, we introduce VERSA, a unified and standardized evaluation toolkit designed for various speech, audio, and music signals. The toolkit features a Pythonic interface wit…
Speech Separation using Neural Audio Codecs with Embedding Loss
Jia Qi Yip, Chin Yuen Kwok, Bin Ma +1
Neural audio codecs have revolutionized audio processing by enabling speech tasks to be performed on highly compressed representations. Recent work has shown that speech separation…
Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks
Chien-yu Huang, Wei-Chih Chen, Shu-wen Yang +77
Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spo…
Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotions
Kun Zhou, You Zhang, Dianwen Ng +3
Emotional text-to-speech (TTS) systems sturggle to capture the full spectrum of human emotions due to the inherent complexity of emotional expressions and the limited coverage of e…
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speech
Jiatong Shi, Jinchuan Tian, Yihan Wu +17
Neural codecs have become crucial to recent speech and audio generation research. In addition to signal compression capabilities, discrete codecs have also been found to enhance do…
Towards Audio Codec-based Speech Separation
Jia Qi Yip, Shengkui Zhao, Dianwen Ng +2
Recent improvements in neural audio codec (NAC) models have generated interest in adopting pre-trained codecs for a variety of speech processing applications to take advantage of t…
Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systems
Chin Yuen Kwok, Jia Qi Yip, Eng Siong Chng
Continual Learning (CL) involves fine-tuning pre-trained models with new data while maintaining the performance on the pre-trained data. This is particularly relevant for expanding…
Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis
Kun Zhou, Shengkui Zhao, Yukun Ma +7
Recent language model-based text-to-speech (TTS) frameworks demonstrate scalability and in-context learning capabilities. However, they suffer from robustness issues due to the acc…
MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation
Shengkui Zhao, Yukun Ma, Chongjia Ni +7
Our previously proposed MossFormer has achieved promising performance in monaural speech separation. However, it predominantly adopts a self-attention-based MossFormer module, whic…
Codec Data Augmentation for Time-domain Heart Sound Classification
Ansh Mishra, Jia Qi Yip, Eng Siong Chng
Heart auscultations are a low-cost and effective way of detecting valvular heart diseases early, which can save lives. Nevertheless, it has been difficult to scale this screening m…
Analysis of Speech Separation Performance Degradation on Emotional Speech Mixtures
Jia Qi Yip, Dianwen Ng, Bin Ma +1
Despite recent strides made in Speech Separation, most models are trained on datasets with neutral emotions. Emotional speech has been known to degrade performance of models in a v…
Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verification
Duc-Tuan Truong, Ruijie Tao, Jia Qi Yip +2
Knowledge distillation (KD) is used to enhance automatic speaker verification performance by ensuring consistency between large teacher networks and lightweight student networks at…
SPGM: Prioritizing Local Features for enhanced speech separation performance
Jia Qi Yip, Shengkui Zhao, Yukun Ma +8
Dual-path is a popular architecture for speech separation models (e.g. Sepformer) which splits long sequences into overlapping chunks for its intra- and inter-blocks that separatel…
ACA-Net: Towards Lightweight Speaker Verification using Asymmetric Cross Attention
Jia Qi Yip, Tuan Truong, Dianwen Ng +7
In this paper, we propose ACA-Net, a lightweight, global context-aware speaker embedding extractor for Speaker Verification (SV) that improves upon existing work by using Asymmetri…
Contrastive Speech Mixup for Low-resource Keyword Spotting
Dianwen Ng, Ruixi Zhang, Jia Qi Yip +6
Most of the existing neural-based models for keyword spotting (KWS) in smart devices require thousands of training samples to learn a decent audio representation. However, with the…
deHuBERT: Disentangling Noise in a Self-supervised Model for Robust Speech Recognition
Dianwen Ng, Ruixi Zhang, Jia Qi Yip +7
Existing self-supervised pre-trained speech models have offered an effective way to leverage massive unannotated corpora to build good automatic speech recognition (ASR). However,…
I2CR: Improving Noise Robustness on Keyword Spotting Using Inter-Intra Contrastive Regularization
Dianwen Ng, Jia Qi Yip, Tanmay Surana +6
Noise robustness in keyword spotting remains a challenge as many models fail to overcome the heavy influence of noises, causing the deterioration of the quality of feature embeddin…
Amino Acid Classification in 2D NMR Spectra via Acoustic Signal Embeddings
Jia Qi Yip, Dianwen Ng, Bin Ma +2
Nuclear Magnetic Resonance (NMR) is used in structural biology to experimentally determine the structure of proteins, which is used in many areas of biology and is an important par…