4 citations · 7 across the 21 of their papers we have counts for
21 papers
Fréchet Distance Loss on Speech Representations for Text-to-Speech Synthesis
Ho-Lam Chung, Kuan-Po Huang, Bo-Ru Lu +1
Few-step diffusion and flow-matching text-to-speech (TTS) models are usually trained with local objectives, such as conditional flow matching, reconstruction, and stop prediction.…
FdAudio: MeanFlow-Anchored Fréchet-Distance Post-Training for One-Step Text-to-Audio Generation
Kuan-Po Huang, Bo-Ru Lu, Ho-Lam Chung +2
While recent few-step sampling text-to-audio generation models like MeanAudio substantially accelerate generation by modeling average velocities, their strict one-step generation q…
Fast Text-to-Audio Generation with One-Step Sampling via Energy-Scoring and Auxiliary Contextual Representation Distillation
Kuan-Po Huang, Bo-Ru Lu, Byeonggeun Kim +8
Autoregressive (AR) models with diffusion heads have recently achieved strong text-to-audio performance, yet their iterative decoding and multi-step sampling process introduce high…
Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction
Shu-wen Yang, Byeonggeun Kim, Kuan-Po Huang +8
Autoregressive next-token prediction with the Transformer decoder has become a de facto standard in large language models (LLMs), achieving remarkable success in Natural Language P…
DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu +25
We introduce DeSTA2.5-Audio, a general-purpose Large Audio Language Model (LALM) designed for robust auditory perception and instruction-following. Recent LALMs augment Large Langu…
A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data
Cheng-Kang Chou, Chan-Jan Hsu, Ho-Lam Chung +5
We propose a self-refining framework that enhances ASR performance with only unlabeled datasets. The process starts with an existing ASR model generating pseudo-labels on unannotat…
IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling
Kuan-Po Huang, Shu-wen Yang, Huy Phan +8
Text-to-audio generation synthesizes realistic sounds or music given a natural language prompt. Diffusion-based frameworks, including the Tango and the AudioLDM series, represent t…
Enhancing Multilingual ASR for Unseen Languages via Language Embedding Modeling
Shao-Syuan Huang, Kuan-Po Huang, Andy T. Liu +1
Multilingual Automatic Speech Recognition (ASR) aims to recognize and transcribe speech from multiple languages within a single system. Whisper, one of the most advanced ASR models…
How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario
Shih-Heng Wang, Zih-Ching Chen, Jiatong Shi +6
The utilization of speech Self-Supervised Learning (SSL) models achieves impressive performance on Automatic Speech Recognition (ASR). However, in low-resource language ASR, they e…
Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks
Chien-yu Huang, Wei-Chih Chen, Shu-wen Yang +77
Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spo…
Dataset-Distillation Generative Model for Speech Emotion Recognition
Fabian Ritter-Gutierrez, Kuan-Po Huang, Jeremy H. M Wong +4
Deep learning models for speech rely on large datasets, presenting computational challenges. Yet, performance hinges on training data size. Dataset Distillation (DD) aims to learn…
Do Prompts Really Prompt? Exploring the Prompt Understanding Capability of Whisper
Chih-Kai Yang, Kuan-Po Huang, Hung-yi Lee
This research explores how the information of prompts interacts with the high-performing speech recognition model, Whisper. We compare its performances when prompted by prompts wit…
Investigating Zero-Shot Generalizability on Mandarin-English Code-Switched ASR and Speech-to-text Translation of Recent Foundation Models with Self-Supervision and Weak Supervision
Chih-Kai Yang, Kuan-Po Huang, Ke-Han Lu +3
This work evaluated several cutting-edge large-scale foundation models based on self-supervision or weak supervision, including SeamlessM4T, SeamlessM4T v2, and Whisper-large-v3, o…
Noise robust distillation of self-supervised speech models via correlation metrics
Fabian Ritter-Gutierrez, Kuan-Po Huang, Dianwen Ng +4
Compared to large speech foundation models, small distilled models exhibit degraded noise robustness. The student's robustness can be improved by introducing noise at the inputs du…
Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages
Kuan-Po Huang, Chih-Kai Yang, Yu-Kuan Fu +2
We introduce a new zero resource code-switched speech benchmark designed to directly assess the code-switching capabilities of self-supervised speech encoders. We showcase a baseli…
Ensemble knowledge distillation of self-supervised speech models
Kuan-Po Huang, Tzu-hsun Feng, Yu-Kuan Fu +5
Distilled self-supervised models have shown competitive performance and efficiency in recent years. However, there is a lack of experience in jointly distilling multiple self-super…
Improving generalizability of distilled self-supervised speech processing models under distorted settings
Kuan-Po Huang, Yu-Kuan Fu, Tsu-Yuan Hsu +5
Self-supervised learned (SSL) speech pre-trained models perform well across various speech processing tasks. Distilled versions of SSL models have been developed to match the needs…
Improving the transferability of speech separation by meta-learning
Kuan-Po Huang, Yuan-Kuei Wu, Hung-yi Lee
Speech separation aims to separate multiple speech sources from a speech mixture. Although speech separation is well-solved on some existing English speech separation benchmarks, i…
Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptation
Kuan Po Huang, Yu-Kuan Fu, Yu Zhang +1
Speech distortions are a long-standing problem that degrades the performance of supervisely trained speech processing models. It is high time that we enhance the robustness of spee…
Multi-accent Speech Separation with One Shot Learning
Kuan-Po Huang, Yuan-Kuei Wu, Hung-yi Lee
Speech separation is a problem in the field of speech processing that has been studied in full swing recently. However, there has not been much work studying a multi-accent speech…
One Shot Learning for Speech Separation
Yuan-Kuei Wu, Kuan-Po Huang, Yu Tsao +1
Despite the recent success of speech separation models, they fail to separate sources properly while facing different sets of people or noisy environments. To tackle this problem,…