195 citations · 407 across the 25 of their papers we have counts for
27 papers
AlgoPilot: Cross-Paradigm Reasoning in Language Models via Strategy Selection and Guidance
Yilun Hao, Ruixiao Yang, Hudson Hilal +5
Real-world problems span diverse domains, including program synthesis, symbolic reasoning, planning, and optimization, and often require fundamentally different solution paradigms.…
SHarP: Saliency-based Pruning of Agent Harnesses
Xinyi Gao, Qiucheng Wu, Kaizhi Qian +3
Agent harnesses are systems that coordinate model calls, tool use, and task execution to help large language models complete complex tasks. To meet task requirements and address fa…
Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuning
Congrui Du, Yang Zhang, Kaizhi Qian +1
Instruction tuning for speech language models (SLMs) is substantially more challenging than for text-based large language models (LLMs), as it requires learning a new modality and…
KT4EQG: Personalized Exercise Question Generation via Knowledge Tracing
Xinyi Gao, Qiucheng Wu, Lu Ding +5
Educational Question Generation (EQG) aims to synthesize customized exercise questions that enhance student learning. An effective EQG system should ideally personalize questions f…
ProsodyLM: Uncovering the Emerging Prosody Processing Capabilities in Speech Language Models
Kaizhi Qian, Xulin Fan, Junrui Ni +4
Speech language models refer to language models with speech processing and understanding capabilities. One key desirable capability for speech language models is the ability to cap…
A Hierarchical Probabilistic Framework for Incremental Knowledge Tracing in Classroom Settings
Xinyi Gao, Qiucheng Wu, Yang Zhang +4
Knowledge tracing (KT) aims to estimate a student's evolving knowledge state and predict their performance on new exercises based on performance history. Many realistic classroom s…
ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning
Bairu Hou, Yang Zhang, Jiabao Ji +4
We present ThinkPrune, a simple yet effective method for pruning the thinking length for long-thinking LLMs, which has been found to often produce inefficient and redundant thinkin…
PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play
Wei Fang, Yang Zhang, Kaizhi Qian +2
Large language models (LLMs) are increasingly integrated with specialized external tools, yet many tasks demand zero-shot tool usage with minimal or noisy documentation. Existing s…
Towards Unsupervised Speech Recognition Without Pronunciation Models
Junrui Ni, Liming Wang, Yang Zhang +4
Recent advancements in supervised automatic speech recognition (ASR) have achieved remarkable performance, largely due to the growing availability of large transcribed speech corpo…
RapVerse: Coherent Vocals and Whole-Body Motions Generations from Text
Jiaben Chen, Xin Yan, Yihang Chen +7
In this work, we introduce a challenging task for simultaneously generating 3D holistic body motions and singing vocals directly from textual lyrics inputs, advancing beyond existi…
Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling
Bairu Hou, Yujian Liu, Kaizhi Qian +3
Uncertainty decomposition refers to the task of decomposing the total uncertainty of a predictive model into aleatoric (data) uncertainty, resulting from inherent randomness in the…
Master-ASR: Achieving Multilingual Scalability and Low-Resource Adaptation in ASR with Modular Learning
Zhongzhi Yu, Yang Zhang, Kaizhi Qian +2
Despite the impressive performance recently achieved by automatic speech recognition (ASR), we observe two primary challenges that hinder its broader applications: (1) The difficul…
Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos
Kun Su, Kaizhi Qian, Eli Shlizerman +2
Modeling sounds emitted from physical object interactions is critical for immersive perceptual experiences in real and virtual worlds. Traditional methods of impact sound synthesis…
Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing
Yonggan Fu, Yang Zhang, Kaizhi Qian +4
Self-supervised learning (SSL) for rich speech representations has achieved empirical success in low-resource Automatic Speech Recognition (ASR) and other speech processing tasks,…
WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Models
Heting Gao, Junrui Ni, Kaizhi Qian +3
Large-scale auto-regressive language models pretrained on massive text have demonstrated their impressive ability to perform new natural language tasks with only a few text example…
ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers
Kaizhi Qian, Yang Zhang, Heting Gao +5
Self-supervised learning in speech involves training a speech representation network on a large-scale unannotated speech corpus, and then applying the learned representations to do…
SpeechSplit 2.0: Unsupervised speech disentanglement for voice conversion Without tuning autoencoder Bottlenecks
Chak Ho Chan, Kaizhi Qian, Yang Zhang +1
SpeechSplit can perform aspect-specific voice conversion by disentangling speech into content, rhythm, pitch, and timbre using multiple autoencoders in an unsupervised manner. Howe…
Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition
Junrui Ni, Liming Wang, Heting Gao +4
An unsupervised text-to-speech synthesis (TTS) system learns to generate speech waveforms corresponding to any written sentence in a language by observing: 1) a collection of untra…
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis
Cheng-I Jeff Lai, Erica Cooper, Yang Zhang +8
Are end-to-end text-to-speech (TTS) models over-parametrized? To what extent can these models be pruned, and what happens to their synthesis capabilities? This work serves as a sta…
Global Rhythm Style Transfer Without Text Transcriptions
Kaizhi Qian, Yang Zhang, Shiyu Chang +4
Prosody plays an important role in characterizing the style of a speaker or an emotion, but most non-parallel voice or emotion style transfer algorithms do not convert any prosody…
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
Cheng-I Jeff Lai, Yang Zhang, Alexander H. Liu +7
Self-supervised speech representation learning (speech SSL) has demonstrated the benefit of scale in learning rich representations for Automatic Speech Recognition (ASR) with limit…
Speech Denoising with Auditory Models
Mark R. Saddler, Andrew Francl, Jenelle Feather +3
Contemporary speech enhancement predominantly relies on audio transforms that are trained to reconstruct a clean speech waveform. The development of high-performing neural network…
F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder
Kaizhi Qian, Zeyu Jin, Mark Hasegawa-Johnson +1
Non-parallel many-to-many voice conversion remains an interesting but challenging speech processing task. Many style-transfer-inspired methods such as generative adversarial networ…
Unsupervised Speech Decomposition via Triple Information Bottleneck
Kaizhi Qian, Yang Zhang, Shiyu Chang +2
Speech information can be roughly decomposed into four components: language content, timbre, pitch, and rhythm. Obtaining disentangled representations of these components is useful…