8 papers
Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditions
Joonyong Park, Shinnosuke Takamichi, David M. Chan +3
Neural audio codecs (NACs) convert speech into discrete token sequences, and prior work has reported that these sequences follow language-like statistical laws. This paper analyzes…
Auditing Protocol-Level Shortcuts in Large Audio Language Model Judges for Speech Evaluation
Joonyong Park, David M. Chan, Yuki Saito +1
Large audio-language models (LALMs) are increasingly used as automatic judges for speech evaluation. However, high agreement with human ratings does not guarantee that their verdic…
Analysing the Language of Neural Audio Codecs
Joonyong Park, Shinnosuke Takamichi, David M. Chan +3
This study presents a comparative analysis of the statistical and linguistic properties of neural audio codecs (NACs). We investigate discrete speech tokens produced by various NAC…
An Efficient Self-Learning Framework For Interactive Spoken Dialog Systems
Hitesh Tulsiani, David M. Chan, Shalini Ghosh +6
Dialog systems, such as voice assistants, are expected to engage with users in complex, evolving conversations. Unfortunately, traditional automatic speech recognition (ASR) system…
Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognition
Yash Jain, David Chan, Pranav Dheram +4
Recent advances in machine learning have demonstrated that multi-modal pre-training can improve automatic speech recognition (ASR) performance compared to randomly initialized mode…
conv_einsum: A Framework for Representation and Fast Evaluation of Multilinear Operations in Convolutional Tensorial Neural Networks
Tahseen Rabbani, Jiahao Su, Xiaoyu Liu +3
Modern ConvNets continue to achieve state-of-the-art results over a vast array of vision and image classification tasks, but at the cost of increasing parameters. One strategy for…
Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognition
David M. Chan, Shalini Ghosh, Hitesh Tulsiani +2
While word error rates of automatic speech recognition (ASR) systems have consistently fallen, natural language understanding (NLU) applications built on top of ASR systems still a…
Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classification
Anirudh S. Sundar, Chao-Han Huck Yang, David M. Chan +3
Training large foundation models using self-supervised objectives on unlabeled data, followed by fine-tuning on downstream tasks, has emerged as a standard procedure. Unfortunately…