788 citations · 789 across the 10 of their papers we have counts for
11 papers
HARP: Agentic Hybrid Retrieval and Analysis for Long-Form Audio
Chin-Jou Li, Masao Someki, Woojeong Jin +4
Long-form audio analysis requires systems to localize and integrate evidence distributed across extended recordings. While existing work primarily retrieves semantic content throug…
ESPnet3: Infrastructure for Scalable Speech and Audio Research in the Foundation Model Era
Masao Someki, Alexander Polok, Carlos Carvalho +14
Recent speech research involves increasingly large datasets, complex models, and diverse experimental workflows. However, existing frameworks require substantial engineering effort…
Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions
Jinchuan Tian, Haoran Wang, Bo-Hao Su +14
Current audio foundation models typically rely on rigid, task-specific supervision (e.g., speech recognition), addressing isolated factors of audio rather than the whole. In contra…
SingingSDS: A Singing-Capable Spoken Dialogue System for Conversational Roleplay Applications
Jionghao Han, Jiatong Shi, Masao Someki +5
With recent advances in automatic speech recognition (ASR), large language models (LLMs), and text-to-speech (TTS) technologies, spoken dialogue systems (SDS) have become widely ac…
On-device Streaming Discrete Speech Units
Kwanghee Choi, Masao Someki, Emma Strubell +1
Discrete speech units (DSUs) are derived from clustering the features of self-supervised speech models (S3Ms). DSUs offer significant advantages for on-device streaming speech appl…
Context-Driven Dynamic Pruning for Large Speech Foundation Models
Masao Someki, Shikhar Bharadwaj, Atharva Anand Joshi +7
Speech foundation models achieve strong generalization across languages and acoustic conditions, but require significant computational resources for inference. In the context of sp…
Towards Inclusive ASR: Investigating Voice Conversion for Dysarthric Speech Recognition in Low-Resource Languages
Chin-Jou Li, Eunjung Yeo, Kwanghee Choi +7
Automatic speech recognition (ASR) for dysarthric speech remains challenging due to data scarcity, particularly in non-English languages. To address this, we fine-tune a voice conv…
ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration
Masao Someki, Kwanghee Choi, Siddhant Arora +7
We introduce ESPnet-EZ, an extension of the open-source speech processing toolkit ESPnet, aimed at quick and easy development of speech models. ESPnet-EZ focuses on two major aspec…
Segment-Level Vectorized Beam Search Based on Partially Autoregressive Inference
Masao Someki, Nicholas Eng, Yosuke Higuchi +1
Attention-based encoder-decoder models with autoregressive (AR) decoding have proven to be the dominant approach for automatic speech recognition (ASR) due to their superior accura…
ESPnet-ONNX: Bridging a Gap Between Research and Production
Masao Someki, Yosuke Higuchi, Tomoki Hayashi +1
In the field of deep learning, researchers often focus on inventing novel neural network models and improving benchmarks. In contrast, application developers are interested in maki…
A Comparative Study on Transformer vs RNN in Speech Applications
Shigeki Karita, Nanxin Chen, Tomoki Hayashi +10
Sequence-to-sequence models have been widely used in end-to-end speech processing, for example, automatic speech recognition (ASR), speech translation (ST), and text-to-speech (TTS…