10 papers
MamTra: A Hybrid Mamba-Transformer Backbone for Speech Synthesis
Tan Dat Nguyen, Sangmin Bae, Joon Son Chung +1
Despite the remarkable quality of LLM-based text-to-speech systems, their reliance on autoregressive Transformers leads to quadratic computational complexity, which severely limits…
Plug-and-Steer: Decoupling Separation and Selection in Audio-Visual Target Speaker Extraction
Doyeop Kwak, Suyeon Lee, Joon Son Chung
The goal of this paper is to provide a new perspective on audio-visual target speaker extraction (AV-TSE) by decoupling separation and target selection. Conventional AV-TSE systems…
SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning
KiHyun Nam, Jungwoo Heo, Siu Bae +2
As audio-first agents become increasingly common in physical AI, conversational robots, and screenless wearables, audio large language models (audio-LLMs) must integrate speaker-sp…
FiTS: Interpretable Spiking Neurons via Frequency Selectivity and Temporal Shaping
Jongmin Choi, Joon Son Chung
Spiking Neural Networks (SNNs) are a promising framework for event-driven temporal processing. Prior work has improved temporal modeling through richer neuron dynamics and network-…
Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs
Chaeyoung Jung, Kyeongha Rho, Joon Son Chung
Omnimodal Large Language Models (Omni-LLMs) incur substantial computational overhead due to the large number of multimodal input tokens they process, making token reduction essenti…
Probing Cross-modal Information Hubs in Audio-Visual LLMs
Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim +1
Audio-visual large language models (AVLLMs) have recently emerged as a powerful architecture capable of jointly reasoning over audio, visual, and textual modalities. In AVLLMs, the…