From the 1 of 6 linked papers with an AI index.
6 papers
Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPO
Xilin Jiang, Riki Shimizu, Sukru Samet Dindar +3
The paper presents Cocktail-Talker, a speech‑language model framework that lets a spoken assistant decide whether to respond, listen, or ignore in multi‑speaker, noisy social conve…
Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioning
Sukru Samet Dindar, Riki Shimizu, Xilin Jiang +1
Empathetic spoken dialogue systems must infer a user's emotional state to respond appropriately, yet everyday speech often carries weak, neutral, or ambiguous affective cues. To ad…
AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
Xilin Jiang, Qiaolin Wang, Junkai Wu +30
Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand suc…
MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow
Riki Shimizu, Xilin Jiang, Nima Mesgarani
Target speaker extraction (TSE) aims to isolate a desired speaker's voice from a multi-speaker mixture using auxiliary information such as a reference utterance. Although recent ad…
Interpretable Embeddings of Speech Enhance and Explain Brain Encoding Performance of Audio Models
Riki Shimizu, Richard J. Antonello, Chandan Singh +1
Speech foundation models (SFMs) are increasingly hailed as powerful computational models of human speech perception. However, since their representations are inherently black-box,…
XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs
Linyang He, Ercong Nie, Sukru Samet Dindar +10
We introduce XCOMPS in this work, a multilingual conceptual minimal pair dataset covering 17 languages. Using this dataset, we evaluate LLMs' multilingual conceptual understanding…