12 papers · 1 filter
MultiPartyTurnBench: Benchmarking Floor Control in Multi-Party Spoken Dialogue
Puneet Mathur, Dinesh Manocha
Full-duplex speech systems must decide when to enter, hold, or relinquish the conversational floor, even as multiple speakers overlap or backchannel without taking it. Existing eva…
DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents
Puneet Mathur, Nedim Lipka, Zeyu Jin +1
Voice agents enable low-latency, natural interaction, yet their ability to faithfully ground responses in external documents remains underexplored. We introduce DuplexSpeechBench-D…
APEX-Voice: Can Voice Agents Complete Professional Workflows Through Full-Duplex Interaction
Puneet Mathur, Dinesh Manocha
Full-duplex voice agents can now listen, speak, use tools, and act during spoken interactions, but fluent dialogue does not guarantee correct completion of delegated professional w…
ASSEMBLE: Atomic Skills for Evidence-Grounded Video Reasoning
Xiyang Wu, Zongxia Li, Shengxin Zhang +2
Complex video reasoning often depends on evidence scattered across distant moments, entities, and events, yet a correct answer alone does not reveal whether a model relied on the r…
FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech
Puneet Mathur, Dinesh Manocha
Natural turn-taking in full-duplex voice interaction requires determining from partial speech whether a pause reflects hesitation or a completed conversational intent. Acoustic voi…
DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents
Puneet Mathur, Dinesh Manocha
Full-duplex voice agents must continuously decide when to speak, listen, backchannel, interrupt, overlap, and yield the conversational floor. Existing benchmarks evaluate these beh…
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Puneet Mathur, Manan Suri, Dinesh Manocha
Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally prohibitive. While recent toke…
Frames2LoRA: Parametric Video Internalization for Vision-Language Models
Manan Suri, Sarvesh Baskar, Dinesh Manocha
Processing video in vision-language models is expensive: each frame occupies hundreds of tokens, and inference cost scales with every frame and every repeated query. We introduce F…
DIAGRAMS: A Review Framework for Reasoning-Level Attribution in Diagram QA
Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Manan Suri +4
Diagram question answering (Diagram QA) requires reasoning-level attribution that links each question-answer pair to all visual regions needed to derive the answer, rather than onl…
DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams
Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Gaurav Najpande +4
Diagram question answering (DQA) requires models to interpret structured visual representations such as charts, maps, infographics, circuit schematics, and scientific diagrams. Rec…
Learning Illumination Control in Diffusion Models
Nishit Anand, Manan Suri, Christopher Metzler +2
Controlling illumination in images is essential for photography and visual content creation. While closed-source models have demonstrated impressive illumination control, open-sour…
InfinityStory: Unlimited Video Generation with World Consistency and Character-Aware Shot Transitions
Mohamed Elmoghany, Liangbing Zhao, Xiaoqian Shen +27
Generating long-form storytelling videos with consistent visual narratives remains a significant challenge in video synthesis. We present a novel framework, dataset, and a model th…