7 papers
Semantic Action Graph: A Shared Representation for Agent Grounding and Human Interpretation of Sports Highlights
Tica Lin, Deepak Chandran, Gauri Jagatap +4
Generative agents are increasingly used to select and narrate video highlights, but they typically operate over unstructured or frame-level representations. Their output is consequ…
Video2Reaction: Training Foundation Video Models to Predict Audience Reaction
Sidong Zhang, Trang Nguyen, Shiv Shankar +4
We introduce Video2Reaction, a multimodal dataset that maps short movie segments to the induced emotional reactions of viewers in the wild, as expressed through social media commen…
DuplexJail: Spoken Interruption Attacks on Full-Duplex Speech Models
Jaechul Roh, Deepak Chandran, Amir Houmansadr +1
Full-duplex speech models accept user speech while generating responses, making input timing a potential safety concern. We introduce DuplexJail, which delivers fixed, request-inde…
Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild
Trang Nguyen, Sidong Zhang, Shiv Shankar +4
Understanding and forecasting audience reactions to video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reactio…
OnPoint: Offline-to-Online Multi-Level Distillation for Point-Supervised Online Temporal Action Localization
Sakib Reza, Gauri Jagatap, Mohsen Moghaddam +2
Temporal Action Localization (TAL) typically relies on segment annotations or offline access to full videos, limiting scalability and online use. We introduce Point-Supervised Onli…
Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings
Aakriti Agrawal, Gouthaman KV, Rohith Aralikatti +6
Hallucinations in Large Vision-Language Models (LVLMs) remain a persistent challenge, often stemming from inadequate integration of visual information during multimodal reasoning.…
Moment Sampling in Video LLMs for Long-Form Video QA
Mustafa Chasmai, Gauri Jagatap, Gouthaman KV +3
Recent advancements in video large language models (Video LLMs) have significantly advanced the field of video question answering (VideoQA). While existing methods perform well on…