3 citations · 5 across the 5 of their papers we have counts for
8 papers
Meaningful Moments: A Corpus of VLM-Derived Temporal Importance Labels for Video Action Recognition
Cameron Keith, SouYoung Jin
Not every moment in a video is equally useful for recognizing an action. A few decisive moments often carry the evidence: the instant an object moves, the peak of a dive, or one ob…
StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description
Seung Hyun Hahm, Minh T. Dinh, SouYoung Jin
Long-form audio description (AD) requires more than describing visible actions: it must preserve characters, events, relationships, and story context across scenes so that blind an…
Learning Video Dynamics with Predictive Differentiable Rendering
Yujin Tang, Tian Zhou, Xin Lin +5
How to accurately predict a high-fidelity future world? While the visual world is inherently continuous, existing deterministic video prediction models operate in discrete pixel sp…
Beyond Final Answers: CRYSTAL Benchmark for Transparent Multimodal Reasoning Evaluation
Wayner Barrios, SouYoung Jin
We introduce CRYSTAL (Clear Reasoning via Yielded Steps, Traceability, and Logic), a diagnostic benchmark with 6,372 instances that evaluates multimodal reasoning through verifiabl…
MoDA: Modulation Adapter for Fine-Grained Visual Grounding in Instructional MLLMs
Wayner Barrios, Andrés Villa, Juan León Alcázar +2
Multimodal Large Language Models (MLLMs) have achieved remarkable success in instruction-following tasks by integrating pretrained visual encoders with large language models (LLMs)…
A Foundation Model for Wearable Movement Data in Mental Health Research
Franklin Y. Ruan, Aiwei Zhang, Jenny Y. Oh +2
Wearable movement data is collected by nearly all commercially available smartwatches and is a valuable resource for mental health research, reflecting fine-grained temporal behavi…
Multi-layer Learnable Attention Mask for Multimodal Tasks
Wayner Barrios, SouYoung Jin
While the Self-Attention mechanism in the Transformer model has proven to be effective in many domains, we observe that it is less effective in more diverse settings (e.g. multimod…
FT2TF: First-Person Statement Text-To-Talking Face Generation
Xingjian Diao, Ming Cheng, Wayner Barrios +1
Talking face generation has gained immense popularity in the computer vision community, with various applications including AR, VR, teleconferencing, digital assistants, and avatar…