6 papers · 1 filter
VC-Inspector: Advancing Reference-free Evaluation of Video Captions with Factual Analysis
Shubhashis Roy Dipta, Tz-Ying Wu, Subarna Tripathi
We propose VC-Inspector, a lightweight, open-source large multimodal model (LMM) for reference-free evaluation of video captions, with a focus on factual accuracy. Unlike existing…
Harnessing Object Grounding for Time-Sensitive Video Understanding
Tz-Ying Wu, Sharath Nittur Sridhar, Subarna Tripathi
We propose to improve the time-sensitive video understanding (TSV) capability of video large language models (Video-LLMs) with grounded objects (GO). We hypothesize that TSV tasks…
Toward Scalable Video Narration: A Training-free Approach Using Multimodal Large Language Models
Tz-Ying Wu, Tahani Trigui, Sharath Nittur Sridhar +2
In this paper, we introduce VideoNarrator, a novel training-free pipeline designed to generate dense video captions that offer a structured snapshot of video content. These caption…
EASG-Bench: Video Q&A Benchmark with Egocentric Action Scene Graphs
Ivan Rodin, Tz-Ying Wu, Kyle Min +4
We introduce EASG-Bench, a question-answering benchmark for egocentric videos where the question-answering pairs are created from spatio-temporally grounded dynamic scene graphs ca…
Keystep Recognition using Graph Neural Networks
Julia Lee Romero, Kyle Min, Subarna Tripathi +1
We pose keystep recognition as a node classification task, and propose a flexible graph-learning framework for fine-grained keystep recognition that is able to effectively leverage…
Graph-Based Multimodal and Multi-view Alignment for Keystep Recognition
Julia Lee Romero, Kyle Min, Subarna Tripathi +1
Egocentric videos capture scenes from a wearer's viewpoint, resulting in dynamic backgrounds, frequent motion, and occlusions, posing challenges to accurate keystep recognition. We…