9 citations · 24 across the 30 of their papers we have counts for
34 papers
BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender
Yolo Y. Tang, Daiki Shimada, Jiayue Meng +14
Multimodal agents can create complex videos in software such as Blender by writing code instead of using diffusion models. Yet video understanding benchmarks still evaluate models…
StreamScout: Learning When to Look Deeper for Streaming Video Understanding
Ce Zhang, Jing Bi, Jinxi He +9
Streaming video understanding requires answering questions that arrive at arbitrary moments over an unbounded video stream. Existing systems primarily focus on what to retain in a…
AdaTurn: Budget-Aware Test-Time Scaling for Active Visual Perception Agents
Susan Liang, Chao Huang, Filippos Bellos +3
Active visual agents solve fine-grained image tasks by interleaving reasoning with image-grounding actions across multiple turns. However, deployment-time rollout budgets are rarel…
SurgAtlas: A Large-Scale Surgical Video-Language Dataset with 2,391 Hours of Open and Minimally Invasive Surgery
Filippos Bellos, Andre S. Gala-Garza, Miaowei Wang +8
We introduce SurgAtlas, the largest surgical video-language dataset to date, comprising 15,291 videos (2,391 hours) spanning 18 surgical specialties and over 5,000 procedure types,…
TDMM-LM: Bridging Facial Understanding and Animation via Language Models
Luchuan Song, Pinxin Liu, Haiyang Liu +7
Text-guided human body animation has advanced rapidly, yet facial animation lags due to the scarcity of well-annotated, text-paired facial corpora. To close this gap, we leverage f…
Omni-Judge: Can Omni-LLMs Serve as Human-Aligned Judges for Text-Conditioned Audio-Video Generation?
Susan Liang, Chao Huang, Filippos Bellos +7
State-of-the-art text-to-video generation models such as Sora 2 and Veo 3 can now produce high-fidelity videos with synchronized audio directly from a textual prompt, marking a new…
VisualActBench: Can VLMs See and Act like a Human?
Daoan Zhang, Pai Liu, Xiaofei Zhou +6
Vision-Language Models (VLMs) have achieved impressive progress in perceiving and describing visual environments. However, their ability to proactively reason and act based solely…
Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Rumination
Yolo Y. Tang, Daiki Shimada, Hang Hua +4
Understanding text-rich videos requires reading small, transient textual cues that often demand repeated inspection. Yet most video QA models rely on single-pass perception over fi…
Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models
Yolo Y. Tang, Jing Bi, Pinxin Liu +24
Video understanding represents the most challenging frontier in computer vision, requiring models to reason about complex spatiotemporal relationships, long-term dependencies, and…
When to Think and When to Look: Uncertainty-Guided Lookback
Jing Bi, Filippos Bellos, Junjia Guo +8
Test-time thinking (that is, generating explicit intermediate reasoning chains) is known to boost performance in large language models and has recently shown strong gains for large…
Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
Jing Bi, Guangyu Sun, Ali Vosoughi +2
Multimodal large language models (MLLMs) that integrate visual and textual reasoning leverage chain-of-thought (CoT) prompting to tackle complex visual tasks, yet continue to exhib…
What to Do Next? Memorizing skills from Egocentric Instructional Video
Jing Bi, Chenliang Xu
Learning to perform activities through demonstration requires extracting meaningful information about the environment from observations. In this research, we investigate the challe…
ACTLLM: Action Consistency Tuned Large Language Model
Jing Bi, Lianggong Bruce Wen, Zhang Liu +1
This paper introduces ACTLLM (Action Consistency Tuned Large Language Model), a novel approach for robot manipulation in dynamic environments. Traditional vision-based systems ofte…
Can Sound Replace Vision in LLaVA With Token Substitution?
Ali Vosoughi, Jing Bi, Pinxin Liu +2
What happens when we push audio-visual alignment to its absolute limits? To systematically investigate this question, we needed datasets with granular alignment quality annotations…
ZeroSep: Separate Anything in Audio with Zero Training
Chao Huang, Yuesheng Ma, Junxuan Huang +6
Audio source separation is fundamental for machines to understand complex acoustic environments and underpins numerous audio applications. Current supervised deep learning approach…
: Generating Instructional Illustrations via Text-Conditioned Diffusion
Jing Bi, Pinxin Liu, Ali Vosoughi +3
The effective communication of procedural knowledge remains a significant challenge in natural language processing (NLP), as purely textual instructions often fail to convey comple…
MMPerspective: Do MLLMs Understand Perspective? A Comprehensive Benchmark for Perspective Perception, Reasoning, and Robustness
Yolo Y. Tang, Pinxin Liu, Zhangyun Tan +11
Understanding perspective is fundamental to human visual perception, yet the extent to which multimodal large language models (MLLMs) internalize perspective geometry remains uncle…
Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting
Yunlong Tang, Jing Bi, Chao Huang +16
We present CAT-V (Caption AnyThing in Video), a training-free framework for fine-grained object-centric video captioning that enables detailed descriptions of user-selected objects…
Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1)
Jing Bi, Susan Liang, Xiaofei Zhou +16
Reasoning is central to human intelligence, enabling structured problem-solving across diverse tasks. Recent advances in large language models (LLMs) have greatly enhanced their re…
VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
Jing Bi, Junjia Guo, Susan Liang +8
Visual reasoning is central to human cognition, enabling individuals to interpret and abstractly understand their environment. Although recent Multimodal Large Language Models (MLL…
Generative AI for Cel-Animation: A Survey
Yolo Y. Tang, Junjia Guo, Pinxin Liu +14
Traditional Celluloid (Cel) Animation production pipeline encompasses multiple essential steps, including storyboarding, layout design, keyframe animation, inbetweening, and colori…
Enhancing the Reasoning Capabilities of Small Language Models via Solution Guidance Fine-Tuning
Jing Bi, Yuting Wu, Weiwei Xing +1
Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks. Advances in prompt engineering and fine-tuning techniques have further enhanced…
Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach
Jing Bi, Junjia Guo, Yunlong Tang +3
Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated remarkable progress in visual understanding. This impressive leap raises a compelling question: ho…
VidComposition: Can MLLMs Analyze Compositions in Compiled Videos?
Yolo Y. Tang, Junjia Guo, Hang Hua +9
The advancement of Multimodal Large Language Models (MLLMs) has enabled significant progress in multimodal understanding, expanding their capacity to analyze video content. However…