4 citations · 5 across the 8 of their papers we have counts for
11 papers · 1 filter
Vision-Language-Action Autonomous Driving Agent with Language-based Memory
Kai Yan, Xiangyu Chen, Yulong Cao +9
Vision-Language-Action (VLA) foundation models have recently emerged as one of the prevailing solutions for autonomous driving, as they can utilize knowledge acquired during vision…
Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies
Hui Ren, Lei Fan, Henry Pao +5
Answering questions about long videos often requires connecting events involving the same objects across hours or days. Chronological descriptions and text-derived entities can lea…
MoVISA: Multi-Token Reasoning for Video Object Segmentation
Ruining Zhao, Ho Kei Cheng, Alexander G Schwing
Recent advances in video object segmentation with Multimodal Large Language Model (MLLM) reasoning have demonstrated the effectiveness of using a single textual token, such as SEG,…
SLIDEFORGE: An LLM Agent for Controllable Editing of Slides as Structured Artifacts
Haozhen Zheng, Fulin Wang, Tianhu Xiong +6
Current AI agents compellingly describe slides. However, AI-assisted slide editing requires more than understanding: the output must retain layout, style, component structure, and…
Seeing Without Eyes: 4D Human-Scene Understanding from Wearable IMUs
Hao-Yu Hsu, Tianhang Cheng, Jing Wen +2
Understanding human activities and their surrounding environments typically relies on visual perception, yet cameras pose persistent challenges in privacy, safety, energy efficienc…
T-REN: Learning Text-Aligned Region Tokens Improves Dense Vision-Language Alignment and Scalability
Savya Khosla, Sethuraman T, Aryan Chadha +2
Despite recent progress, vision-language encoders struggle with two core limitations: (1) weak alignment between language and dense vision features, which hurts tasks like open-voc…
SimpliHuMoN: Simplifying Human Motion Prediction
Aadya Agrawal, Alexander Schwing
Human motion prediction combines the tasks of trajectory forecasting and human pose prediction. For each of the two tasks, specialized models have been developed. Combining these m…
Spatio-Temporal LLM: Reasoning about Environments and Actions
Haozhen Zheng, Beitong Tian, Mingyuan Wu +3
Despite significant recent progress of Multimodal Large Language Models (MLLMs), current MLLMs are challenged by "spatio-temporal" prompts, i.e., prompts that refer to 1) the entir…
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis
Ho Kei Cheng, Masato Ishii, Akio Hayakawa +3
We propose to synthesize high-quality and synchronized audio, given video and optional text conditions, using a novel multimodal joint training framework MMAudio. In contrast to si…
Putting the Object Back into Video Object Segmentation
Ho Kei Cheng, Seoung Wug Oh, Brian Price +2
We present Cutie, a video object segmentation (VOS) network with object-level memory reading, which puts the object representation from memory back into the video object segmentati…
Tracking Anything with Decoupled Video Segmentation
Ho Kei Cheng, Seoung Wug Oh, Brian Price +2
Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary set…