4 papers
AnyUser: Translating Sketched User Intent into Domestic Robots
Songyuan Yang, Huibin Tan, Kailun Yang +2
We introduce AnyUser, a unified robotic instruction system for intuitive domestic task instruction via free-form sketches on camera images, optionally with language. AnyUser interp…
Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning
Songyuan Yang, Weijiang Yu, Jilin Ma +5
Video reasoning has advanced with large multimodal models (LMMs), yet their inference is often a single pass that returns an answer without verifying whether the reasoning is evide…
Graph-to-Frame RAG: Visual-Space Knowledge Fusion for Training-Free and Auditable Video Reasoning
Songyuan Yang, Weijiang Yu, Ziyu Liu +4
When video reasoning requires external knowledge, many systems with large multimodal models (LMMs) adopt retrieval augmentation to supply the missing context. Appending textual or…
Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks
Mei Chee Leong, Ying Gu, Hui Li Tan +2
Frontier Multimodal Large Language Models (MLLMs) exhibit remarkable capabilities in Visual-Language Comprehension (VLC) tasks. However, they are often deployed as zero-shot soluti…