11 papers
UNISON: A Unified Framework for Learning from Scored Bags
Ofek Kirshenboim, Or Shkuri, Shani Finkelshtein +1
We introduce UNISON, a unified framework that bridges multiple instance learning and recommendation systems by treating both as instances of learning from scored bag collections of…
PReD: An LLM-based Foundation Multimodal Model for Electromagnetic Perception, Recognition, and Decision
Zehua Han, Jing Xiao, Yiqi Duan +13
Multimodal Large Language Models have demonstrated powerful cross-modal understanding and reasoning capabilities in general domains. However, in the electromagnetic (EM) domain, th…
Towards Cross-View Point Correspondence in Vision-Language Models
Yipu Wang, Yuheng Ji, Yuyang Liu +10
Cross-view correspondence is a fundamental capability for spatial understanding and embodied AI. However, it is still far from being realized in Vision-Language Models (VLMs), espe…
Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
Shuanghao Bai, Wenxuan Song, Jiayi Chen +14
Recent advances in vision, language, and multimodal learning have significantly accelerated progress in robotic foundation models, with robotic manipulation remaining one of the mo…
Scaling Up AI-Generated Image Detection with Generator-Aware Prototypes
Ziheng Qin, Yuheng Ji, Renshuai Tao +4
The pursuit of a universal AI-generated image (AIGI) detector often relies on aggregating data from numerous generators to improve generalization. However, this paper identifies a…
Towards a Unified Understanding of Robot Manipulation: A Comprehensive Survey
Shuanghao Bai, Wenxuan Song, Jiayi Chen +15
Embodied intelligence has witnessed remarkable progress in recent years, driven by advances in computer vision, natural language processing, and the rise of large-scale multimodal…
MathSticks: A Benchmark for Visual Symbolic Compositional Reasoning with Matchstick Puzzles
Yuheng Ji, Huajie Tan, Cheng Chi +8
We introduce \textsc{MathSticks}, a benchmark for Visual Symbolic Compositional Reasoning (VSCR), which unifies visual perception, symbolic manipulation, and arithmetic consistency…
VisualTrans: A Benchmark for Real-World Visual Transformation Reasoning
Yuheng Ji, Yipu Wang, Yuyang Liu +5
Visual transformation reasoning (VTR) is a vital cognitive capability that empowers intelligent agents to understand dynamic scenes, model causal relationships, and predict future…
Alleviating Performance Disparity in Adversarial Spatiotemporal Graph Learning Under Zero-Inflated Distribution
Songran Bai, Yuheng Ji, Yue Liu +3
Spatiotemporal Graph Learning (SGL) under Zero-Inflated Distribution (ZID) is crucial for urban risk management tasks, including crime prediction and traffic accident profiling. Ho…
RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
Yuheng Ji, Huajie Tan, Jiayu Shi +14
Recent advancements in Multimodal Large Language Models (MLLMs) have shown remarkable capabilities across various multimodal contexts. However, their application in robotic scenari…
Can Foundation Models Watch, Talk and Guide You Step by Step to Make a Cake?
Yuwei Bao, Keunwoo Peter Yu, Yichi Zhang +6
Despite tremendous advances in AI, it remains a significant challenge to develop interactive task guidance systems that can offer situated, personalized guidance and assist humans…