25 citations · 67 across the 39 of their papers we have counts for
11 papers · 1 filter
MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction
Jianing Zhang, Chenhao Zheng, Yajun Yang +10
Motion forecasting is central to visual intelligence: agents must anticipate how objects will move in order to plan actions, reason about physical interactions, and synthesize real…
WildDet3D: Scaling Promptable 3D Detection in the Wild
Weikai Huang, Jieyu Zhang, Sijun Li +15
Understanding objects in 3D from a single image is a cornerstone of spatial intelligence. A key step toward this goal is monocular 3D object detection--recovering the extent, locat…
PointArena: Probing Multimodal Grounding Through Language-Guided Pointing
Long Cheng, Jiafei Duan, Yi Ru Wang +12
Pointing serves as a fundamental and intuitive mechanism for grounding language within visual contexts, with applications spanning robotics, assistive technologies, and interactive…
SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models
Arijit Ray, Jiafei Duan, Ellis Brown +9
Reasoning about motion and space is a fundamental cognitive capability that is required by multiple real-world applications. While many studies highlight that large multimodal lang…
Selective Visual Representations Improve Convergence and Generalization for Embodied AI
Ainaz Eftekhar, Kuo-Hao Zeng, Jiafei Duan +3
Embodied AI models often employ off the shelf vision backbones like CLIP to encode their visual observations. Although such general purpose representations encode rich syntactic an…
Read My Mind: A Multi-Modal Dataset for Human Belief Prediction
Jiafei Duan, Samson Yu, Nicholas Tan +2
Understanding human intentions is key to enabling effective and efficient human-robot interaction (HRI) in collaborative settings. To enable developments and evaluation of the abil…
BOSS: A Benchmark for Human Belief Prediction in Object-context Scenarios
Jiafei Duan, Samson Yu, Nicholas Tan +2
Humans with an average level of social cognition can infer the beliefs of others based solely on the nonverbal communication signals (e.g. gaze, gesture, pose and contextual inform…
A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories
Arijit Dasgupta, Jiafei Duan, Marcelo H. Ang +4
Recent work in computer vision and cognitive reasoning has given rise to an increasing adoption of the Violation-of-Expectation (VoE) paradigm in synthetic datasets. Inspired by in…
AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognition
Arijit Dasgupta, Jiafei Duan, Marcelo H. Ang +1
Recent work in cognitive reasoning and computer vision has engendered an increasing popularity for the Violation-of-Expectation (VoE) paradigm in synthetic datasets. Inspired by wo…
PIP: Physical Interaction Prediction via Mental Simulation with Span Selection
Jiafei Duan, Samson Yu, Soujanya Poria +2
Accurate prediction of physical interaction outcomes is a crucial component of human intelligence and is important for safe and efficient deployments of robots in the real world. W…
Actionet: An Interactive End-To-End Platform For Task-Based Data Collection And Augmentation In 3D Environment
Jiafei Duan, Samson Yu, Hui Li Tan +1
The problem of task planning for artificial agents remains largely unsolved. While there has been increasing interest in data-driven approaches for the study of task planning for a…