9 citations · 14 across the 12 of their papers we have counts for
9 papers · 1 filter
Physical Object Understanding with a Physically Controllable World Model
Rahul Venkatesh, Klemen Kotar, Lilian Naing Chen +9
A central challenge in visual intelligence is learning the physical structure of scenes from raw videos: how regions form objects and the laws that govern their interactions. Solvi…
Unified 3D Scene Understanding Through Physical World Modeling
Wanhee Lee, Klemen Kotar, Rahul Mysore Venkatesh +4
Understanding 3D scenes requires flexible combinations of visual reasoning tasks, including depth estimation, novel view synthesis, and object manipulation, all of which are essent…
Memory-Efficient Transfer Learning with Fading Side Networks via Masked Dual Path Distillation
Yutong Zhang, Jiaxin Chen, Honglin Chen +5
Memory-efficient transfer learning (METL) approaches have recently achieved promising performance in adapting pre-trained models to downstream tasks. They avoid applying gradient b…
World Modeling with Probabilistic Structure Integration
Klemen Kotar, Wanhee Lee, Rahul Venkatesh +13
We present Probabilistic Structure Integration (PSI), a system for learning richly controllable and flexibly promptable world models from data. PSI consists of a three-step cycle.…
Discovering and using Spelke segments
Rahul Venkatesh, Klemen Kotar, Lilian Naing Chen +10
Segments in computer vision are often defined by semantic considerations and are highly dependent on category-specific conventions. In contrast, developmental psychology suggests t…
3D Scene Understanding Through Local Random Access Sequence Modeling
Wanhee Lee, Klemen Kotar, Rahul Mysore Venkatesh +4
3D scene understanding from single images is a pivotal problem in computer vision with numerous downstream applications in graphics, augmented reality, and robotics. While diffusio…
Understanding Physical Dynamics with Counterfactual World Modeling
Rahul Venkatesh, Honglin Chen, Kevin Feigelis +9
The ability to understand physical dynamics is critical for agents to act in the world. Here, we use Counterfactual World Modeling (CWM) to extract vision structures for dynamics u…
Unifying (Machine) Vision via Counterfactual World Modeling
Daniel M. Bear, Kevin Feigelis, Honglin Chen +5
Leading approaches in machine vision employ different architectures for different tasks, trained on costly task-specific labeled datasets. This complexity has held back progress in…
Unsupervised Segmentation in Real-World Images via Spelke Object Inference
Honglin Chen, Rahul Venkatesh, Yoni Friedman +4
Self-supervised, category-agnostic segmentation of real-world images is a challenging open problem in computer vision. Here, we show how to learn static grouping priors from motion…