12 papers
MotionForesight: Re-purposing Video Models for Future 3D Scene-Flow Prediction
Homanga Bharadhwaj, Yash Jangir
Humans can infer how objects are likely to move from passive observation: a cup may be lifted, a drawer may slide, and a lid may rotate shut. Such predictions expose the physical c…
Web2Grasp: Learning Functional Grasps from Web Images of Hand-Object Interactions
Hongyi Chen, Yunchao Yao, Yufei Ye +8
Functional grasping is essential for enabling dexterous multi-finger robot hands to manipulate objects effectively. Prior work largely focuses on power grasps, which only involve h…
DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy
Sungjae Park, Homanga Bharadhwaj, Shubham Tulsiani
We propose DemoDiffusion, a simple method for enabling robots to perform manipulation tasks by imitating a single human demonstration, without requiring task-specific training or p…
Continual Model-Based Reinforcement Learning with Hypernetworks
Yizhou Huang, Kevin Xie, Homanga Bharadhwaj +1
Effective planning in model-based reinforcement learning (MBRL) and model-predictive control (MPC) relies on the accuracy of the learned dynamics model. In many instances of MBRL a…
Functional Force-Aware Retargeting from Virtual Human Demos to Soft Robot Policies
Uksang Yoo, Mengjia Zhu, Evan Pezent +8
We introduce SoftAct, a framework for teaching soft robot hands to perform human-like manipulation skills by explicitly reasoning about contact forces. Leveraging immersive virtual…
ObjectForesight: Predicting Future 3D Object Trajectories from Human Videos
Rustin Soraki, Homanga Bharadhwaj, Ali Farhadi +1
Humans can effortlessly anticipate how objects might move or change through interaction--imagining a cup being lifted, a knife slicing, or a lid being closed. We aim to endow compu…