62 citations · 71 across the 11 of their papers we have counts for
13 papers
Box-WAM: Box Trajectories as Compact State Representations for World Action Models
Qinxi Yu, Wenzhou Ding, Gloria X. Wang +2
World action models (WAMs) are a category of policy models that ex- plicitly predict both actions and future world states. They have been shown to outperform action-only models acr…
Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos
Albert J. Zhai, Kuo-Hao Zeng, Jiasen Lu +3
The ability to learn manipulation skills by watching videos of humans has the potential to unlock a new source of highly scalable data for robot learning. Here, we tackle prehensil…
Human-like Navigation in a World Built for Humans
Bhargav Chandaka, Gloria X. Wang, Haozhe Chen +3
When navigating in a man-made environment they haven't visited before--like an office building--humans employ behaviors such as reading signs and asking others for directions. Thes…
Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video
David Yifan Yao, Albert J. Zhai, Shenlong Wang
This paper presents a unified approach to understanding dynamic scenes from casual videos. Large pretrained vision foundation models, such as vision-language, video depth predictio…
AutoVFX: Physically Realistic Video Editing from Natural Language Instructions
Hao-Yu Hsu, Zhi-Hao Lin, Albert Zhai +2
Modern visual effects (VFX) software has made it possible for skilled artists to create imagery of virtually anything. However, the creation process remains laborious, complex, and…
CropCraft: Complete Structural Characterization of Crop Plants From Images
Albert J. Zhai, Xinlei Wang, Kaiyuan Li +6
The ability to automatically build 3D digital twins of plants from images has countless applications in agriculture, environmental science, robotics, and other fields. However, cur…
EarthGen: Generating the World from Top-Down Views
Ansh Sharma, Albert Xiao, Praneet Rathi +4
In this work, we present a novel method for extensive multi-scale generative terrain modeling. At the core of our model is a cascade of superresolution diffusion models that can be…
Physical Property Understanding from Language-Embedded Feature Fields
Albert J. Zhai, Yuan Shen, Emily Y. Chen +5
Can computers perceive the physical properties of objects solely through vision? Research in cognitive science and vision science has shown that humans excel at identifying materia…
On the Overconfidence Problem in Semantic 3D Mapping
Joao Marcos Correia Marques, Albert Zhai, Shenlong Wang +1
Semantic 3D mapping, the process of fusing depth and image segmentation information between multiple views to build 3D maps annotated with object classes in real-time, is a recent…
Sim-on-Wheels: Physical World in the Loop Simulation for Self-Driving
Yuan Shen, Bhargav Chandaka, Zhi-hao Lin +4
We present Sim-on-Wheels, a safe, realistic, and vehicle-in-loop framework to test autonomous vehicles' performance in the real world under safety-critical scenarios. Sim-on-wheels…
PEANUT: Predicting and Navigating to Unseen Targets
Albert J. Zhai, Shenlong Wang
Efficient ObjectGoal navigation (ObjectNav) in novel environments requires an understanding of the spatial and semantic regularities in environment layouts. In this work, we presen…
Learning Visually Guided Latent Actions for Assistive Teleoperation
Siddharth Karamcheti, Albert J. Zhai, Dylan P. Losey +1
It is challenging for humans -- particularly those living with physical disabilities -- to control high-dimensional, dexterous robots. Prior work explores learning embedding functi…
Using Deep Space Climate Observatory Measurements to Study the Earth as An Exoplanet
Jonathan H. Jiang, Albert J. Zhai, Jay Herman +7
Even though it was not designed as an exoplanetary research mission, the Deep Space Climate Observatory (DSCOVR) has been opportunistically used for a novel experiment, in which Ea…