6 papers
Towards Robust Sequential Decomposition for Complex Image Editing
Zilai Zeng, Mingdeng Cao, Zijie Li +5
Recent advances in visual generative models have enabled high-fidelity image editing guided by human instructions. However, these models often struggle with complex instructions in…
Self-Improving Loops for Visual Robotic Planning
Calvin Luo, Zilai Zeng, Mingxi Jia +2
Video generative models trained on expert demonstrations have been utilized as performant text-conditioned visual planners for solving robotic tasks. However, generalization to uns…
Solving New Tasks by Adapting Internet Video Knowledge
Calvin Luo, Zilai Zeng, Yilun Du +1
Video generative models demonstrate great promise in robotics by serving as visual planners or as policy supervisors. When pretrained on internet-scale data, such video models inti…
Text-Aware Diffusion for Policy Learning
Calvin Luo, Mandy He, Zilai Zeng +1
Training an agent to achieve particular goals or perform desired behaviors is often accomplished through reinforcement learning, especially in the absence of expert demonstrations.…
Emergence of Abstract State Representations in Embodied Sequence Modeling
Tian Yun, Zilai Zeng, Kunal Handa +4
Decision making via sequence modeling aims to mimic the success of language models, where actions taken by an embodied agent are modeled as tokens to predict. Despite their promisi…
Goal-Conditioned Predictive Coding for Offline Reinforcement Learning
Zilai Zeng, Ce Zhang, Shijie Wang +1
Recent work has demonstrated the effectiveness of formulating decision making as supervised learning on offline-collected trajectories. Powerful sequence models, such as GPT or BER…