5 papers
VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents
Zirui Wang, Junyi Zhang, Jiaxin Ge +9
Modern Vision-Language Models (VLMs) remain poorly characterized in multi-step visual interactions, particularly in how they integrate perception, memory, and action over long hori…
Robo-DM: Data Management For Large Robot Datasets
Kaiyuan Chen, Letian Fu, David Huang +9
Recent results suggest that very large datasets of teleoperated robot demonstrations can be used to train transformer-based models that have the potential to generalize to new scen…
FogROS2-PLR: Probabilistic Latency-Reliability For Cloud Robotics
Kaiyuan Chen, Nan Tian, Christian Juette +4
Cloud robotics enables robots to offload computationally intensive tasks to cloud servers for performance, cost, and ease of management. However, the network and cloud computing in…
In-Context Imitation Learning via Next-Token Prediction
Letian Fu, Huang Huang, Gaurav Datta +5
We explore how to enhance next-token prediction models to perform in-context imitation learning on a real robot, where the robot executes new tasks by interpreting contextual infor…
Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with Reset
Andrew Goldberg, Kavish Kondap, Tianshuang Qiu +7
Generative AI systems have shown impressive capabilities in creating text, code, and images. Inspired by the rich history of research in industrial ''Design for Assembly'', we intr…