103 citations · 256 across the 18 of their papers we have counts for
18 papers
SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile Manipulation
Ruisen Tu, Arth Shukla, Sohyun Yoo +5
Vision-Language-Action (VLA) models show promise for robotic control, yet performance in complex household environments remains sub-optimal. Mobile manipulation requires reasoning…
Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representations
Shresth Grover, Akshay Gopalkrishnan, Bo Ai +3
Vision-language-action (VLA) models finetuned from vision-language models (VLMs) hold the promise of leveraging rich pretrained representations to build generalist robots across di…
ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI
Stone Tao, Fanbo Xiang, Arth Shukla +20
Simulation has enabled unprecedented compute-scalable approaches to robot learning. However, many existing simulation frameworks typically support a narrow range of scenes/tasks an…
Evaluating Real-World Robot Manipulation Policies in Simulation
Xuanlin Li, Kyle Hsu, Jiayuan Gu +13
The field of robotics has made significant advances towards generalist robot manipulation policies. However, real-world evaluation of such policies is not scalable and faces reprod…
PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimation
Yuchen Zhou, Jiayuan Gu, Xuanlin Li +3
Open-world 3D part segmentation is pivotal in diverse applications such as robotics and AR/VR. Traditional supervised methods often grapple with limited 3D data availability and st…
Unleashing the Creative Mind: Language Model As Hierarchical Policy For Improved Exploration on Challenging Problem Solving
Zhan Ling, Yunhao Fang, Xuanlin Li +5
Large Language Models (LLMs) have achieved tremendous progress, yet they still often struggle with challenging reasoning problems. Current approaches address this challenge by samp…
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, thi…
Reparameterized Policy Learning for Multimodal Trajectory Optimization
Zhiao Huang, Litian Liang, Zhan Ling +3
We investigate the challenge of parametrizing policies for reinforcement learning (RL) in high-dimensional continuous action spaces. Our objective is to develop a multimodal policy…
Distilling Large Vision-Language Model with Out-of-Distribution Generalizability
Xuanlin Li, Yunhao Fang, Minghua Liu +3
Large vision-language models have achieved outstanding performance, but their size and computational requirements make their deployment on resource-constrained devices and time-sen…
OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding
Minghua Liu, Ruoxi Shi, Kaiming Kuang +6
We introduce OpenShape, a method for learning multi-modal joint representations of text, image, and point clouds. We adopt the commonly used multi-modal contrastive learning framew…
On the Efficacy of 3D Point Cloud Reinforcement Learning
Zhan Ling, Yunchao Yao, Xuanlin Li +1
Recent studies on visual reinforcement learning (visual RL) have explored the use of 3D visual representations. However, none of these work has systematically compared the efficacy…
Deductive Verification of Chain-of-Thought Reasoning
Zhan Ling, Yunhao Fang, Xuanlin Li +4
Large Language Models (LLMs) significantly benefit from Chain-of-Thought (CoT) prompting in performing various reasoning tasks. While CoT allows models to produce more comprehensiv…
ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
Jiayuan Gu, Fanbo Xiang, Xuanlin Li +12
Generalizable manipulation skills, which can be composed to tackle long-horizon and complex daily chores, are one of the cornerstones of Embodied AI. However, existing benchmarks,…
Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds
Minghua Liu, Xuanlin Li, Zhan Ling +2
We study how choices of input point cloud coordinate frames impact learning of manipulation skills from 3D point clouds. There exist a variety of coordinate frame choices to normal…
Improving Policy Optimization with Generalist-Specialist Learning
Zhiwei Jia, Xuanlin Li, Zhan Ling +3
Generalization in deep reinforcement learning over unseen environment variations usually requires policy learning over a large set of diverse training variations. We empirically ob…
Discovering Non-monotonic Autoregressive Orderings with Variational Inference
Xuanlin Li, Brandon Trabucco, Dong Huk Park +4
The predominant approach for language modeling is to process sequences from left to right, but this eliminates a source of information: the order by which the sequence was generate…
ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
Tongzhou Mu, Zhan Ling, Fanbo Xiang +6
Object manipulation from 3D visual inputs poses many challenges on building generalizable perception and policy models. However, 3D assets in existing benchmarks mostly lack the di…
Regularization Matters in Policy Optimization
Zhuang Liu, Xuanlin Li, Bingyi Kang +1
Deep Reinforcement Learning (Deep RL) has been receiving increasingly more attention thanks to its encouraging performance on a variety of control tasks. Yet, conventional regulari…