44 citations · 165 across the 16 of their papers we have counts for
16 papers
Vesta: A Generalist Embodied Reasoning Model
Johan Bjorck, Zhiqi Li, Yunze Man +29
Robots operating in open-world environments must seamlessly integrate localization, spatial reasoning, navigation, and long-horizon planning. While specialist models excel at indiv…
See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents
Siyi Chen, Xiaoyan Zhang, Meng Wu +7
Language-model agents build internal representations of the information they observe and the reasoning they perform. Sharing these representations offers a way to communicate both…
VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation
Siyi Chen, Hugo Hadfield, Alex Zook +9
Open-vocabulary long-horizon manipulation requires robots to reason over flexible instructions and complex multi-object scenes while adaptively planning, executing, monitoring, and…
RVT: Robotic View Transformer for 3D Object Manipulation
Ankit Goyal, Jie Xu, Yijie Guo +3
For 3D object manipulation, methods that build an explicit 3D representation perform better than those relying only on camera images. But using explicit 3D representations like vox…
Partial-View Object View Synthesis via Filtered Inversion
Fan-Yun Sun, Jonathan Tremblay, Valts Blukis +11
We propose Filtering Inversion (FINV), a learning framework and optimization process that predicts a renderable 3D object representation from one or few partial views. FINV address…
TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation
Taeyeop Lee, Jonathan Tremblay, Valts Blukis +6
Test-time adaptation methods have been gaining attention recently as a practical solution for addressing source-to-target domain gaps by gradually updating the model without requir…
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Bowen Wen, Jonathan Tremblay, Valts Blukis +6
We present a near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence, while simultaneously performing neural 3D reconstruction of the obj…
One-Shot Neural Fields for 3D Object Understanding
Valts Blukis, Taeyeop Lee, Jonathan Tremblay +5
We present a unified and compact scene representation for robotics, where each object in the scene is depicted by a latent code capturing geometry and appearance. This representati…
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Ishika Singh, Valts Blukis, Arsalan Mousavian +6
Task planning can require defining myriad domain knowledge about the world in which a robot needs to act. To ameliorate that effort, large language models (LLMs) can be used to sco…
Correcting Robot Plans with Natural Language Feedback
Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis +5
When humans design cost or goal specifications for robots, they often produce specifications that are ambiguous, underspecified, or beyond planners' ability to solve. In these case…
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution
Valts Blukis, Chris Paxton, Dieter Fox +2
Natural language provides an accessible and expressive interface to specify long-term tasks for robotic agents. However, non-experts are likely to specify such tasks with high-leve…
Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following
Valts Blukis, Ross A. Knepper, Yoav Artzi
We study the problem of learning a robot policy to follow natural language instructions that can be easily extended to reason about new objects. We introduce a few-shot language-co…
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
Valts Blukis, Yannick Terme, Eyvind Niklasson +2
We propose a joint simulation and real-world learning framework for mapping navigation instructions and raw first-person observations to continuous control. Our model estimates the…
Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction
Valts Blukis, Dipendra Misra, Ross A. Knepper +1
We propose an approach for mapping natural language instructions and raw observations to continuous control of a quadcopter drone. Our model predicts interpretable position-visitat…
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction
Dipendra Misra, Andrew Bennett, Valts Blukis +3
We propose to decompose instruction execution to goal prediction and action generation. We design a model that maps raw visual observations to goals using LINGUNET, a language-cond…
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning
Valts Blukis, Nataly Brukhim, Andrew Bennett +2
We introduce a method for following high-level navigation instructions by mapping directly from images, instructions and pose estimates to continuous low-level velocity commands fo…