84 citations · 262 across the 47 of their papers we have counts for
51 papers
Flow Matching in Feature Space for Stochastic World Modeling
Francois Porcher, Nicolas Carion, Karteek Alahari +1
World modeling requires forecasting uncertain futures while preserving information useful for downstream perception. Existing visual world models often struggle to satisfy both goa…
PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interaction
Shizhe Chen, Paul Pacaud, Cordelia Schmid
Vision-Language-Action (VLA) models have shown strong potential for general-purpose robotic manipulation by leveraging large pretrained vision-language backbones. However, most exi…
MAGICIAN: Efficient Long-Term Planning with Imagined Gaussians for Active Mapping
Shiyao Li, Antoine Guédon, Shizhe Chen +1
Active mapping aims to determine how an agent should move to efficiently reconstruct unknown environments. Most existing approaches rely on greedy next-best-view prediction, result…
HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching
Zerui Chen, Rolandos Alexandros Potamias, Shizhe Chen +3
Generating realistic 3D hand-object interactions (HOI) is a fundamental challenge in computer vision and robotics, requiring both temporal coherence and high-fidelity physical plau…
Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models
Paul Pacaud, Ricardo Garcia, Shizhe Chen +1
Robust robotic manipulation requires reliable failure detection and recovery. Although recent Vision-Language Models (VLMs) show promise in robot failure detection, their generaliz…
FOM-Nav: Frontier-Object Maps for Object Goal Navigation
Thomas Chabal, Shizhe Chen, Jean Ponce +1
This paper addresses the Object Goal Navigation problem, where a robot must efficiently find a target object in an unknown environment. Existing implicit memory-based methods strug…
FACap: A Large-scale Fashion Dataset for Fine-grained Composed Image Retrieval
François Gardères, Shizhe Chen, Camille-Sovanneary Gauthier +1
The composed image retrieval (CIR) task is to retrieve target images given a reference image and a modification text. Recent methods for CIR leverage large pretrained vision-langua…
Gondola: Grounded Vision Language Planning for Robotic Manipulation
Shizhe Chen, Ricardo Garcia, Paul Pacaud +1
Vision-language-action (VLA) models have shown promising progress in robotic manipulation. However, directly mapping visual observations and language instructions to low-level acti…
ComposeAnything: Composite Object Priors for Text-to-Image Generation
Zeeshan Khan, Shizhe Chen, Cordelia Schmid
Generating images from text involving complex and novel object arrangements remains a significant challenge for current text-to-image (T2I) models. Although prior layout-based meth…
HORT: Monocular Hand-held Objects Reconstruction with Transformers
Zerui Chen, Rolandos Alexandros Potamias, Shizhe Chen +1
Reconstructing hand-held objects in 3D from monocular images remains a significant challenge in computer vision. Most existing approaches rely on implicit 3D representations, which…
NextBestPath: Efficient 3D Mapping of Unseen Environments
Shiyao Li, Antoine Guédon, Clémentin Boittiaux +2
This work addresses the problem of active 3D mapping, where an agent must find an efficient trajectory to exhaustively reconstruct a new scene. Previous approaches mainly predict t…
Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policy
Ricardo Garcia, Shizhe Chen, Cordelia Schmid
Generalizing language-conditioned robotic policies to new tasks remains a significant challenge, hampered by the lack of suitable simulation benchmarks. In this paper, we address t…
Think-Program-reCtify: 3D Situated Reasoning with Large Language Models
Qingrong He, Kejun Lin, Shizhe Chen +2
This work addresses the 3D situated reasoning task which aims to answer questions given egocentric observations in a 3D environment. The task remains challenging as it requires com…
ViViDex: Learning Vision-based Dexterous Manipulation from Human Videos
Zerui Chen, Shizhe Chen, Etienne Arlaud +2
In this work, we aim to learn a unified vision-based policy for multi-fingered robot hands to manipulate a variety of objects in diverse poses. Though prior work has shown benefits…
SUGAR: Pre-training 3D Visual Representations for Robotics
Shizhe Chen, Ricardo Garcia, Ivan Laptev +1
Learning generalizable visual representations from Internet data has yielded promising results for robotics. Yet, prevailing approaches focus on pre-training 2D representations, be…
PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation
Shizhe Chen, Ricardo Garcia, Cordelia Schmid +1
The ability for robots to comprehend and execute manipulation tasks based on natural language instructions is a long-term goal in robotics. The dominant approaches for language-gui…
Explore and Tell: Embodied Visual Captioning in 3D Environments
Anwen Hu, Shizhe Chen, Liang Zhang +1
While current visual captioning models have achieved impressive performance, they often assume that the image is well-captured and provides a complete view of the scene. In real-wo…
Object Goal Navigation with Recursive Implicit Maps
Shizhe Chen, Thomas Chabal, Ivan Laptev +1
Object goal navigation aims to navigate an agent to locations of a given object category in unseen environments. Classical methods explicitly build maps of environments and require…
Robust Visual Sim-to-Real Transfer for Robotic Manipulation
Ricardo Garcia, Robin Strudel, Shizhe Chen +3
Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained po…
InfoMetIC: An Informative Metric for Reference-free Image Caption Evaluation
Anwen Hu, Shizhe Chen, Liang Zhang +1
Automatic image captioning evaluation is critical for benchmarking and promoting advances in image captioning research. Existing metrics only provide a single score to measure capt…
gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction
Zerui Chen, Shizhe Chen, Cordelia Schmid +1
Signed distance functions (SDFs) is an attractive framework that has recently shown promising results for 3D shape reconstruction from images. SDFs seamlessly generalize to differe…
TeViS:Translating Text Synopses to Video Storyboards
Xu Gu, Yuchong Sun, Feiyue Ni +5
A video storyboard is a roadmap for video creation which consists of shot-by-shot images to visualize key plots in a text synopsis. Creating video storyboards, however, remains cha…
Language Conditioned Spatial Relation Reasoning for 3D Object Grounding
Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi +2
Localizing objects in 3D scenes based on natural language requires understanding and reasoning about spatial relations. In particular, it is often crucial to distinguish similar ob…
Instruction-driven history-aware policies for robotic manipulations
Pierre-Louis Guhur, Shizhe Chen, Ricardo Garcia +3
In human environments, robots are expected to accomplish a variety of manipulation tasks given simple natural language instructions. Yet, robotic manipulation is extremely challeng…